AWS SNS FIFO Sink
Provided by: "Apache Software Foundation"
Support Level for this Kamelet is: "Stable"
Publishes each message to an Amazon SNS FIFO topic, which delivers the messages of a group in order, and without duplicates, to the SQS FIFO queues subscribed to it.
Use it to fan out ordered updates, such as the changes of one order or one account, to several FIFO queues at once. For a standard topic use aws-sns-sink.
Input: the body becomes the text of the notification, as text. The optional group header (or ce-group) is the message group ID, such as a customer ID: the messages of one group are delivered in order. Without it, each message is a group of its own. The optional subject header (or ce-subject) is the subject; it takes precedence over the subject option. The other headers of the message, except the Camel ones, are sent as SNS message attributes.
Output: the message is unchanged. The CamelAwsSnsMessageId header gives the ID of the published message.
Good to know: the name of the topic ends in .fifo. With contentBasedDeduplication false, the default, each message gets a unique deduplication ID, so SNS drops only a message sent twice by a retry; set it to true to drop messages with the same body, which the topic must allow. With messageStructure set to json, the body is a JSON object with a text for each kind of subscriber. serverSideEncryptionEnabled and kmsMasterKeyId apply only to a topic that autoCreateTopic creates. To reach AWS through an HTTP proxy, set proxyHost and proxyPort, and set proxyProtocol to HTTP for a proxy that takes plain HTTP, such as Squid. Headers that start with CamelAwsSns are removed from the message first, so they cannot change what the Kamelet does.
Configuration Options
The following table summarizes the configuration options available for the aws-sns-fifo-sink Kamelet:
| Property | Name | Description | Type | Default | Example |
|---|---|---|---|---|---|
AWS Region | Required The AWS region of the topic, as a region code such as Enum values: * ap-south-1 * eu-south-1 * us-gov-east-1 * me-central-1 * ca-central-1 * eu-central-1 * us-iso-west-1 * us-west-1 * us-west-2 * af-south-1 * eu-north-1 * eu-west-3 * eu-west-2 * eu-west-1 * ap-northeast-3 * ap-northeast-2 * ap-northeast-1 * me-south-1 * sa-east-1 * ap-east-1 * cn-north-1 * us-gov-west-1 * ap-southeast-1 * ap-southeast-2 * us-iso-east-1 * ap-southeast-3 * us-east-1 * us-east-2 * cn-northwest-1 * us-isob-east-1 * aws-global * aws-cn-global * aws-us-gov-global * aws-iso-global * aws-iso-b-global | string | eu-west-1 | ||
Topic Name | Required The name of the FIFO topic, which ends in | string | order-events.fifo | ||
Access Key | The access key ID of an IAM user, which you create in the AWS console under IAM, Users, Security credentials. Use it together with | string | |||
Autocreate Topic | If true, the topic is created when it does not exist. The credentials need the permission to create topics. | boolean | false | ||
Content-Based Deduplication | If true, SNS finds duplicates by the content of the body: a message with the same body as one published in the last 5 minutes is dropped. The topic must have content-based deduplication turned on. If false, each message gets a unique deduplication ID, so only a message that is sent again by a retry is dropped. | boolean | false | ||
KMS Key ID | The key of AWS KMS for a new topic to encrypt its messages with, as its key ID, such as | string | alias/aws/sns | ||
Message Structure | Set it to Enum values: * json | string | |||
Endpoint Overwrite | If true, it connects to the address in | boolean | false | ||
Profile Credentials Name | The name of the profile in the AWS credentials file. Only used when | string | production | ||
Proxy Host | The host name or IP address of an HTTP proxy to send the calls to AWS through, such as the proxy of a company network. Used only together with | string | proxy.example.com | ||
Proxy Port | The port of the proxy in | integer | 3128 | ||
Proxy Protocol | How to talk to the proxy itself: Enum values: * HTTP * HTTPS | string | HTTP | ||
Secret Key | The secret access key that belongs to | string | |||
Server-Side Encryption | If true, a topic that | boolean | |||
Session Token | The session token of temporary credentials, which comes with their access key and secret key, for example from AWS STS when you assume an IAM role. Only used when | string | |||
Subject | The subject of every notification, which email subscribers see as the subject line, and an SQS queue subscribed to the topic gets in the | string | Order received | ||
Overwrite Endpoint URI | The address to connect to instead of AWS, as a URL such as | string | http://localhost:4566 | ||
Default Credentials Provider | If true, the credentials are found in the environment instead of in | boolean | false | ||
Profile Credentials Provider | If true, the credentials are read from a named profile in the AWS credentials file ( | boolean | false | ||
Session Credentials | If true, it logs in with temporary credentials: | boolean | false |
Dependencies
At runtime, the aws-sns-fifo-sink Kamelet relies upon the presence of the following dependencies:
-
camel:aws2-sns
-
camel:kamelet
Camel JBang usage
Prerequisites
-
You’ve installed JBang.
-
You have executed the following command:
jbang app install camel@apache/camel Supposing you have a file named route.yaml with this content:
- route:
from:
uri: "kamelet:timer-source"
parameters:
period: 10000
message: 'test'
steps:
- to:
uri: "kamelet:aws-sns-fifo-sink" You can now run it directly through the following command
camel run route.yaml AWS SNS FIFO Sink Kamelet Description
Authentication methods
In this Kamelet you can avoid using explicit static credentials by specifying the useDefaultCredentialsProvider option and set it to true.
The order of evaluation for Default Credentials Provider is the following:
-
Java system properties -
aws.accessKeyIdandaws.secretKey. -
Environment variables -
AWS_ACCESS_KEY_IDandAWS_SECRET_ACCESS_KEY. -
Web Identity Token from AWS STS.
-
The shared credentials and config files.
-
Amazon ECS container credentials - loaded from the Amazon ECS if the environment variable
AWS_CONTAINER_CREDENTIALS_RELATIVE_URIis set. -
Amazon EC2 Instance profile credentials.
You can also use the Profile Credentials Provider, by setting the useProfileCredentialsProvider option to true and profileCredentialsName to the profile name.
Only one of access key/secret key or default credentials provider could be used
For more information, see the AWS credentials documentation
Optional Headers
In the Kamelet you can optionally set the following header:
-
subject/ce-subject: the subject of the message. It takes precedence over thesubjectoption. -
group/ce-group: the message group ID. If you do not set it, each message gets a group of its own.
The Subject
The subject option sets the subject of every notification. A subject or ce-subject header on a message takes precedence over it. Email subscribers see the subject as the subject line, and an SQS queue subscribed to the topic gets it in the Subject field of the JSON notification. A subject is ASCII text without line breaks; a longer one than 100 characters is cut to 100.
A Text for Each Kind of Subscriber
With messageStructure set to json, the body is a JSON object with a default text for all subscribers and, optionally, a text for each protocol, such as sqs, email or sms. SNS rejects a body without default. For example, the body
{"default": "Order 42 received", "email": "Your order 42 was received.", "sqs": "{\"orderId\": 42}"} sends {"orderId": 42} to the SQS queues subscribed to the topic, and Your order 42 was received. to the email addresses.
Encrypting a New Topic
serverSideEncryptionEnabled and kmsMasterKeyId apply only to a topic that autoCreateTopic creates: the topic is then created to encrypt its messages with that key of AWS KMS. With serverSideEncryptionEnabled and no kmsMasterKeyId, the integration does not start when the topic must be created. They do not change a topic that exists already.
For example:
- to:
uri: "kamelet:aws-sns-fifo-sink"
parameters:
topicNameOrArn: "order-events.fifo"
region: "eu-west-1"
subject: "Order received"
messageStructure: "json" Through an HTTP Proxy
Where AWS can be reached only through an HTTP proxy, as in many company networks, set proxyHost and proxyPort to the proxy. proxyProtocol is how the Kamelet talks to the proxy itself, not to AWS: it is HTTPS when it is not set, so set it to HTTP for a proxy that takes plain HTTP, such as Squid. The calls to AWS stay encrypted, as the proxy only passes them on. The Kamelet has no option for a proxy user or password.
The step that sends to the Kamelet, in a route:
- to:
uri: "kamelet:aws-sns-fifo-sink"
parameters:
topicNameOrArn: "order-events.fifo"
region: "eu-west-1"
useDefaultCredentialsProvider: true
proxyHost: "proxy.example.com"
proxyPort: 3128
proxyProtocol: "HTTP"