ElasticSearch Index Sink
Provided by: "Apache Software Foundation"
Support Level for this Kamelet is: "Stable"
Stores each message, a JSON object, as a document in an index of Elasticsearch, so that it can be searched.
Use it to make records searchable, such as orders, log entries or products, or to feed dashboards in Kibana. For OpenSearch, use opensearch-index-sink.
Input: the body is the document, as JSON text, such as {"order": 42, "status": "shipped"}. The optional indexId header (or ce-indexid) is the id of the document: a document with the same id is replaced. Without it, Elasticsearch makes up a new id, so each message adds a new document. The optional indexName header (or ce-indexname) is the index to write to, instead of the indexName option.
Output: when it is used inside a route, the body afterwards is the id of the document, as text.
Good to know: a body that is not JSON fails the message, and so does a message with no index, from the option or the header. It connects with TLS by default (enableSSL). For a cluster whose certificate is self-signed, or signed by a private certificate authority (CA), give the certificate of that CA in certificate; without it, the message fails with PKIX path building failed. When Elasticsearch does not answer, the message fails after 30 seconds; connectionTimeout and socketTimeout change that. Headers that start with CamelElasticsearch are removed from the message first, so they cannot change what the Kamelet does.
Configuration Options
The following table summarizes the configuration options available for the elasticsearch-index-sink Kamelet:
| Property | Name | Description | Type | Default | Example |
|---|---|---|---|---|---|
ElasticSearch Cluster Name | Required The name of the Elasticsearch cluster. Required, but not checked against the cluster. | string | quickstart | ||
Host Addresses | Required The address of the Elasticsearch HTTP API, as | string | quickstart-es-http:9200 | ||
Certificate | The certificate of the certificate authority (CA) that signed the certificate of the cluster, such as the | string | |||
Connection Timeout | How long to wait for the connection to Elasticsearch, in milliseconds. When it is not set, it is 30000, which is 30 seconds. | integer | 5000 | ||
Enable SSL | If true, it connects with HTTPS. Set it to false for a cluster that is reached with plain HTTP. | boolean | true | ||
Index in ElasticSearch | The name of the index to store the documents in. Elasticsearch creates it when it does not exist. The | string | orders | ||
Password | The password of the Elasticsearch user in | string | |||
Socket Timeout | How long to wait for data from Elasticsearch, in milliseconds. When Elasticsearch sends nothing for this long, during the TLS handshake or while it stores the document, the message fails with a | integer | 10000 | ||
Username | The Elasticsearch user to log in as, such as | string |
Dependencies
At runtime, the elasticsearch-index-sink Kamelet relies upon the presence of the following dependencies:
-
camel:jackson3
-
camel:kamelet
-
camel:elasticsearch
-
camel:gson
-
camel:bean
Camel JBang usage
Prerequisites
-
You’ve installed JBang.
-
You have executed the following command:
jbang app install camel@apache/camel Supposing you have a file named route.yaml with this content:
- route:
from:
uri: "kamelet:timer-source"
parameters:
period: 10000
message: 'test'
steps:
- to:
uri: "kamelet:elasticsearch-index-sink" You can now run it directly through the following command
camel run route.yaml ElasticSearch Index Sink Kamelet Description
Authentication
This Kamelet supports username and password authentication for connecting to ElasticSearch clusters.
Trusting a Private Certificate Authority
The Kamelet connects with HTTPS by default (enableSSL). The certificate of the cluster is checked against the certificate authorities (CA) that Java trusts, so a cluster whose certificate is self-signed, or signed by a private CA, fails with PKIX path building failed.
Give the certificate of that CA in certificate, as a PEM or DER file encoded in base64 on one line. Elasticsearch generates such a CA when it starts with security on, in config/certs/http_ca.crt. Encode it with base64 -w0 http_ca.crt (base64 -i http_ca.crt on macOS). Only this CA is trusted then.
Timeouts
When Elasticsearch does not answer, the message fails after 30 seconds. connectionTimeout is how long to wait for the connection, and socketTimeout how long to wait for data from Elasticsearch, both in milliseconds.
Example
- route:
from:
uri: "kamelet:timer-source"
parameters:
message: '{"order": 42, "status": "shipped"}'
contentType: application/json
steps:
- to:
uri: "kamelet:elasticsearch-index-sink"
parameters:
hostAddresses: "es.example.com:9200"
clusterName: "quickstart"
indexName: "orders"
user: "elastic"
password: "{{search.password}}"
certificate: "{{search.ca}}"
connectionTimeout: 5000
socketTimeout: 10000 Here search.ca is an application property that holds the certificate of the CA, encoded in base64.