Couchbase Source
Provided by: "Apache Software Foundation"
Support Level for this Kamelet is: "Preview"
Runs a SQL++ (N1QL) query on a Couchbase bucket again and again, and sends each document or row it finds as a message, as JSON text.
Use it to pass the documents of a Couchbase bucket on to another system, for example to copy them to a search index or a data warehouse. To store documents, use couchbase-sink.
Output: the body is one result row as JSON text. Without statement, the query reads all the documents of the default collection, and a row is {"<bucket>":<the document>,"id":"<document ID>"}. With collection set, it reads that collection of scope, and a row is {"<collection>":<the document>,"id":"<document ID>"}. The CamelCouchbaseId header holds the ID of the document.
Good to know: the query runs again every delay milliseconds and, unless processedStrategy is delete, nothing marks a document as read, so the same documents are sent on every run. With delete, each document is deleted once its message has been processed. A statement must name the document ID id (SELECT META().id AS id, …), or its rows are skipped. With fullDocument set to true, the body is an object of the Couchbase SDK instead of JSON text. The connection is not encrypted, also with protocol set to https.
Configuration Options
The following table summarizes the configuration options available for the couchbase-source Kamelet:
| Property | Name | Description | Type | Default | Example |
|---|---|---|---|---|---|
Bucket | Required The name of the bucket to read the documents from. They are read from its default collection, unless | string | orders | ||
Hostname | Required The host name or IP address of a node of the Couchbase cluster. The Kamelet connects to its data service, on port 11210. | string | couchbase.example.com | ||
Protocol | Required The protocol part of the address of the server, | string | http | ||
Collection | The collection to read from, in | string | orders | ||
Connection String | The address of the cluster for the Couchbase SDK, which starts with | string | couchbase://couchbase.example.com:11210 | ||
Port | The port of the web console and REST interface of Couchbase, 8091 by default. The Kamelet does not connect to this port; to reach the data service on a port other than 11210, set | integer | 8091 | ||
Delay | How long to wait after one poll before the next, in milliseconds. | integer | 500 | ||
Descending | If true, the results come in descending order. | boolean | false | ||
Design Document Name | The design document that holds the view to read. Only used when | string | beer | ||
Full Document | If true, the Kamelet reads each document found by its ID and sends it as an object of the Couchbase SDK, not as JSON text. If false, it sends the row of the query as JSON text. | boolean | false | ||
Limit | The most documents to send on each poll. A negative value means no limit. | integer | -1 | ||
Password | The password of the Couchbase user | string | |||
Processed Strategy | What to do with a document once its message has been processed: Enum values: * none * delete | string | delete | ||
Scope | The scope of the bucket to read from. A | string | sales | ||
Skip | How many documents to skip on each poll before sending the others. A negative value skips none. | integer | -1 | ||
SQL++ Statement | The SQL++ (N1QL) query to run on each poll, instead of reading all documents. It runs in | string | SELECT META().id AS __id, * FROM _default | ||
Use View | If true, the Kamelet reads a MapReduce view, chosen with | boolean | false | ||
Username | The Couchbase user to log in as. It needs the rights to read the documents of | string | |||
View Name | The view to read. Only used when | string | brewery_beers |
Dependencies
At runtime, the couchbase-source Kamelet relies upon the presence of the following dependencies:
-
camel:couchbase
-
camel:kamelet
Camel JBang usage
Prerequisites
-
You’ve installed JBang.
-
You have executed the following command:
jbang app install camel@apache/camel Supposing you have a file named route.yaml with this content:
- route:
from:
uri: "kamelet:couchbase-source"
parameters:
.
.
.
steps:
- to:
uri: "kamelet:log-sink" You can now run it directly through the following command
camel run route.yaml Couchbase Source Kamelet Description
Query Modes
The bucket is polled in one of two modes.
By default the N1QL statement in the statement property is run against the bucket on each poll. Set useView to true to poll a MapReduce view instead, selected with designDocumentName and viewName.
Writing the Statement
The statement runs in the scope set by scope, or in the _default scope when it is not set, so the keyspace is a collection of that scope rather than the bucket name - use _default for the default collection.
It must also alias the document id as __id, because the consumer skips any row without that field. A query that omits it produces a source that polls without emitting anything:
SELECT META().id AS __id, * FROM _default
Example with a Scope and a Collection
Without statement, set scope and collection to read all the documents of that collection. Each row is {"<collection>":<the document>,"__id":"<document ID>"}. With processedStrategy set to delete, each document is deleted from the collection once its message has been processed, so it is sent only once. A message that fails keeps its document, and the next poll sends it again.
- route:
from:
uri: "kamelet:couchbase-source"
parameters:
protocol: "http"
couchbaseHostname: "couchbase.example.com"
bucket: "shop"
scope: "sales"
collection: "inbox"
processedStrategy: "delete"
delay: 5000
username: "{{couchbase.username}}"
password: "{{couchbase.password}}"
steps:
- to:
uri: "kamelet:log-sink" A statement runs in scope too, so it names the collection alone. With fullDocument true, or processedStrategy delete, also set collection to the collection the statement reads: the documents are read, or deleted, in collection, the default collection when it is not set.
scope: "sales"
collection: "orders"
statement: "SELECT META().id AS __id, o.* FROM orders o WHERE o.status = 'new'"