Ceph Source
Provided by: "Apache Software Foundation"
Support Level for this Kamelet is: "Stable"
Reads the files (objects) in a bucket of a Ceph Object Gateway, one message per file, and by default deletes each file once it has been read.
Use it to process files that another system drops into a Ceph bucket, such as uploaded documents, exported reports or log files. To react to the bucket notifications of Ceph instead of reading the bucket over and over, use ceph-event-based-source.
Output: the body is the content of the file, as bytes. The headers give the bucket name, the key (the path of the file in the bucket, CamelAwsS3Key), the content type and the ETag of the file.
Good to know: files are deleted from the bucket after they are read, unless deleteAfterRead is set to false; then the same files are read again on every poll. Files in folders are read too. The bucket is addressed as part of the host name (such as my-bucket.ceph.example.com) unless cephUrl uses an IP address; set forcePathStyle to true when the DNS of the gateway has no wildcard name for its buckets.
Configuration Options
The following table summarizes the configuration options available for the ceph-source Kamelet:
| Property | Name | Description | Type | Default | Example |
|---|---|---|---|---|---|
Access Key | Required The S3 access key of a Ceph Object Gateway user, as shown by | string | |||
Bucket Name | Required The name of the bucket to read the files from. | string | my-bucket | ||
Ceph Url Address | Required The address of the Ceph Object Gateway (RGW), as a URL with the scheme and the port. | string | http://ceph-rgw.example.com:7480 | ||
Secret Key | Required The S3 secret key that goes with | string | |||
Bucket Zone Group | Required The name of the zone group of the Ceph Object Gateway, used as the region of the requests. A Ceph cluster that was not set up with several zone groups uses | string | default | ||
Autocreate Bucket | If true, the bucket is created when it does not exist yet. If false, every poll fails while the bucket does not exist. | boolean | false | ||
Delay | How long to wait between two polls of the bucket, in milliseconds. | integer | 500 | ||
Auto-delete Objects | If true, each file is deleted from the bucket after it has been read. Set it to false to keep the files; they are then read again on every poll. | boolean | true | ||
Force Path Style | If true, the bucket name is put in the path of the address ( | boolean | false | ||
Ignore Body | If true, the content of the file is not read, and the body is empty; only the headers describe the file. It takes precedence over | boolean | false | ||
Include Body | If true, the body is the content of the file, as bytes. If false, the body is a stream of the content, which suits large files. | boolean | true | ||
Prefix | Read only the files whose key starts with this text, such as a folder name followed by | string | folder/ |
Dependencies
At runtime, the ceph-source Kamelet relies upon the presence of the following dependencies:
-
camel:aws2-s3
-
camel:kamelet
Camel JBang usage
Prerequisites
-
You’ve installed JBang.
-
You have executed the following command:
jbang app install camel@apache/camel Supposing you have a file named route.yaml with this content:
- route:
from:
uri: "kamelet:ceph-source"
parameters:
.
.
.
steps:
- to:
uri: "kamelet:log-sink" You can now run it directly through the following command
camel run route.yaml