HTTP Source
Provided by: "Apache Software Foundation"
Support Level for this Kamelet is: "Stable"
Fetches a web address with an HTTP request, a GET unless method says otherwise, at a regular interval, and sends each response as a message.
Use it to poll a REST API, a feed or a status page for its current data, for example fetch the state of the orders every minute. To receive the requests that other systems send instead, use webhook-source.
Output: the body is the body of the response, as bytes. The headers of the response, such as Content-Type, are copied to the message, and CamelHttpResponseCode holds the status code.
Good to know: it fetches the address every period and sends a message for each response, even when the data has not changed. A response with a status code outside okStatusCodeRange, which is 200-299 when it is not set, sends no message: the error is logged, and the address is fetched again at the next period. Log in either with authUsername and authPassword, with a static token in authBearerToken, or with oauth2ClientId, oauth2ClientSecret and oauth2TokenEndpoint: the Kamelet then gets a token from that URL and sends it as a Bearer token. To go through an HTTP proxy, set proxyHost and proxyPort, and proxyUser and proxyPassword when the proxy asks for a login; an https:// address goes through the proxy too. The certificate of an https:// service is checked against the certificate authorities that Java trusts. To trust a private one with Camel JBang or Camel Main, set the application properties camel.ssl.enabled=true, camel.ssl.trustStore, camel.ssl.trustStorePassword and camel.component.https.useGlobalSslContextParameters=true: that trust store then replaces the one of Java for every https:// call of the application. It keeps the cookies that the service sets, as a browser does, and sends them back on the next calls. To turn that off, set the application property camel.component.http.cookie-management-disabled=true, or camel.component.https.cookie-management-disabled=true for an https:// address: it then applies to every call of the application to such addresses.
Configuration Options
The following table summarizes the configuration options available for the http-source Kamelet:
| Property | Name | Description | Type | Default | Example |
|---|---|---|---|---|---|
URL | Required The address to fetch, with the path and any query parameters. It must start with | string | https://api.example.com/v1/orders/status | ||
Bearer Token | A static token, such as an API key, sent in the | string | |||
Authentication Method | The login method, | string | Basic | ||
Authentication Password | The password of | string | |||
Authentication Username | The user to log in as, together with | string | |||
Authentication Preemptive | If true, | boolean | false | ||
Content Type | The value of the | string | application/json | text/csv | |
Follow Redirects | If true, a redirect, a response with a status code such as 301, 302 or 307, is followed to the address in its | boolean | |||
Method | The HTTP method of the request, such as Enum values: * GET * POST * PUT * DELETE * HEAD * OPTIONS * TRACE * PATCH | string | POST | ||
OAuth2 Client ID | The client ID of the application, as registered with the OAuth 2.0 provider. With | string | |||
OAuth2 Client Secret | The client secret of the application, as given by the OAuth 2.0 provider. | string | |||
OAuth2 Scope | The scope to ask for in the token request, when the OAuth 2.0 provider needs one. | string | orders.write | ||
OAuth2 Token URL | The URL to get the tokens from, as given by the OAuth 2.0 provider. The Kamelet logs in to it with | string | https://login.example.com/oauth2/token | ||
OK Status Codes | The status codes of a response that is sent as a message, as ranges such as | string | 200-299,404 | ||
Period between Updates | How long to wait between two requests, in milliseconds. The default, 10000, is ten seconds. | integer | 10000 | 60000 | |
Proxy Host | The host name or IP address of an HTTP proxy to send the requests through, without the port. Use it with | string | proxy.example.com | ||
Proxy Password | The password of | string | |||
Proxy Port | The port of the HTTP proxy in | integer | 3128 | ||
Proxy User | The user to log in to the HTTP proxy as, when the proxy asks for a login. Use it with | string |
Dependencies
At runtime, the http-source Kamelet relies upon the presence of the following dependencies:
-
camel:http
-
camel:kamelet
-
camel:timer
Camel JBang usage
Prerequisites
-
You’ve installed JBang.
-
You have executed the following command:
jbang app install camel@apache/camel Supposing you have a file named route.yaml with this content:
- route:
from:
uri: "kamelet:http-source"
parameters:
.
.
.
steps:
- to:
uri: "kamelet:log-sink" You can now run it directly through the following command
camel run route.yaml HTTP Source Kamelet Description
Configuration
The HTTP Source Kamelet periodically fetches data from an HTTP endpoint. It supports the following configurations:
-
URL: The HTTP endpoint to fetch data from (required, must start with http:// or https://)
-
Period: The interval between fetches in milliseconds (default: 10000)
-
Content Type: The Accept header content type for the request (default: "application/json")
-
Method: The HTTP method of the request (default: GET)
Logging In
The login options are all optional. Without them, the requests are sent without a login.
| Login | Options | When the credentials are sent |
|---|---|---|
Basic or Digest |
| After the service answers 401 and asks for them. With |
Static token, such as an API key |
| After the service answers 401 with a |
OAuth 2.0 client credentials |
| With every request: the Kamelet gets a token from |
Output Format
The Kamelet outputs the HTTP response body and preserves any HTTP headers from the response.
Usage Example
- route:
from:
uri: "kamelet:http-source"
parameters:
url: "https://api.example.com/data"
period: 30000
contentType: "application/json"
steps:
- to:
uri: "kamelet:log-sink" Example with XML Content
- route:
from:
uri: "kamelet:http-source"
parameters:
url: "https://api.example.com/xml-feed"
period: 60000
contentType: "application/xml"
steps:
- to:
uri: "kamelet:log-sink" Example with High Frequency Polling
- route:
from:
uri: "kamelet:http-source"
parameters:
url: "https://api.example.com/status"
period: 5000
contentType: "text/plain"
steps:
- to:
uri: "kamelet:log-sink" This example polls the endpoint every 5 seconds for status updates.
Example with a Static Token
- route:
from:
uri: "kamelet:http-source"
parameters:
url: "https://api.example.com/v1/orders/status"
period: 60000
authMethod: Bearer
authBearerToken: "{{api.token}}"
authenticationPreemptive: true
steps:
- to:
uri: "kamelet:log-sink" The token is sent as the header Authorization: Bearer followed by the token.
Status Codes That Send a Message
Only a response with a status code in okStatusCodeRange is sent as a message. When the option is not set, it is 200-299: a response with a status code of 300 or higher sends no message, the error is logged, and the address is fetched again at the next period. To also receive the responses with another code, such as 404 while a resource does not exist yet, list it together with the 2xx codes:
- route:
from:
uri: "kamelet:http-source"
parameters:
url: "https://api.example.com/v1/reports/today"
okStatusCodeRange: "200-299,404"
steps:
- to:
uri: "kamelet:log-sink" The CamelHttpResponseCode header of the message then tells a 404 from a 200. Setting only 404 would stop the 200 responses.
Method
The request is a GET when method is not set. Some APIs return data only for a POST; set method: POST for them. The request has no body.
- route:
from:
uri: "kamelet:http-source"
parameters:
url: "https://api.example.com/v1/reports/search"
method: POST
period: 60000
steps:
- to:
uri: "kamelet:log-sink" Going Through a Proxy
To fetch the address through an HTTP proxy, set proxyHost and proxyPort. The proxy is used only when both are set. When the proxy asks for a login, also set proxyUser and proxyPassword: they are sent after the proxy answers 407 and asks for them.
- route:
from:
uri: "kamelet:http-source"
parameters:
url: "https://api.example.com/v1/orders/status"
proxyHost: proxy.example.com
proxyPort: 3128
proxyUser: "{{proxy.user}}"
proxyPassword: "{{proxy.password}}"
steps:
- to:
uri: "kamelet:log-sink" An https:// address goes through the proxy too, in a tunnel.
Following Redirects
A redirect, a response with a status code such as 301, 302 or 307, is not followed when followRedirects is not set: it sends no message, the error is logged, and the address is fetched again at the next period. With followRedirects: true, the address in the Location header of the response is fetched, and its response is sent as the message.
Trusting a Private Certificate Authority
The certificate of an https:// service is checked against the certificate authorities that Java trusts. A service whose certificate is signed by a private certificate authority fails with PKIX path building failed.
With Camel JBang or Camel Main, put the certificate of the authority in a trust store, for example with keytool -importcert -file ca.crt -keystore truststore.p12 -storetype PKCS12, and set these application properties:
camel.ssl.enabled = true
camel.ssl.trustStore = file:/etc/certs/truststore.p12
camel.ssl.trustStorePassword = changeit
camel.component.https.useGlobalSslContextParameters = true That trust store then replaces the one of Java for every https:// call of the application, so it must also hold the authorities of any public service the application calls. The http:// calls are not changed.