...

The HTTP Client Function function enables communication with a third-party HTTP server. Using it the functions, stream editors can control outgoing requests handling. This Function function supports REST API and also supports authentication via the OAuth 2.0 framework. The following HTTP Client Functions functions are available in Usage Engine:

...

...

...

Forwarder

...

You can set the values for how the HTTP Client Function function would behave when communication communicating with the third-party HTTP server. The Configuration tab allows you to , and specify the method of communication.

The options specific to each of the 3 Functions, whether it is for Collector, Processor or Forwarder, will be mentioned in their corresponding sections at the end of this page.

Parameters

Under Parameters, you get the Connection and Operational Settings that are applicable to HTTP Client Collectors, Processors, and Forwarders.

Connection

Depending on the users' configuration, two authentication methods are available: Basic authentication and OAuth 2.0 implementation. A selectable option from the Authentication type is used to make the selection and based on that separate configuration options are available. There will be additional options in the Configuration tab for the Authentication type selected.

Basic Authentication

...

Add the Username and Password if there are any. These options can be left empty.

...

Add the Target URL of the HTTP request to be made to the HTTP server.

In the Configuration tab,  select an HTTP request method from the following options:

...

• Select Body or Form and add the corresponding details in the content of the request message. You can use Variable Insertion.
  

• Expand the examples below to view samples of HTTP request messages for Body or Form:

...

{
"name" : "my_username",
"first-name" : "My",
"last-name" : "Username",
"display-name" : "My Username",
"email" : "user@example.test",
"active" : true
}

...

name=my_username&first-name=My&last-name=Username&display-name=My%20Username&email=user@example.test&active=true

In Target URL, enter the URL of the target HTTP server.  You can use Variable Insertion.  You can select a combination of the following options to be included in the URL:

...

Select to customize the headers of the URL and specify multiple keys and their respective values.

The default value for the HTTP headers “content-type” and “accept” is application/json.

You can override the default value for HTTP header “content-type”.

It is recommended that the default value for HTTP header "accept" is left unchanged. The HTTP Functions only support application/json.

OAuth 2.0 Authorization

The HTTP Method, Target URL, and associated options are configured in the same way. When the Authentication type selection is selected as OAuth 2.0 additional options are displayed. Their configuration is mandatory to properly activate the Function. 

It is also possible to use the Secrets Wallet functionality to apply stored credentials for the three HTTP Functions. 

The Use Redirect Fields checkbox allows users to toggle URL redirection using the available options. Two fields are displayed when this option is enabled – the Target URL field where the designated address is to be entered, and an optional Resource Path. 

The Configure Token section of the Function controls the authentication token credentials. The Header Prefix field is the Authorization header placed before the access token. Based on the Grant Type selection there are two types of credentials access provided for the authentication provisions – Password and Client credentials. Each of them has a distinctive configuration option. 

Password Credentials Grant Type Options

• Access Token URL – In this field specify the URL containing the authorization token. 

• Client ID – In this field enter the client identifier. 

• Client Secret – in this field enter the associated client secret. This parameter may be omitted if the client secret is an empty string. 

• Username – This field is used to specify the relevant username, it is mandatory. 

• Password – This field is used to specify the password associated with the username, it is mandatory. 

• Scope – The authorization and token endpoints allow the client to specify the access request scope using this parameter. The entered value must be expressed as a list of space-delimited, case-sensitive strings. This is an optional parameter.

• Client Authentication – This option controls the type of authentication mechanism as defined by the connecting server – Auth Header or Auth Body are available. The options are named "Send as Basic Auth Header" and "Send client credentials in body". 

Client Credentials Grant Type Options 

• Access Token URL – In this field specify the URL containing the authorization token. 

• Client ID – In this field enter the client identifier. 

• Client Secret – in this field enter the associated client secret. This parameter may be omitted if the client secret is an empty string. 

• Scope – The authorization and token endpoints allow the client to specify the access request scope using this parameter. The entered value must be expressed as a list of space-delimited, case-sensitive strings. This is an optional parameter.

• Client Authentication – This option controls the type of authentication mechanism as defined by the connecting server – Auth Header or Auth Body are available. The options are named "Send as Basic Auth Header" and "Send client credentials in body". 

Both grant-type options also include optional parameters: 

• Include query string in URL – This enables a query string in the token URL. 

• Customize header - This allows for custom headers to be specified.

Operational Settings

In Operational Settings, enter the following details that determine how you want to handle the errors. The following options are available for HTTP Collector and Processor Functions only:

In Action on error, specify the action you want to take in case of an error. You can select from the following options:

...

Select to ignore the error and the stream keeps on running

...

Any response with the HTTP status code 4xx or 5xx is considered an error. All errors get logged in the system logs. You can refer to the Logs for more information. The default value is Ignore.

In Result Key, you can assign a name to the result that you receive in response to the request. It is recommended to configure the resulting key to make future references of the same response easy and traceable. The default value is httpResponse.

Response Headers

...

HTTP Response Headers Support

By enabling response headers, the overall data size of the HTTP packets will also increase. Caching of the body checkbox is tied to the headers. 

...

HTTP Client Collector

...

The HTTP Client collector Function collects data from a third-party HTTP server.

To configure this Function, take the following steps in addition to the ones mentioned in the Connection and Operational Settings:

In Request run limit, specify the number of times the operation should run. 

HTTP Client Processor

The HTTP Client processor Function processes the data received from a third-party HTTP server.

In addition to the steps mentioned in the Connection and Operational Settings, you can take the following optional steps to configure HTTP Client processor:

• Select Enable data cache if you want to store information received in response to an HTTP request in the cache. You can assign a key to the response stored in the cache. In this case, whenever you send the same HTTP request, the response is fetched from the cache. This helps in the faster turnaround time. Enter the following details:

  • In Response key, assign a cache key to the response to an HTTP request. By default, it is the URL that you specify in Target URL.

  • In Clear cache after, specify the duration (in seconds) after which the response is removed from the cache. The duration must be greater than zero. The default duration is 300 seconds.

Preview Function Toggle

This Function is available only in the Client Processor node. Below the Advanced Settings dropdown menu, a preview toggle (“Enable preview for this function”) can be enabled or disabled. 

By default, this Function will automatically send requests when the preview function is turned on, when the toggle is turned off, it will disable this functionality. When this function is off, POST attempts will not be made to the server. This protects external systems from being modified by the preview function. 

HTTP Client Forwarder

...

Select this Function to send data to the third-party HTTP server.

In addition to the steps mentioned in the Connection and Operational Settings, you can take the following optional steps to configure HTTP Client forwarder:

...

In Maximum number of parallel requests, specify the maximum number of parallel requests that can be handled simultaneously.  

You can toggle on the Retry on fail functionality, allowing the service to retry the operation when an error has occurred. You can specify the number of retries in the Number of retries box, where the retries will be attempted inside the dynamically adjusted period of retry time.

...

You can choose to either Ignore or Stop the stream when an error is detected.  

...