An Elasticsearch profile is used to read and write data in an Elasticsearch Service in AWS and can be accessed by batch workflows using Aggregation agents. Elasticsearch is supported to provide a scalable distributed storage option. It is a search and analytics engine that provides high scalability on a cloud infrastructure with cluster scaling options and replication across several domains, and the capacity to query large amounts of data.
Compared with storing to disk, Elasticsearch provides greater capacity, however you must bear in mind that heap and execution time are increased, by what percentage depends on the amount of data being processed.
Note!
You cannot run more than one batch workflow to use the same aggregation profile, configured to use Elasticsearch, at the same time.
The Elasticsearch profile is loaded when you start a workflow that depends on it. Changes to the profile become effective when you restart the workflow.
The profile has been verified on Elasticsearch version 6.0.
For further information on the use of Elasticsearch, see https://www.elastic.co/guide/en/elasticsearch/reference/6.0/index.html.
Configuration
To create a new Elasticsearch profile, click the New Configuration button in the upper left part of the Desktop window, and then select Elasticsearch Profile from the menu.
The contents of the menus in the menu bar may change depending on which configuration type that has been opened in the currently displayed tab. The Elasticsearch profile uses the standard menu items and buttons that are visible for all configurations, and these are described in Menus and Buttons.
The Edit menu is specific for the Elasticsearch profile configurations.
| Item | Description |
|---|---|
External References | Select this menu item to Enable External References in an agent profile field. This can be used to configure the use of the following fields, and the respective external reference keys available:
Note! The Node Config must be entered in in the form of a comma separated list of <host:port> values, for example, 127.0.0.1:8990, 123.0.0.1:9000. For further information, see Using External Reference in Agent Profile Fields and External Reference Profile. |
The Elasticsearch profile configuration consists of two tabs: Connectivity and Advanced.
Connectivity Tab
The Connectivity tab is displayed by default. The following settings are available in the Elasticsearch profile:
| Setting | Description |
|---|---|
| Index | Enter the index that you want to use. For further information on the use of an index in Elasticsearch, see https://www.elastic.co/guide/en/elasticsearch/reference/6.0/_basic_concepts.html. Note! When you name your index, bear in mind that two indices are created in the background for a workflow, which are named <index name>_committed and <index name>_deleted. It is therefore important to avoid naming an index which would duplicate these index names. For example, if you name the index "customer_index", the indices "customer_index_committed" and "customer_index_deleted" are automatically created. These two additional indices are removed after the workflow has been run. |
Host | Enter the host name or IP address of the host on which the Elasticsearch service is running. |
Port | Enter the port of the Elasticsearch service. |
Test Connection | Click to test the connection to the service, using the configured values. |
Advanced Tab
The Advanced tab contains additional properties that you can use for performance tuning for the Elasticsearch service.
For the Elasticsearch service, you can only modify the properties listed as shown above in the Advanced tab.
Add Comment