Azure Event Hub Consumer Agent Configuration

Owned by Kevin Wu
Last updated: Jan 03, 2024
3 min read

To open the Azure Event Hub Consumer agent configuration dialog from a workflow configuration, you can do either one of the following:

  • double-click the agent icon

  • select the agent icon and click the

    Edit button

Configuration

The Azure Event Hub Agent Configuration consists of the following tabs:

General Tab

Azure Event Hub Consumer agent configuration - General tab 

Setting

Description

Setting

Description

Profile

Select the Azure profile that has been configured with the connection details for the Azure Event Hub. For more information about how to configure the profile, refer to Azure Profile.

Partition

Enter the partition ID of the event hub to be used by the agent. This field is optional.

Storage Directory

Enter the directory path of where the data checkpoints are to be stored on the local directory of the EC server the workflow is running on. The data checkpoint consists of a sequence number stored in a binary file that is used by the Event Hub Consumer agent to resume data collection from where it stopped.

Should the field be left empty, the agent will either Start at latest or Start at earliest, depending on the configuration set below.

Consumer Group

Enter the name of the consumer group the agent will be retrieving from.

Start at latest (from end)

Events are selected from the last offset from when the workflow was started. If you select this option, there is a risk that data can be lost after a restart. This configuration will only be applied when the workflow is running for the first time or if the Storage Directory is left empty or missing.

Start at earliest (from beginning)

Events are collected from the first offset. If you select this option, there is a risk that events will be processed multiple times after a restart. This configuration will only be applied when the workflow is running for the first time or if the Storage Directory is left empty or missing.

Use EventDataUDR

Select this option to send the EventDataUDR which contains the body as a string along with additional metadata that the Event Hub provides.

Note!

By default, the Azure Event Hub Consumer agent sends the event body as a bytearray.

Advanced Tab

The Advanced tab contains additional settings for checkpointing the offset of events collection when the workflow is initiated. When a directory path is specified in the Storage Directory field, the settings under the Advanced tab will override the settings for Start at latest (from end) and Start at earliest (from beginning) in the General tab. 

Azure Event Hub Consumer agent configuration - Advanced tab

Setting

Description

Setting

Description

Ignore out of sequence range

Select this checkbox to enable the following options to predefine where you would like the reading of offset to take place after a workflow restarts. 

Start at latest (from end)

Events are selected from the last offset from when the workflow was started. If you select this option, there is a risk that data can be lost after a restart.

Start at earliest (from beginning)

Events are collected from the first offset. If you select this option, there is a risk that events will be processed multiple times after a restart.