The Kafka real-time collection agent consumes messages from the topic and partitions stated in the Kafka collection agents' configuration, which contains the following settings:
| Setting | Description |
|---|---|
Profile | Browse and select the profile the agent should use, as defined in the Kafka Profile [hide]profile[/hide](3.0). |
All | Select this option if you want messages to be collected from all of the partitions. |
| Range | Select this option if you want messages to be collected from the range that you specify. |
| Specific | Select this option if you want messages to be collected from the specified partition(s). This is a comma separated list. |
| Offset You must determine from which offset you want to start collecting. | |
| Start at Beginning | Select this option to collect messages from the first offset. If you select this option, there is a risk that messages will be processed multiple times after a restart . |
| Start at End | Select this option to select messages from the last offset from when the workflow was started. If you select this option, there is a risk that data can be lost after a restart . |
| Start at Requested | Select this option to start collection from the given offset on the incoming route specified in the UDRs. You set the offset using the |
| Managed by kafka (auto commit) | This function should be used in the case you have a kafka profile using the Group id, to enable load sharing between several Kafka collectors. Use this in combination with enable.auto.commit=true in the kafka profile, in the advanced, consumer tab. |
Input/Output Data
Input Data
Messages from the Kafka Log.
Add Comment