Kafka Batch Forwarding Agent

The Kafka batch forwarding agent sends messages to Kafka. The agent does not have any configuration settings and must therefore be preceded by an Analysis or Aggregation agent that populates the KafkaRecord UDRs with the messages. The same forwarding agent can send messages to several topics and partitions.

Workflow Example

A simple workflow with a Kafka batch forwarding agent can look like this:

kafkaBatchForw_wf.png
The Kafka batch forwarding agent in a workflow

This workflow example has been created as follows:

Workflow Design

Create the workflow with the following agents:

Agent

Configuration

Disk

Collects files from disk.

Decoder

Decodes the input data to internal UDRs.

Analysis

Creates KafkaRecord UDRs and then maps the content of the input UDR. Sets the destination topic for the data.

Kafka

No configurations made in the agent.

Kafka Profile

The Kafka profile defines the broker to which the messages should be forwarded. You need to create the Kafka profile before selecting it in the Execution tab of the Workflow Properties.

kafkaWF_prop.png
Workflow Properties with Kafka profile selected

Analysis Agent

Configure the Analysis agent to create KafkaRecord UDRs, and then map the content of the input UDR. Set the topic to which the data shall be forwarded.

kafka.KafkaRecord myUDR; consume { myUDR = udrCreate(kafka.KafkaRecord); strToBA(myUDR.value, input.value); myUDR.topic = "myNewMZTopic"; udrRoute(myUDR); }

 

 

Â