Versions Compared

Old Version 3

changes.mady.by.user Jenni Wallin

Saved on

compared with

New Version Current

changes.mady.by.user Felicity Borrelli

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

The Kafka batch collection agent consumes messages from Kafka. A batch will be forwarded when the configured batch size has been reached, or when the hardcoded timeout of 5 seconds has been reached.

Note!

The workflow will remain running when all messages have been consumed and will wait for more messages. You can stop the workflow from Desktop Online, mzcli mzsh commands, or the operations REST interface. The stop will be delayed for 5 seconds.

...

Agent

Configuration

Kafka

Collects messages from Kafka. You define Define the size of the files forwarded by the Disk forwarding agent with the Batch Size setting.

Analysis

Receives KafkaRecord UDRs and creates output UDRs based on the contents of the input UDR. An offset is used to create a unique id.

Encoder

Encodes the data to the format the files will be forwarded in.

Disk

Creates files. The size of the files is configured in the Kafka collector agent.

...