This is a new set of Kafka agents that will replace the previous agents in a future major version. In Desktop Online, the previous agents and profile are marked as deprecated and have the following names: Kafka Profile Deprecated Batch collection agent: Kafka Deprecated Realtime collection agent: Kafka Deprecated Realtime forwarding agent: Kafka Deprecated
The main differences between the versions are that the new version: Includes a batch forwarding agent. Stores transactions in the Kafka cluster (transactions are only tracked for batch workflows). Stores offset in the Kafka cluster only. One consumer can collect from several topics, and one producer can forward to several topics. Includes automatic rebalancing for collection of messages. Several workflows can collect messages in parallel from the same topics, provided that there are several partitions. When you start a new workflow, the running workflows will automatically be rebalanced, see Automatic Scale Out and Rebalancing (4.2). Batch collection workflows will remain in running state, even when all data has been collected. Uses only two UDR types:
As the UDR types differ between the deprecated and new agents, you have to rewrite workflows with Legacy Kafka agents after having upgraded to this version. | 