Kafka(4.3)

This section describes the Kafka agents and profile. The agents are used to forward and collect messages using Kafka. Kafka is a cluster-based software executed externally outside of Usage Engine. Kafka uses a high throughput publish-subscriber messaging model. It persists all messages and by design it decouples the publisher from the subscriber. That means that the forwarding agent will keep writing to the log even if the collection agent terminate.

The Kafka forwarding agent is available in real-time workflows and is listed among the processing agents while the Kafka collection agent is available in both batch and real-time workflows and is listed among the collection agents.

In this release we have built two new Kafka agents that will replace the previous ones in a later release. In this documentation, the previous agents and profile are referred to as Legacy Kafka agents and Legacy Kafka profile.