Kafka and zookeeper are required for sending data to and from the Spark cluster.
...
The topics are for transferring data to the Spark Application, receiving calculated KPIs from Spark, and a third topic for alarms. The default names of the topics are kpi-input, kpi-output, and kpi-alarm, but the names can be altered in the KPI Management Profile. Ensure that the number of partitions must match the number of Kafka brokers.
Retention Settings
The default data retention period in Kafka is one day. You can change the length of this period to conserve disk space.
...
| Tip | ||
|---|---|---|
| ||
The instruction above will change the retention settings for all topics in the Kafka cluster. You can also override the retention setting for individual topics during creation. For further information see 5.3.3 Starting Clusters and Creating Topics. |
For further information about Kafka, see /wiki/spaces/MD82/pages/3785234 in the /wiki/spaces/MD82/pages/3768690.
...