Spark applications must be configured with a set of Kafka topics that are either shared between multiple applications or dedicated to specific applications. The assigned topics must be created before you submit an application to the Spark service. Before you can create the topics you must start the Zookeeper and Kafka and Zookeeper services. 

Starting Services

The prerequisites for starting services are the following:

1. Spark, ZooKeeper and Kafka are installed and up and running.

2. Scripts are prepared according to:  5.2 Preparing and Creating Scripts for KPI Management

You can now confirm the status of the Spark cluster. Open a browser and go to http://<master host>:8080.
Image Modified 
Spark UI UI 

Creating Kafka Topics

Create Kafka topics and partitions using the mzsh command kafka. The names of the topics must correspond to the Spark application configuration.

Use a replication factor that is greater than one (1) to make sure that data is replicated between Kafka brokers. This decreases the risk of losing data in case of issues with the brokers. This is how to create topics, assuming the current working directory is the Kafka software folder:

$ mzsh mzadmin/<password> kafka./bin/kafka-topics.sh --servicecreate --keytopic <key><input \topic> --create bootstrap--topicserver <output\
topic>localhost:9092 --partitions <number of partitions> --replication-factor <number of replicas>
$ mzsh mzadmin/<password> kafka./bin/kafka-topics.sh --servicecreate --keytopic \<output <key>topic> --create bootstrap--topicserver <input\
topic>localhost:9092 --partitions <number of partitions> --replication-factor <number of replicas>
$ mzsh mzadmin/<password> kafka --service-key <key> \./bin/kafka-topics.sh --create --topic <alarm topic> --bootstrap-server \
localhost:9092 --partitions <number of partitions> --replication-factor <number of replicas>

$ mzsh mzadmin/<password> kafka --service-key kafka1 \
--create --topic kpi-output --partitions 6 --replication-factor 1
$ mzsh mzadmin/<password> kafka --service-key kafka1 \
--create --topic kpi-input --partitions 6 --replication-factor 1
$ mzsh mzadmin/<password> kafka --service-key kafka1 \
--create --topic kpi-alarm --partitions 6 --replication-factor 1

$ mzsh mzadmin/<password> kafka --service-key kafka1 \
--create --topic kpi-output --partitions 6 --replication-factor 1 --config retention.ms=86400000
$ mzsh mzadmin/<password> kafka --service-key kafka1 \
--create --topic kpi-input --partitions 6 --replication-factor 1 --config retention.ms=86400000
$ mzsh mzadmin/<password> kafka --service-key kafka1 \
--create --topic kpi-alarm --partitions 6 --replication-factor 1 --config retention.ms=86400000

...