...
Create Kafka topics and partitions using the mzsh command kafkascripts included in the Kafka installation. The names of the topics must correspond to the Spark application configuration.
...
| Code Block |
|---|
|
$ ./bin/kafka-topics.sh --create --topic <input topic> --bootstrap-server \
localhost:9092 --partitions <number of partitions> --replication-factor <number of replicas>
$ ./bin/kafka-topics.sh --create --topic <output topic> --bootstrap-server \
localhost:9092 --partitions <number of partitions> --replication-factor <number of replicas>
$ ./bin/kafka-topics.sh --create --topic <alarm topic> --bootstrap-server \
localhost:9092 --partitions <number of partitions> --replication-factor <number of replicas>
|
| Info |
|---|
Example - Creating Kafka Topics |
Example - Creating Kafka topics./bin/kafka-topics.sh --create --topic kpi-output --partitions 6 --replication-factor 1 ./bin/kafka-topics.sh --create --topic kpi-input --partitions 6 --replication-factor 1 ./bin/kafka-topics.sh --create --topic kpi-alarm --partitions 6 --replication-factor 1 |
| Code Block |
|---|
./bin/kafka-topics.sh --create --topic kpi-output --partitions 6 --replication-factor 1
./bin/kafka-topics.sh --create --topic kpi-input --partitions 6 --replication-factor 1
./bin/kafka-topics.sh --create --topic kpi-alarm --partitions 6 --replication-factor 1 |
| Info |
|---|
Example - Creating Kafka topics| Code Block |
|---|
| $ mzsh mzadmin/<password> kafka --service-key kafka1 \
--create --topic kpi-output --partitions 6 --replication-factor 1
$ mzsh mzadmin/<password> kafka --service-key kafka1 \
--create --topic kpi-input --partitions 6 --replication-factor 1
$ mzsh mzadmin/<password> kafka --service-key kafka1 \
--create --topic kpi-alarm --partitions 6 --replication-factor 1 |
|
...
| Info |
|---|
Example - Creating Kafka topics, overriding retention settings./bin/kafka-topics.sh --create --topic kpi-output --partitions 6 --replication-factor 1 --config retention.ms=86400000 ./bin/kafka-topics.sh --create --topic kpi-input --partitions 6 --replication-factor 1 --config retention.ms=86400000 ./bin/kafka-topics.sh --create --topic kpi-alarm --partitions 6 --replication-factor 1 --config retention.ms=86400000
| Code Block |
|---|
$ mzsh mzadmin/<password> kafka --service-key kafka1 \
--create --topic kpi-output --partitions 6 --replication-factor 1 --config retention.ms=86400000
$ mzsh mzadmin/<password> kafka --service-key kafka1 \
--create --topic kpi-input --partitions 6 --replication-factor 1 --config retention.ms=86400000
$ mzsh mzadmin/<password> kafka --service-key kafka1 \
--create --topic kpi-alarm --partitions 6 --replication-factor 1 --config retention.ms=86400000 |
|
...