Preparing and Creating Scripts for KPI Management

Owned by Kevin Wu
Last updated: May 28, 2024
3 min read

In preparation for using KPI Management, you need to extract the following scripts:

The scripts are as follows:

  • flush.sh

  • kpi_params.sh

  • spark_common_param.sh

  • start_master_workers.sh

  • stop.sh

  • submit.sh

These scripts will be used for different procedures in the KPI Management - Distributed Processing sections.

Preparations before extracting scripts:

A Prerequisite is that Spark, ZooKeeper, and Kafka are installed. Zookeeper and Kafka should be up and running as well. For more information about this, see KPI Management - External Software.

Before running the command to extract the scripts, these parameters need to be set as environment variables as they will be entered into some scripts:

export KAFKA_BROKERS="127.0.0.1:9092"
export SPARK_UI_PORT=4040 
export MZ_PLATFORM_AUTH="mzadmin:DR-4-1D2E6A059AF8120841E62C87CFDB3FF4"
export MZ_KPI_PROFILE_NAME="kpi_common.SalesModel"
export MZ_PLATFORM_URL="http://127.0.0.1:9036"
export ZOOKEEPER_HOSTS="127.0.0.1:2181"
export SPARK_HOME=opt/spark-3.3.2-bin-hadoop3-scala2.13
export KAFKA_HOME=/opt/kafka_2.13-3.3.1
export $PATH=$SPARK_HOME/bin:$KAFKA_HOME/bin:$PATH

Extracting scripts and KPI app: