In preparation for using KPI Management in MediationZone, a number of scripts need to be extracted and some setup is required. This is described on this page.
...
These scripts will be used for different procedures that you find in the sections for KPI Management - Distributed Processing.
Preparations before extracting scripts:
A Prerequisite is that Spark, ZooKeeper, and Kafka are installed and up and running. For more information about this, see LINK needed
Before running the command to extract the scripts, these parameters need to be set as environment variables as they will be entered into some scripts:
...
1. Set up your preferred KPI profile or use the simplified example configuration which can be found in kpi_tst.zip ← Where can we find this?
2. Find and the kpi_spark*.mzp among the installation files and copy it to a place you want to keep your KPI application files.
...
7. The next step is to modify the scripts under the bin folder according to your specifications and requirements, as they are when extracted they are to be considered as more of examples than a finished configuration.
The scripts submit.sh, kpi_params.sh and spark_common_params.sh need to be updated. In addition, the settings in spark-defaults.conf need to be added to the spark configuration file equally named.
The changes that need to be done in most cases are the following:
In kpi_params.sh:
KAFKA_BROKERS need to be configured with the hosts and ports of the kafka brokers, e.g:
...
The memory settings may need to be altered depending on the expected load, as well as the UI port for the KPI App inside Spark (default 4040).
In addition the addresses and ports of the platform, kafka and zookeeper may need to be updated.
In spark_common_params.sh:
Here all that needs to be changed is the master host ip, as well as the ports if needed. ← I don’t know what SPARK_LOCAL_IP is used for.
Submitsubmit.sh only needs to be updated if the log format is to be altered. In that case, add this parameter:
log4j_setting="-Dlog4j.configuration=/home/davids/userstories/xe10095/log4j/spark_log4j.properties"
With the path to a log4j properties file. Below is an example of this:
...