Using Multiple Service Models

If you have two or more service models that you want to process in the same cluster simultaneously:

• Create separate KPI profiles for each kpi model. See KPI Profile.
• Create separate input, output, and alarm topics in Kafka for each service model. See Starting Clusters and Creating Topics.

• Create separate Spark application configurations for each service model. This can be done by updating the script kpi_params.sh in the folder mz_kpiapp/bin, making one entry per kpi model.

  Copy the whole section in the if-statement for "kpiapp", and at least alter these parameters:

  if [ "kpiapp" = "$1" ] ← Obviously one app per KPI Model
  export MZ_KPI_PROFILE_NAME="kpisales.SalesModel" ← The name of the profile used

  
  

  Note!

  Set the Spark application property spark.cores.max to limit the cluster resources that an application can use. 

  If you do not set this property, the first submitted Spark application will use all resources available. Other applications have to wait for the resources to be freed. 

You can use the same instance of the kpi-model service to provision all the service models.

Submit two different Spark apps.

You can define different configurations:

If it is required, you can specify the amount of memory or the number of cores that are reserved for each slave individually, by setting properties in the deployment-info block. These will override property values that you have set in the spark-environment block.

Example - Overriding properties in spark-environment block

spark-environment: { SPARK_WORKER_MEMORY : 6g SPARK_WORKER_CORES: 3 } deployment-info: { sparkmaster: {} sparkslave1: {SPARK_WORKER_MEMORY: 6} sparkslave2: {SPARK_WORKER_MEMORY: 6} }