...
The following guide can be used to assist you when creating your unique batch scaling solution. An important thing to remember is that you cannot mix standard agents with scaling agents in the same workflows. Workflows with standard agents save the state in Usage Engine and workflows with batch agents save the state in Kafka.
Creating a scalable solution (example)
These are high-level steps to creating a scalable batch solution in Usage Engine. The following example solution is made up of several profiles including the newly created Partition Profile (4.3) and Scalable Inter Workflow Profile (4.3), and two workflow types, Batch Scaling Collection and Batch Scaling Processing.
...
Note!
When creating a scalable workflow you need to add the Kafka profile in the execution tab of the workflow properties.
UI Parameters
...
Parameter
...
Comment
...
ID Field
...
Defines how to match a UDR to a partition.
...
.
...
Number of partitions, which is the same as maximum number of workflows that can execute in parallel. - this means that there can be fewer workflows - but not more.
Note!
If any of the parameters need to be changed, it is considered a new configuration, and they need to start with empty topics.
Scaling Batch Workflows
Usage Engine will scale out and in and re-balance scalable batch workflows automatically and you can schedule when to start a scale-out or scale-in.
Deploying a scale-out configuration with ECDs:
Use the regular https://infozone.atlassian.net/wiki/x/IgMkEg definition using with Dynamic Workflows to define how to package a scale-out. See the tabs on https://infozone.atlassian.net/wiki/x/VgQkEg to define how to package a scale-out. You need to define when these ECDs will activate.
...
for more information.
You can schedule the ECD and workflow to start or stop at specific times, alternativelyor, these can also be started manually. See the above links for more information.
...
Automatic Scaling
...
you can start them manually.
Scaling option | Description |
|---|---|
Automatic Scaling | Based on a configured metric, for example, scaling starts when more than X number of UDRs are received for processing and stops when the received UDRs drop below this number. |
You can start up ECDs manually.
Manual Scaling | There are two options for manual scaling;
|