Page Comparison

...

The following guide can be used to assist you when creating your unique batch scaling solution. An important thing to remember is that you cannot mix standard agents with scaling agents in the same workflows. Workflows with standard agents save the state in Usage Engine and workflows with batch agents save the state in Kafka.

Creating a scalable solution

...

Always start with a Batch Scaling Collection Workflow that collects from the original file source and forwards UDRs to Kafka.

...

example

These are high-level steps to creating a scalable batch solution in Usage Engine. The following example solution is made up of several profiles including the newly created Partition Profile (4.3), and two workflow types, Batch Scaling Collection and Batch Scaling Processing.

1. Create a Partition Profile

2. Create an Aggregation, Duplicate UDR, and Inter Workflow profile and link the Partition Profile created in Step 1 to each.

3. Then create the workflows.

   • Batch Scaling Collection Workflow

   • Batch Scaling Processing Workflow(s) - can be one or a series of workflows.

   Batch Duplication Check and Aggregation can be part of the same workflow. There can only be one Aggregation agent and one Deduplication agent per workflow.

4. Decide how many maximum workflows should execute in parallel. Think about how you can evenly distribute your data into different groups. For example….add an example here. -

...