Versions Compared

Version Old Version 34 New Version 35
Changes made by

Felicity Borrelli

Felicity Borrelli

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

These are high-level steps to creating a scalable batch solution in Usage Engine. The following example solution is made up of several profiles including the newly created Partition Profile (4.3) and Scalable Inter Workflow Profile (4.3), and two workflow types, Batch Scaling Collection and Batch Scaling Processing.

  1. Choose your scaling factor, this will be the MAX no. WFs maximum number of workflows that can effectively cooperate to process a batch. This is an important choice and will be difficult to change once your WFs workflows are in production.

Note

Warning!
Try to pick a Max Scale Factor that is divisible by many other numbers, like 6 or 12. You need to ensure that it is high enough to handle the data coming in, but not so high that you will overload resources.

  1. You must choose one or more fields in your UDRs that will be used to partition data. These fields may be based on a record group like a customer ID or an account number.

  2. Create a Partition Profile where you define your Max Scale Factor and your partitioning fields.

  3. Create the Aggregation, Duplicate UDR, and Scalable Inter Workflow profiles and link the Partition Profile created in Step 2 to each.

  4. Create your workflows.

    • Standard WFs workflows to prep data for scaling by sending it to scalable IWF forwarderInterWF Forwarder

    • Scalable processing WFs workflows start with a Scalable IWF InterWF Collector.

Note!
When creating a scalable workflow you need to add the Kafka profile in the execution tab of the workflow properties.

...