...
Usage Engine Private Edition now supports automatic batch scaling, making it possible to increase or decrease processing capacity as needed without any manual intervention. This will enable parallelism within batch workflows and is all about splitting a job to get it done faster. The new solution consists of two new agents, the scalable InterWF forwarder and collector, and a new profile, the partition profile. it also makes use of the existing agents, Data Aggregator and Deduplication. The solution uses Kafka for both transaction /state and working storage for aggregation and duplication data. The feature consists of several new agents that work in the same way as their existing counterparts but with an extra storage state in the profiles. These new agents cannot be grouped in the same workflows as regular agents because they have different storage needs.
Note!
Note that you cannot mix standard agents with scaling agents in the same workflow. Workflows with standard agents save the state in PE. Workflows with batch agents save the state in Kafka.
...