Overview - the Why
Support for scaling means that PE shall be able to increase processing capacity as needed, without the need of changing configuration or any manual intervention.
Batch scaling in 9.2 is not easily achieved:
Transactions are stored per WF and file.
File collection cannot scale using the same WF config (you need to have different reg exp).
Aggregation is a separate storage (file/db). Only one WF at a time can work on one Profile/storage. The only alternative option you have is sharding (additional Profiles).
Duplicate UDR is a separate storage (file/db). Only one WF at a time can work on one Profile/storage. The only alternative option you have is sharding (additional Profiles).
The solution we implemented uses Kafka for both transaction/state and working storage for aggregation and duplication data.
Mention the new agents - but don’t go into detail.
Note that you cannot mix standard agents with scaling agents in the same workflow. Workflows with standard agents save the state in PE. Workflows with batch agents save the state in Kafka.
Subsections
This section contains the following subsections:
Configuration
How it works
New agents