Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The A batch scaling solution requires you to use a combination of new agents and some existing agents that have had configuration setting updates. Three The key things to note are:

  • All storage related to the new agents is in Kafka. This includes transactions and backend working storage.

  • Workflows containing these new agents can be scaled without any configuration changes. The system will automatically handle the re-balancing of which workflows use which partition.

  • You cannot mix standard InterWF Inter WF agents with scaling InterWF Inter WF agents in the same workflow. Workflows This is because workflows with standard Inter workflow WF agents save the state in PE. Workflows Usage Engine and workflows with batch agents save the state in Kafka.

...

Key agents used in scalable workflows

Agent nameName

Description

Scalable Inter Workflow WF Collection Agentinclude image

Scalable_IW_Collection_agent_icon.pngImage Added

This is a newly created agent specifically for batch scaling workflow groupsworkflows. It functions similarly and has the same configuration as the https://infozone.atlassian.net/wiki/x/fRgkEgagent. The key difference is…standard Inter Workflow Collection Agent.

Scalable Inter Workflow WF Forwarding Agentinclude image

Scalable_IW_Forwarding_agent_icon.pngImage Added

This is a newly created agent specifically for batch scaling workflow groupsworkflows. It functions similarly and has the same configuration as the https://infozone.atlassian.net/wiki/x/uBckEg agent. The key difference is…

File Collection Agent

Collects files from a local file system and inserts them into a workflow. Initially, the source directory is scanned for all files matching the current filter. See the following for more information on File Collection agents, https://infozone.atlassian.net/wiki/x/PoIkEg https://infozone.atlassian.net/wiki/x/GYkkEg https://infozone.atlassian.net/wiki/x/BYQkEg https://infozone.atlassian.net/wiki/x/6JwkEg https://infozone.atlassian.net/wiki/x/MRwkEg https://infozone.atlassian.net/wiki/x/Y44kEg https://infozone.atlassian.net/wiki/x/5pAkEg https://infozone.atlassian.net/wiki/x/IrkkEg.

Aggregation Agent

standard Inter Workflow Forwarding Agent.

Aggregation Agent

Aggregation_agent_icon.pngImage Added

A new Kafka storage option has been added to the existing Aggregation Agent. When setting up your batch scaling solution, you must add this storage option to the agent configuration. Some use cases may need to switch to the new profile. See, Aggregation Agent(4.3). , see Storage Tab in https://infozone.atlassian.net/wiki/x/chwcEg.

Duplicate UDR Agent

du_udr_agent_icon.pngImage Added

A new Kafka storage option has been added to the existing Duplicate UDR Agent. When setting up your batch scaling solution, you must add this storage option in the agent configuration. See Duplicate UDR(4.3) Storage Tab in https://infozone.atlassian.net/wiki/x/yYkkEg.

Subsections

This section contains the following subsections:

Child pages (Children Display)
depth1
allChildrentrue
style
sortAndReverse
first0

...

Questions:

Can agents other than those in the table above be used in a scaling WF?

In the aggregation agent description, we advise that some use cases may require the user to switch to the new profile. when would this need to happen?

...