Workflow Bridge Agents
This section describes the Workflow Bridge agents. The agents are used for fast collection and forwarding of data between workflows. The agents can be part of both batch and real-time workflows. The Workflow Bridge forwarding agent is listed among the processing agents in Desktop while the Workflow Bridge collection agent is listed among the collection agents.
The Workflow Bridge agents act as a bridge for communication between real-time workflows, or between a batch and a real-time workflow, within the same MediationZone system.
The Workflow Bridge agents do not use any storage server to manage the data. Data is instead stored in memory cache when executing on the same EC, or streamed directly from one agent to another, over TCP/IP, when executing on different ECs. This provides efficient transfer of data, especially from batch to real-time workflows.
The forwarding and collection workflows communicate by using a dedicated set of UDRs:
Data is sent from a forwarding workflow to a collection workflow in a
ConsumeCycleUDR. Refer to ConsumeCycleUDR for further information.In the
ConsumeCycleUDRthere are also fields that enable broadcasting and load balancing. Broadcasting, i e sending the same UDR to several different workflow instances in the collecting workflow, can be made to a configurable number of workflow instances. Load balancing enables you to configure to which workflow instance each UDR should be sent, based on criteria of your choice.
Note!
If any other UDR than ConsumeCycleUDR is routed to the forwarding agent, the bridge will only support one collector, which means that it will not be possible to broadcast or load balance.
Each state of a forwarding workflow is sent in a separate
WorkflowStateUDR to the real-time collection agent. The batch forwarding workflow sends all states from initialize to deinitialize, while a real-time forwarding workflow only sends the initialize state. The deinitialize state is sent by the real-time collection workflow if the connection goes down between the collection workflow and a forwarding workflow (batch or real-time).For more information regarding the workflow execution states, see Workflow Monitor.
User defined action UDRs can be sent from the collection workflow to communicate actions back to the forwarding workflow. For further information, see User Defined Action UDRs.
In the collecting workflow, the APL has to be configured to communicate responses for the UDRs to the Workflow Bridge collection agent. When both the forwarding and collecting workflows are real-time workflows, only responses for WorkflowState UDRs have to be configured. However, when the forwarding workflow is a batch workflow, responses for ConsumeCycle UDRs have to be configured as well.
Responses for WorkflowState UDRs are always communicated back to the forwarding workflow. If you want to communicate responses for ConsumeCycle UDRs back to the forwarding workflow as well, the Send Reply Over Bridge option has to be selected in the Workflow Bridge profile, see Workflow Bridge Profile for further information.
Workflow Bridge has two essential features; Session Context and Bulk Forwarding.
Session Context
By using a session context, arbitrary data can be stored and managed by the Workflow Bridge real-time collection agent during the whole transaction. This is achieved by populating the SessionContext field in the InitializeCycleUDR or BeginBatchCycleUDR with the data, which is then included in all subsequent UDRs after "initialize" or "begin batch". The field is cleared at "deinitialize", when the connection to the forwarding workflow goes down.
Note!
The SessionContext field is only writable in the InitializeCycleUDR and BeginBatchUDR and the session context is only available in a collection workflow.
Refer to Workflow Bridge UDR Types for more information about the Workflow Bridge UDR types.
Bulk Forwarding of Data
To enhance performance, it is possible to collect and send data in a bulk from the forwarding agent. When the data bulk has been received by the Workflow Bridge real-time collection agent, it is unpacked and forwarded as separate UDRs by the agent.
The bulk is created by the Workflow Bridge forwarding agent after a configured number of UDRs has been reached, or after a configured timeout. This is specified in the Workflow Bridge profile, see Workflow Bridge Profile for more information.
Bulking of data can only be performed for data being sent in the ConsumeCycleUDR and not for the states that are sent in the state-specific UDRs.
Bulk forwarding is not performed when the Workflow Bridge agents are executing on the same EC.
Transaction Safety
The Workflow Bridge collection workflow is a real-time workflow and as such, the transaction safety will have to be handled with APL code. The forwarding workflow execution states are transferred as Workflow State UDRs from the Workflow Bridge forwarding Agent, which enables the collection workflow to take different actions depending on the execution state of the forwarding workflow.
The Workflow Bridge collection agent requires that all Workflow State UDRs are returned before the next Workflow State UDR is forwarded into the workflow. The only exception to this is that the collection agent accepts to have several ConsumeCycleUDRs outstanding at the same time. However, all ConsumeCycleUDRs must have been returned before the next type of Workflow State UDRs is forwarded. If the forwarding workflow is a batch workflow, this means that all ConsumeCycleUDRs must be returned before the DrainCycleUDR is forwarded into the workflow.
For more information on the workflow execution states, see Workflow Monitor.
Broadcasting and Load Balancing
The same UDR can be sent to multiple workflow instances in the collection workflow using broadcasting. If the forwarding workflow has many workflow instances as well, all the UDRs sent from these instances can also be sent to all the instances in the collecting workflow.
Load balancing can be used to direct different UDRs to different workflow instances using a static load-balancing strategy (see Workflow Bridge Profile Configuration General Tab). Each workflow instance in the collection workflow is assigned a LoadId by adding this field to the workflow instance table in the Workflow Properties and then entering specific Ids for each instance. In the APL code, it is possible to determine specific UDRs that can have the forwarding workflow routed to a designated LoadId.
You configure the number of workflow instances you want to send UDRs to in the Workflow Bridge profile.
Transport
You can select the best transport for your needs. See also the Transport setting in Workflow Bridge Profile Configuration General Tab.
Option | Description |
|---|---|
Transport | Select the transport protocol that you prefer for the best performance. |
Queue Behavior
Two queues sit between the sending workflow thread and the receiving workflow's own processing: a send queue on the forwarder side, sized by forwardingQueueSize (set on the profile's Advanced tab), and the receiving workflow's own queue. Back-pressure works by blocking: when the receiving side falls behind, threads wait rather than data being dropped. Of the two, only the send queue can produce an error; the receiving queue backs up silently.
The forwarder builds one connection, and therefore one send queue and one writer thread, per collector. Every capacity setting in the profile applies to each connection separately, so with Number of Collectors set to 4 and forwardingQueueSize set to 1000, there is room for 4000 UDRs across all connections, not 1000.
Same EC and across EC are different paths, not the same path with more delay. Within one Execution Context, the writer thread hands a UDR to the receiving queue itself. Across Execution Contexts, the UDR also passes through network buffers and a bulk handler if Bulk Size is above 0, before the collector's own network thread picks it up. Both the network buffers and the bulk handler are only created for the cross-EC path.
Queue | Capacity | When full |
|---|---|---|
Send queue | forwardingQueueSize (default 1000), per collector connection | The workflow thread waits up to forwardingQueueTimeout, then fails with "The forwarding queue is full." and stops the workflow. |
Receiving workflow's queue | that workflow's own queue size setting | Waits with no time limit. No error, nothing dropped. |
Because the send queue is the last of the two to fill, a queue-full error can point at forwardingQueueSize even when the receiving side is the actual bottleneck. Increasing forwardingQueueSize only delays when the error appears. It does not address a slow collector.
Troubleshooting
Use the following to work out which part of the bridge is behind when a workflow is running slowly or timing out.
Symptom or message | Likely meaning |
|---|---|
"The forwarding queue is full." | The send queue filled and the wait configured in |
Far more UDRs sitting at the collector than | This is expected (not an error). |
Forwarding Queue Utilization stays low during an overload | This is expected (not an error), not a sign the send queue is empty. The writer thread keeps draining the queue as fast as it can, so this counter reads low right through the overload it is meant to help diagnose. |
Note!
To find the real bottleneck, check the receiving workflow's own queue-full counter (MIM get_route_entry_queue_full_count_rt) rather than the bridge's own Forwarding Queue Utilization value.