Management of Batch Collection Workflow States
Kafka Batch Workflow Behavior
A batch workflow containing a Kafka collection agent will remain in a running state and wait for more data, even if there is no more data to consume. This is a major difference compared to how batch workflows behave for other collection agents, which will stop when the data from the last start cycle has been consumed, see Workflow Monitor for more information about workflow execution states.
Batch Completion Conditions
A batch will be completed if either of the following occurs:
The configured Batch Size has not been reached
5 seconds have passed since the batch started
A Kafka collection workflow will only stop automatically if it aborts due to errors.
Stopping the Workflow
A workflow stop can also be triggered by evaluating MIMs like Workflow Throughput and Estimated Lag - where no lag means that there is no data left to process.
In other circumstances, you can stop it via:
Desktop
mzsh/mzcli.
the operational REST interface.