Function Blocks for Agents in Batch Workflows

Owned by Yeok Hong Ng
Apr 26, 2024
2 min read

The function blocks described in this section can only be used in Python agents in batch workflows. The available function blocks are:

  • beginBatch

  • drain

  • endBatch

  • endBatchHinted

  • cancelBatch

  • splittingBatch

  • commit

  • rollback

beginBatch

Note!

This function block only applies for the Python processing agent.

This function block is executed at the beginning of each batch.

def beginBatch()

Example - beginBatch function block



def beginBatch(): debug('beginBatch called')



drain

Note!

This function block only applies for the Python processing agent.

This function block is called before the current batch ends. The agent must flush all internal buffers to make sure all pending data has been processed before the transaction is ended. This method is the last point in the batch processing where the agent is permitted to route data.

def drain()



endBatch 

This function block is executed at the end of each batch.

endBatchHinted

This function block is called when another agent has called hintEndBatch. The collection agent may choose to ignore this method, that is, not to implement it if it cannot be supported.

cancelBatch

This function block is executed if a Cancel Batch is emitted anywhere in the workflow.

splittingBatch

This function block is called when the collection agent has split the input batch. If the agent keeps internal buffers to be flushed differently depending on the nature of the transaction, this method serves as a hint to the drain call.

commit

This function block is executed for each batch when the transaction is successful. If the commit block fails with an exception, it will be executed again until it succeeds.

rollback

This function block is executed for a batch if it fails.