HDFS Forwarding Agent Transaction Behavior

Owned by Kevin Wu
Jan 03, 2024
1 min read

The transaction behavior for the HDFS forwarding agent is presented here. For more information about general transaction behavior please refer to the section Transactions in Workflow Monitor. 

The agent does not send out anything.

Retrieves

The agent retrieves commands from other agents and based on them generates a state change of the file currently processed.

Command

Description

Begin Batch

When a Begin Batch message is received, the temporary directory DR_TMP_DIR is first created in the target directory, if not already created. Then a target file is created and opened in the temporary directory.

End Batch

When an End Batch message is received, the target file in DR_TMP_DIR is first closed and then the Command, if specified in After Treatment, is executed. Finally, the file is moved from the temporary directory to the target directory.

Cancel Batch

If a Cancel Batch message is received, the target file is removed from the DR_TMP_DIR directory.