Data Hub Forwarding Agent Transaction Behavior

This section describes the transaction behavior of the Data Hub agent. For more information about general transactions, see, Transactions, in Workflow Monitor.


This agent does not emit anything.


The agent retrieves commands from other agents. Based on these commands the agent changes the state of the processed local file.



Begin Batch

When a Begin Batch message is received a local temporary file is created in MZ_HOME/tmp.

ConsumeWhen a consume message is received the agent will write the incoming data to the local temporary file.


When a Commit message is received, the agent will send the local temporary file to the staging directory in Impala. Once the file has been transferred, the agent will insert the transferred data into a temporary database table. The agent then copies the data in the temporary table to the selected database table and removes all temporary files and tables.

If the agent is recovering from an error, the agent will only remove the local temporary file.

RollbackIf a Rollback message is received, the agent will remove the local temporary file.