4.1.2 Data Hub Forwarding Agent Transaction Behavior

Owned by Jenni Wallin
Last updated: May 15, 2023 by Former user (Deleted)
1 min read

This section describes the transaction behavior of the Data Hub agent. For more information about general  transactions, see, Transactions, in /wiki/spaces/MZD73/pages/5605083.

Emits

This agent does not emit anything.

Retrieves

The agent retrieves commands from other agents. Based on these commands the agent changes the state of the processed local file.

Command

Description

Begin Batch

When a Begin Batch message is received a local temporary file is created in MZ_HOME/tmp.

ConsumeWhen a consume message is received the agent will write the incoming data to the local temporary file.

Commit

When a Commit message is received, the agent will send the local temporary file to the staging directory in Impala. Once the file has been transferred, the agent will insert the transferred data into a temporary database table. The agent then copies the data in the temporary table to the selected database table and removes all temporary files and tables.

If the agent is recovering from an error, the agent will only remove the local temporary file.

RollbackIf a Rollback message is received, the agent will remove the local temporary file.
Next:

4.1.3 Data Hub Forwarding Agent Input/Output Data and MIM