...
- The Data table must have a Transaction ID column.
- The Batch Status table must be created with a Transaction ID column and a Status column.
- At commit and rollback stage, the Batch Status is updated with a status code reflecting the current stage that can be used for auditing.
- Consumers of the loaded data are expected to always access that data through a view.
This view should join the two tables on Transaction ID where status = 0, for example:
Info title Example - View Joining Data Table and Batch Status Table The following DDL query is used in the BigQuery Query Editor to create a view under the user_analytics Dataset with the table named view1.
Code Block CREATE VIEW IF NOT EXISTS user_analytics.view1 AS SELECT * FROM user_analytics.data_tbl1 AS t1 FULL JOIN user_analytics.batch_status_tbl1 AS t2 USING (id) WHERE t2.status = 0;
Emits
The agent emits commands that changes the state of the file currently processed.
Command | Description |
|---|---|
Cancel Batch | Emitted if any error occurs during insertion of rows into Data table or error when mapping the UDR to rows. |
Retrieves
The agent retrieves commands from other agents and based on them generates a state change of the file currently processed.
...
| Scroll ignore | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||