The GCP BigQuery agent is a batch forwarding agent that inserts UDR data into the BigQuery Data Warehouse in a GCP Project specified by the agent configuration, based on user-defined mappings between UDR fields and BigQuery database table columns.
A special column in the target data table is also assigned a unique Transaction ID, generated for each batch. In relation to this, a batch status table is utilized to indicate the batch status for the unique Transaction ID.
GCP BigQuery Agent Workflow
Note!
Due to the nature of BigQuery, the order of insertion of the UDR data into the BigQuery table is done out of sequence and will appear as such in the target table. Sorting will have to be done from BigQuery itself.
Note!
The GCP BigQuery batch forwarding agent is implemented using a streaming API called the tabledata.insertAll method. You will need to refer to the GCP Bigquery documentation to ensure it is supported for your BigQuery tier.
This section includes the following subsections: