9.78.1 SQL Loader Agent Overview

Owned by Jenni Wallin
Last updated: May 13, 2023 by Former user (Deleted)

The SQL Loader agent is a batch processing agent designed to populate the database with data from existing files, either residing in a local directory or on the server filesystem of the database.

Data can be collected using either the Disk, FTP, or SFTP agents, and supported databases are; MySQL, Netezza, PostgreSQL SAP HANA, and Sybase IQ. 

Workflow Configuration 

 

Workflow with the SQL Loader agent

The Disk, FTP and SFTP agents now have an additional check box called Route FileReferenceUDR  in their configuration dialogs:

 

This check box should be selected when using the SQL Loader agent. 

The SQL Loader agent then forwards an SQLLoaderResultUDR containing information about loaded file, number of inserted rows, execution time and any error messages, for logging purposes. 

UDR Types

The UDR types used in combination with the SQL Loader agent are FileReferenceUDR and SQLLoaderResultUDR:

FileReferenceUDR

The FileReferenceUDR is the UDR format used to send data from the collection agent to the SQL Loader agent.

The following fields are included in the FileReferenceUDR: 

FieldDescription
directory (string)

This field states the name of the directory the data file is located in. 

filename (string)

This field states the name of the file data should be collected from. 

fullpath (string)

This field states the full path to the data file.

OriginalData (long)

This field contains the original data in byte array format. 

SQLLoaderResultUDR 

The SQLLoaderResultUDR is the UDR that the SQL Loader agent sends out after having loaded the data into the database. This UDR contains information for logging purposes that should be handled by another agent in the workflow. 

The following fields are included in the SQLLoaderResultUDR: 

FieldDescription
errorMessage (string)
This field contains any error message that might have been returned during the loading of data. 

executionTime (long)

This field indicates the time it took to load the data into the database.
filename (string)
This field contains the name of the file from which data was uploaded.
rowsAffected (long)
This field indicates the number of affected rows.
OriginalData (long)
This field contains the original data in byte array format.
Next:

9.78.2 SQL Loader Agent Configuration