ADLS2 File Collection Agent(3.0)

The ADLS2 file collection agent collects files from an Azure Data Lake Storage Gen2 using the access credentials defined in an Azure profile and inserts them into a workflow. Initially, the source location is scanned for all files matching the current filter. In addition, the Filename Sequence and Sort Order settings may be used to further manage the matching of files, although they may not be used at the same time since it will cause the workflow to abort. All files found will be fed one after the other into the workflow.

When a file has been successfully processed by the workflow, the agent offers the possibility of moving, renaming, removing or ignoring the original file. The agent can also be configured to keep files for a set number of days. In addition, the agent offers the possibility of decompressing compressed (gzip) files after they have been collected. When all the files are successfully processed, the agent stops to await the next activation, whether it is scheduled or manually initiated.

The section contains the following subsections:

• ADLS2 File Collection Agent Configuration(3.0)
• ADLS2 File Collection Agent Transaction Behavior(3.0)
• ADLS2 File Collection Agent Input/Output Data and MIM(3.0)
• ADLS2 File Collection Agent Events(3.0)