Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

You open the ADLS2 file forwarding agent configuration dialog from a workflow configuration by either right-clicking the agent icon and select selecting Configuration..., or double-clicking the agent icon.

ADLS2

...

File Tab

The ADLS2 File tab contains contains the General and Advanced tab.

General Tab

Image RemovedImage Added

ADLS2 file File forwarding agent configuration dialog - ADLS2 file General tab

SettingDescription

Profile

Select the Azure profile you want the agent to use, see 8.4 Azure Profile for further information about this profile.

Container Name

Enter the name of the container where the files will be collected from. The container name can be found in the Containers section of the storage account that has been configured in the Azure Profile.

Info
titleExample

An example container name.


Input Type

The agent can act on two input types. Depending on which one the agent is configured to work with, the behavior will differ.

The default input type is bytearray, that is the agent expects bytearrays. If nothing else is stated the documentation refer to input of bytearray.

If the input type is MultForwardingUDR, the behavior is different. For further information about the agent's behavior in MultiForwardingUDR input, see 9.2.3.2 ADLS2 File Forwarding MultiForwardingUDR Input

Directory

Absolute The absolute pathname of the target directory on the location stated in the referenced File System profile, where the forwarded files will be stored.

The files will be temporarily stored in the automatically created subdirectory DR_TMP_DIR, in the target directory. When an End Batch message is received, the files are moved from the subdirectory to the target directory.

Create Directory

Select this setting to create the directory, or the directory structure, of the path that you specify in the  Directory field.

Note
titleNote!

The directories are created when the workflow is executed.


Compression

Compression type of the target files. Determines if the agent will compress the files or not.

- No Compression - The agent does not compress the files. Default setting.

- Gzip - The agent compresses the files using gzip.

Note
titleNote!

No extra extension will be appended to the target filenames, even if compression is selected. The configuration of the filenames is managed in the Filename

Template  tab

Template tab only.


Command

If a UNIX command is supplied, it will be executed on each successfully closed temporary file, using the parameter values declared in the Arguments field.

Note
titleNote!

At this point the temporary file is created and closed, however, the final filename has not yet been created.

The entered command has to exist in the execution environment, either including an absolute path, or to be found in the PATH for the execution environment.


Arguments

This field is optional. Each entered parameter value has to be separated from the preceding value with a space.

The temporary filename is inserted as the second last parameter, and the final filename is inserted as the last parameter, automatically. This means that if, for instance, no parameter is given in the field, the arguments will be as follows:

$1=<temporary_filename>   $2=<final_filename>

If three parameters are given in the field Arguments, the arguments are set as:

$1=<parameter_value_#1> $2=<parameter_value_#2> $3=<parameter_value_#3> $4=<temporary_filename> $5=<final_filename>

Produce Empty Files

If enabled, files will be produced although containing no data.

Advanced Tab

Image Added

ADLS2 File forwarding agent configuration dialog - Advanced tab

SettingDescription
Block Size (MB)

The size of each block to be staged. It determines the number of requests that are required.

Note
titleNote!

Entering a large block size will cause the upload to make fewer network calls. While each call will send more data, this will cause the call to take longer.


Max Concurrency

The maximum number of parallel requests to be provided at any given time. The value entered will apply to each API.

Max Single Upload Size (MB)

The maximum size to upload a single batch of files. If the size of the data entered is less than or equal to this value, the data will be uploaded in a single batch instead of several batches. 

If the data is uploaded as a single batch, the Block Size will be ignored.

Filename Template Tab

For a detailed description of the Filename Template tab, see Filename Template Tab in 3.1.6 Workflow Template.

...