HDFS Forwarding Agent Configuration
You open the HDFS forwarding agent configuration dialog from a workflow configuration. To open the HDFS processing agent configuration, click Build → New Configuration. Select Workflow from the Configurations dialog. When prompted to Select workflow type, select Batch. Click Add agent and select HDFS from the Forwarding tab of the Agent Selection dialog.
HDFS
HDFS forwarding agent configuration dialog - HDFS tab
Item | Description |
---|---|
Profile | Select the File System profile you want the agent to use, see File System Profile for further information about this profile. |
Input Type | The agent can act on two input types. Depending on which one the agent is configured to work with, the behavior will differ. The default input type is bytearray, that is the agent expects bytearrays. If nothing else is stated, the documentation refers to input of bytearray. If the input type is MultForwardingUDR, the behavior is different. For further information about the agent's behavior in MultiForwardingUDR input, see HDFS Forwarding Agent MultiForwardingUDR Input. |
File Information Settings | |
Directory | Enter the absolute path to the target directory on the remote file system, where the forwarded files will be stored. The files will be temporarily stored in the automatically created subdirectory |
Create Directory | Select this check box to create the directory, or the directory structure, of the path that you specify in Directory. Note The directories are created when the workflow is executed. |
Compression | Select the compression type of the target files. Determines if the agent will compress the files or not.
|
After Treatment Settings | |
Command | If a Command is supplied, it will be executed on each successfully closed temporary file, using the parameter values declared in Arguments. Please refer to Commands in Desktop User's Guide for further information. Note At this point the temporary file is created and closed, however the final filename has not yet been created. The entered Command has to exist in the execution environment, either including an absolute path, or to be found in the PATH for the execution environment. |
Arguments | This field is optional. Each entered parameter value has to be separated from the preceding value with a space. The temporary filename is inserted as the second last parameter, and the final filename is inserted as the last parameter, automatically. This means that if, for instance, no parameter is given in the field, the arguments will be as follows: If three parameters are given in the field Arguments, the arguments are set as: $1=<parameter_value_#1> $2=<parameter_value_#2> $3=<parameter_value_#3> $4=<temporary_filename> $5=<final_filename> |
Produce Empty Files | If enabled, files will be produced although containing no data. |
Filename Template Tab
The names of the created files are determined by the settings in the Filename Template tab. For a detailed description of the Filename Template tab, see Workflow Monitor.