Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Current »

You open the Netia FTP agent configuration dialog from a workflow configuration. To open the Archiving forwarding agent configuration, click  Build → New Configuration. Select  Workflow in the Configurations list. When prompted to select workflow type, select Batch. Click Add agent and select NetiaFTP from the Collection tab in the Agent Selection list.

Connection Tab

The Connection tab contains configuration data that is relevant to a remote server.

netiaftp_coll_connection.png

Field

Description

Host

Enter the primary host name or IP address of the remote host to be connected. If a connection cannot be established to this host, the Additional Hosts specified in the Advanced tab, are tried.

Note!

The FTP Agent supports both IPv4 and IPv6 addresses

Username

Enter the username for an account on the remote host, enabling the FTP session to login.

Password

Enter the password for the Username entered.

Transfer Type

Select the data transfer type to be used during file retrieval.

  • Binary - The agent uses the binary transfer type. This is the default setting.

  • ASCII - The agent uses ASCII transfer type.

File System Type

Select the type of file system on the remote host.

  • Unix - The remote host uses a Unix file system. This is the default setting.

  • Windows NT - The remote host uses a Windows NT file system.

  • VAX/VMS - The remote host uses a VAX/VMS file system.

Note!

When the File System Type for VAX/VMS is selected, some issues must be considered. If a file is renamed after collection on a VAX/VMS system, the filename might become too long. In that case the following rules apply:

A VAX/VMS filename consists of <file name>.<extension>;<version>, where the maximum number of characters for each part is:

<file name>: 39 characters

<extension>: 39 characters

<version>: 5 characters

If the new filename turns out to be longer than 39 characters, the agent moves part of the filename to the extension part. If the total sum of the filename and extension part exceeds 78 characters, the last characters are truncated from the extension.

Example - How characters are truncated

A_VERY_LONG_FILENAME_WITH_MORE_THAN_39_ CHARACTERS.DAT;5

is converted to

A_VERY_LONG_FILENAME_WITH_MORE_THAN_39_. CHARACTERSDAT;5

Enable Collection Retries

Select this check box to enable repetitive attempts to connect and collect data.

Retry Interval (s)

Enter the time interval in seconds, between retries.

Max Retries

Enter the maximum number of retries to connect.

Note!

This number does not include the original connection attempt.

Note!

If a connection problem occurs the actual time interval will be the time set in the Timeout field in the Advanced tab plus the time set in the Retry Interval (s) field.

Source Tab

The Source tab contains configurations related to the remote host, source directories and source files. 

netiaftp_coll_source.png

Setting

Description

Collection Strategy

Only the Default Collection Strategy is available for the Netia FTP collection agent, so this option should always be selected in this drop-down-list.

Include Subfolders

This is not supported for the Netia FTP collection agent and selecting this checkbox will not have any effect.

Directory

The directory setting is not used for the Netia FTP collection agent. However, since this field needs to be populated for the agent to validate, simply enter "." or ““/” in this field.

Filename

This field is used for determining the name of the source file(s) on the remote host. If you have disabled the Filename Sequence settings, you have to enter the complete name of the file to be collected in this field. If you have enabled the Filename Sequence settings, enter the filename up to the point where the sequence number begins in this field. For example if the names of the files are "IX.ICAMA.<sequence number>", enter "IX.ICAMA." in this field.

Note!
Filename must be one specific filename. Multiple filenames or wildcards cannot be used.

For further information, see Filename Sequence Tab below.

Compression

Compression type of the source files. This setting determines if the agent will decompress the files before passing them on in the workflow.

  • No Compression - The agent does not decompress the files.

  • Gzip - The agent decompresses the files using gzip.

Before Collection

Move to Temporary Directory

If this option is enabled, the source files will be moved to a subdirectory called DR_TMP_DIR in the source directory, before collection. This option supports safe collection when source files repeatedly use the same name.

Append Suffix to Filename

Enter the suffix that you want to be added to the file name prior to collection.

Important!

Before you execute your workflow, make sure that none of the file names in the collection directory include this suffix.

Inactive Source Warning (h)

If enabled, when the configured number of hours have passed without any file being available for collection, a warning message (event) will appear in the System Log and Event Area:

The source has been idle for more than <n> hours, the last inserted file is <file>.

After Collection

Move to

If enabled, the source files will be moved from the source directory (or from the directory DR_TMP_DIR if using Move to Temporary Directory) to the directory specified in the Destination field, after collection.

Note!

The Directory has to be located in the same file system as the collected files. The absolute path names must be defined.

If a file with the same filename, but with different content, already exists in the target directory, the workflow will abort.

If a file with the same file name, and the same content, already exists in the target directory, this file will be overwritten and the workflow continue running.

Rename Options

Rename

If this option is enabled, the source files will be renamed after the collection, and remain (or moved back from the directory DR_TMP_DIR if using Move to Temporary Directory) in the source directory.

Note!

When the  File System Type for VAX/VMS is selected, there are special considerations. If a file is renamed after collection on a VAX/VMS system, the filename might become too long. In that case, the following rules apply:

A VAX/VMS filename consists of <file name>.<extension>;<version>, where the maximum number of characters for each part is:

  • <file name>: 39 characters

  • <extension>: 39 characters

  • <version>: 5 characters

If the new filename turns out to be longer than 39 characters, the agent will move part of the filename to the extension part. If the total sum of the filename and extension part exceeds 78 characters, the last characters are truncated from the extension.

  An example:

  A_VERY_LONG_FILENAME_WITH_MORE_THAN_39_ CHARACTERS.DAT;5

  will be converted to:

  A_VERY_LONG_FILENAME_WITH_MORE_THAN_39_. CHARACTERSDAT;5

Note!

Creating a new file on the FTP server with the same file name as the original file, but with other content, will cause the workflow to abort.

Creating a new file with the same file name AND the same content as the original file will overwrite the file.

Remove

If enabled, the source files will be removed from the directory (or from the directory DR_TMP_DIR, if using the Move to Temporary Directory option, after collection.

Ignore

If enabled, the source files will remain in the source directory after the collection. This field is not available if the Move to Temporary Directory option is enabled.

Destination

Enter the full pathname to the directory on the remote host into which the source files will be moved after the collection. This field is only available if Move to is enabled.

Prefix and Suffix

Prefix and/or suffix that will be appended to the beginning and the end of the name of the source files, respectively, after the collection. These fields are only available if Move to or Rename is enabled.

Warning!

If  Rename is selected, the source files will be renamed in the current (source or  DR_TMP_DIR ) directory. Be sure not to assign a  Prefix or Suffix , giving files new names still matching the  Filename  regular expression. That will cause the files to be collected over and over again.

Search and Replace

Select either the Move or Rename option.

  • Search: Enter the part of the filename that you want to replace.

  • Replace: Enter the replacement text.

Search and Replace operate on your entries in a way that is similar to the Unix sed utility. The identified filenames are modified and forwarded to the target agent in the workflow.

This functionality enables you to perform advanced filename modifications, as well:

Use regular expression in the Search entry to specify the part of the filename that you want to extract.

Note!

A regular expression that fails to match the original file name will abort the workflow.

  • Enter Replace with characters and meta characters that define the pattern and content of the replacement text

Search and Replace Examples

To rename the file file1.new to file1.old, use:

  • Search: .new

  • Replace: .old

To rename the file JAN2011_file to file_DONE, use:

  • Search: ([A-Z]*[0-9]*)_([a-z]*)

  • Replace: $2_DONE

Note that the search value divides the file name into two parts by using parentheses. The replace value applies to the second part by using the place holder $2.

Keep (days)

Enter the number of days to keep moved or renamed source files on the remote host after collection. In order to delete the source files, the workflow has to be executed (scheduled or manually) again, after the configured number of days.

Note!

A date tag is added to the filename, determining when the file may be removed. This field is only available if the Move to or Rename option is enabled.

Advanced Tab

The Advanced tab contains configurations related to the use of the FTP service.

For example if the FTP server used does not return the file listed in a well defined format, the Disable File Detail Parsing option can be useful, see Disable File Detail Parsing.

netiaftp_coll_advanced.png

Setting

Description

Command Port

The value in this field defines which port number the FTP service will use on the remote host.

Timeout (s)

The maximum time, in seconds, to wait for a response from the server. 0 (zero) means to wait forever.

Passive Mode (PASV)

This setting must be enabled if FTP passive mode is used for data connection.

In passive mode, the channel for data transfer between client and server is initiated by the client instead of by the server. This is useful when a firewall is situated between the client and the server.

Disable File Detail Parsing

This setting disables parsing of file detail information received from the FTP server. This enhances the compatibility with unusual FTP servers but disables some functionality.

If file detail parsing is disabled, file modification timestamps are unavailable to the collector. The collector does not have the ability to distinguish between directories and simple files. For that reason, subdirectories in the input directory must not match the filename regular expression. The agent assumes that a file named DR_TMP_DIR is a directory because a directory named DR_TMP_DIR is used when Move to Temporary Directory in the Source tab is activated. Therefore, you cannot name a regular file in the collection directory DR_TMP_DIR.

Additional Hosts

List of additional host names or IP addresses that may be used to access the source directory, from which the source files are collected. These hosts are tried, in sequence from top to bottom, if the agent fails to connect to the remote host, set in the Connection tab.

Note!

The FTP Agent supports both IPv4 and IPv6 addresses.

Use the Add, Edit, Remove, Move up and Move down buttons to configure the host list. 

Filename Sequence Tab

The Filename Sequence settings are used when you want to collect files containing a sequence number in the file name. The sequence number is expected to be found on a specific position in the file name and will have a fixed or dynamic size. 

Note!

When collecting from several sources, the Filename Sequence settings are applied on data arriving from all the sources, as though all the information arrived from a single source. This means that the Filename Sequence Number count is not specific to any of the sources.

netiaftp_coll_fileseq.png

Setting

Description

Enable Filename Sequence

This setting determines if the service will be used or not. If the Filename Sequence settings are enabled, a list with possible files ending with the range of numbers defined by the Wrap On and Wrap To values will be generated before collection. Otherwise, a list with only one file with the name stated in the Source tab will be generated.

Note!

If the Filename Sequence settings are enabled, the workflow will collect files in the sequence until a file cannot be found. The workflow will then stop. The next time the workflow is started, it will continue to collect where it stopped last, that is if the last collected file ended with number 121, the workflow will start to collect the file ending with 122, the next time it is executed. This may result in a slower startup of the workflow.

Start Position

The offset in the file name where the sequence number starts. The first character has offset 0 (zero). In the example file name TTFILE0001 the start position is 6.

Length

The length of the sequence number, if the sequence number has a static length (padded with leading zeros). If the sequence number length is dynamic this value is set to 0 (zero).

Example - Length setting

TTFILE0001-TTFILE9999 Length: 4

TTFILE1-TTFILE9999 Length: 0

Wrap On

If the Filename Sequence service is enabled, a value must be specified on which the sequence will wrap. This number must be larger than the value entered for Wrap To.

Wrap To

Enter the value that the sequence will wrap to. This value must be less than the value entered for Wrap On.

Warn On Out of Sequence

If you select this check box, the agent will log an informative message to the System Log when detecting an out of sequence file, before deactivating.


  • No labels