9.72.2.1 SCP Collection Agent Configuration

You open the SCP collection agent configuration dialog from a workflow configuration: you can right-click the agent icon and select Configuration..., or double-click the agent icon.

Note!

You can configure part of the parameters can be done in the Filename Sequence or Sort Order service tabs. For further information, see Filename Sequence Tab, and Sort Order Tab in 3.1.6 Workflow Template.


The Configuration view consists of the following tabs:

  • Connection

  • Source

  • Advanced

  • Security

Connection Tab

The Connection tab contains configuration settings related to the remote host and authentication.

The SCP collection agent configuration - Connection tab


SettingDescription

Server Information Provider

The standard behavior is Single Server . If your system is installed with the Multi Server File functionality, you can configure the SFTP agent to collect from more than one server. For further information, contact your System Administrator. 

Host

Primary host name or IP-address of the remote host to be connected. If a connection cannot be established to this host, the Additional Hosts, specified in the Advanced tab, are tried.

File System Type

Type of file system on the remote host. This information is used to construct the remote filenames.

  • Unix - remote host using Unix file system. Default setting.

  • Windows NT - remote host using Windows NT file system.

Authenticate With

Choice of authentication mechanism. Both password and private key authentication are supported.

Username

Username for an account on the remote host, enabling the SCP session to login.

Password

Password related to the specified Username. This option only applies when password authentication is enabled.

Private Key

The Select... button will display a window where the private key may be inserted. If the private key is protected by a passphrase, the passphrase must be provided as well. This option only applies when private key authentication is enabled. For further information, see Authentication in 9.72.1.2 SCP Agents Attributes and Authentication.

Enable Collection Retries

Select this check box to enable repetitive attempts to connect and start a file transfer.

When this option is selected, the agent will attempt to connect to the host as many times as is stated in the Max Retries field described below. If the connection fails, a new attempt will be made after the number of seconds entered in the Retry Interval (s) field described below.

Retry Interval (s)

Enter the time interval in seconds, between retries.

If a connection problem occurs, the actual time interval before the first attempt to reconnect will be the time set in the Timeout field in the Advanced tab plus the time set in the Retry Interval (s) field. For the remaining attempts, the actual time interval will be the number seconds entered in this field.

Max Retries

Enter the maximum number of retries to connect.

In case more than one connection attempt has been made, the number of used retries will be reset as soon as a file transfer is completed successfully.

Note!

This number does not include the original connection attempt.

Source Tab

The Source tab contains configurations related to the remote host, source directories and source files. The configuration available can be modified through the choice of a Collection Strategy. The following text describes the configuration options available when no custom strategy has been chosen.

The SCP collection agent configuration - Source tab


SettingDescription

Collection Strategy

If there are more than one collection strategy available in the system a Collection Strategy drop down list will also be visible. For further information about the collection strategy, see 11. Appendix 4 - Collection Strategies.

Directory

Absolute pathname of the source directory on the remote host, where the source files reside. The pathname might also be given relative to the home directory of the Username account.

Filename

Name of the source files on the remote host. Regular expressions according to Java syntax applies. For further information, see http://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html.


Example

To match all filenames beginning with  TTFILES , type:  TTFILES.* .

Compression

Compression type of the source files. Determines whether the agent will decompress the files before passing them on in the workflow or not.

  • No Compression - the agent agent will not decompress the files.

  • Gzip - the agent decompresses the files using gzip.

Move to Temporary Directory

If enabled, the source files will be moved to the automatically created subdirectory DR_TMP_DIR in the source directory, prior to collection. This option supports safe collection of a source file reusing the same name.

Append Suffix to Filename

Enter the suffix that you want added to the file name prior to collecting it.

Important!

Before you execute your workflow, make sure that none of the file names in the collection directory include this suffix.

Inactive Source Warning (h)

If enabled, when the configured number of hours have passed without any file being available for collection, a warning message (event) will appear in the System Log and Event Area:

The source has been idle for more than <n>
    hours, the last inserted file is <file>.

Move to

If enabled, the source files will be moved from the source directory (or from the directory DR_TMP_DIR, if using Move Before Collecting) after collection, to the directory specified in the Destination field. If Prefix or Suffix are set, the file will be renamed as well.

Note!

If a file with the same filename already exist in the target directory, this file will be overwritten and the workflow will not abort.

Destination

Absolute pathname of the directory on the remote host into which the source files will be moved after the collection. This field is only available if Move to is enabled.

Note!

The  Directory  has to be located in the same file system as the collected files at the remote host. Also, absolute pathnames must be defined. Relative pathnames cannot be used.

Prefix and Suffix

Prefix and/or suffix that will be appended to the beginning and/or the end, respectively, of the source files after the collection. This field is only available if Move to or Rename is enabled.

Note!

If  Rename  is enabled, the source files will be renamed in the current directory (source or  DR_TMP_DIR ). Be sure not to assign a Prefix or Suffix, giving files new names still matching the  Filename Regular Expression. That would cause the files to be collected over and over again.

Search  and  Replace

To apply Search and Replace, select either Move to or Rename.

  • Search : Enter the part of the filename that you want to replace.

  • Replace : Enter the replacement text.

Search and Replace operate on your entries in a way that is similar to the Unix  sed  utility. The identified filenames are modified and forwarded to the following agent in the workflow.

This functionality enables you to perform advanced filename modifications, as well:

  • • Use regular expression in the Search entry to specify the part of the filename that you want to extract.

  • Note!

    A regular expression that fails to match the original file name will abort the workflow.

    Enter Replace with characters and metacharacters that define the pattern and content of the replacement text.

    Search and Replace Examples

    To rename the file file1.new to file1.old, use:

    • Search.new
    • Replace.old

    To rename the file JAN2011_file to file_DONE, use:

    • Search([A-Z]*[0-9]*)_([a-z]*)
    • Replace$2_DONE

    Note that the search value divides the file name into two parts by using brackets. The replace value applies the second part by using the place holder $2.

Keep (days)

Number of days to keep moved or renamed source files on the remote host after the collection. In order to delete the source files, the workflow has to be executed (scheduled or manually) again, after the configured number of days.

Note, a date tag is added to the filename, determining when the file may be removed. This field is only available if Move to or Rename is selected.

Rename

If enabled, the source files will be renamed after the collection, remaining (or moved back from the directory DR_TMP_DIR, if using Move Before Collecting) in the source directory from which they were collected.

Remove

If enabled, the source files will be removed from the source directory (or from the directory DR_TMP_DIR, if using Move Before Collecting), after the collection.

Ignore

If enabled, the source files will remain in the source directory after the collection. This option is not available if Move Before Collecting is enabled.

Route FileReferenceUDRSelect this check box if you want to forward the data to an SQL Loader agent. See the description of the SQL Loader agent in 9.78 SQL Loader Agent  for further information.

Advanced Tab

The  Advanced  tab contains configurations related to more specific use of the SCP service.

The SCP collection agent configuration - Advanced tab


SettingDescription

Port

The port number the SCP service will use on the remote host.

Timeout (s)

The maximum time, in seconds, to wait for response from the server. 0 (zero) means to wait forever.

Accept New Host Keys

If selected, the agent overwrites the existing host key when the host is represented with a new key. The default behavior is to abort when the key mismatches.

Note!

Selecting this option causes a security risk since the agent will accept new keys regardless if they possibly belong to another machine.

Enable Key Re-Exchange

Used to enable and disable automatic re-exchange of session keys during ongoing connections. This can be useful if you have long lived sessions since you may experience connection problems for some servers if one of the sides initiates a key re-exchange during the session.

Additional Hosts

List of additional host names or IP-addresses that may be used to establish a connection. These hosts are tried, in sequence from top to bottom, if the agents fail to connect to the remote host set in their Connection tabs.

Use the AddEditRemoveMove up and Move down buttons to configure the host list.

Security Tab

The Security tab contains configurations related to the Advanced Security Options for SCP. The Configuration available can be modified by enabling the Advanced Security Option check box. If the Advanced Security Option is not enabled, the Cipher Mode will default to aes128-ctr and the HMac Type will default to hmac-sha2-256. If the Advanced Security Option is enabled but the combo box fields are left empty, the Cipher Mode will default to aes128-ctr and the HMac Type will default to hmac-sha2-256.

The SCP collection agent configuration - Security tab

Note!

Due to an upgrade of the Maverick library for   version 8.1.5.0, the default handling of the advanced security has changed. Users should take note of the behaviour change for the Advanced Security Option for the SCP agents. The Advanced Security Option will be disabled by default. Users will have to enable it on their own accord from the Security Tab in the SCP agents configuration.

With Advanced Security Option disabled, Maverick will manage the connection between the SCP agent and the server. Maverick will attempt to connect with the STRONG security level. Failing to do so, it will auto downgrade the security level to WEAK and attempt to connect, this behaviour will allow our agents to work well with backwards compatibility for servers with older instances of the Maverick library.  Furthermore, having STRONG security level will result in a performance degradation for

However, when a user manually enables the Advanced Security Option from the security tab, Maverick will instead assign the WEAK security level, which will not be as strict or resource intensive as the STRONG security level.

For more information about security levels, you can refer to this page: https://www.jadaptive.com/managed-security-in-our-java-ssh-apis/

SettingDescription
Cipher Mode

Algorithms for the Block Cipher Modes supported by the SCP agent. This allows the agent to determine which algorithm for the block cipher to be used when communicating with the servers.

  • 3des-cbc
  • 3des-ctr
  • blowfish-cbc
  • aes128-cbc
  • aes192-cbc
  • aes256-cbc
  • aes128-ctr
  • aes192-ctr
  • aes256-ctr
  • arcfour
  • arcfour128
  • arcfour256

HMac Type

Methods of encryption for Key Exchange. This allows the agent to determine the method of encryption to be used when the keys are exchanged between the servers and the SCP agent.

  • hmac-sha1
  • hmac-sha1-96
  • hmac-sha1-etm@openssh.com
  • hmac-md5
  • hmac-md5-96
  • hmac-md5-etm@openssh.com
  • hmac-sha2-256
  • hmac-sha2-256-96
  • hmac-sha2-256-etm@openssh.com
  • hmac-sha2-512
  • hmac-sha2-512-96
  • hmac-sha2-512-etm@openssh.com
  • hmac-ripemd160
  • hmac-ripemd160-etm@openssh.com