SCP Collection Agent Configuration
- Kevin Wu
You open the SCP collection agent configuration dialog from a workflow configuration. To open the Amazon S3 collection agent configuration, click Build → New Configuration. Select Workflow from the Configurations dialog. When prompted to Select workflow type, select Batch. Click Add agent and select SCP from the Collection tab of the Agent Selection dialog.
Part of the configuration may be done in the Filename Sequence or Sort Order tab described in Workflow Template.
The Configuration view consists of the following tabs:
Connection
Source
Advanced
Security
Connection Tab
The Connection tab contains configuration settings related to the remote host and authentication.
The SCP collection agent configuration - Connection tab
Setting | Description |
|---|---|
Connection Information | |
Host | Primary host name or IP-address of the remote host to be connected. If a connection cannot be established to this host, the Additional Hosts, specified in the Advanced tab, are tried. |
File System Type | Type of file system on the remote host. This information is used to construct the remote filenames.
|
Authentication Mechanism | |
Authenticate With | Select the authentication mechanism to use. Password and Private Key modes are supported. |
Username | Enter the username for an account on the remote host, enabling the SCP session to log in. |
Password | Enter the associated password. This option only applies when password authentication is enabled. |
Private Key | The Select... button will display a window where the private key may be inserted. If the private key is protected by a passphrase, the passphrase must be provided as well. This option only applies when private key authentication is enabled. For further information, see Authentication in SCP Agents Attributes and Authentication. |
Collection Retries | |
Enable | Select this check box to enable repetitive attempts to connect and start a file transfer. When this option is selected, the agent will attempt to connect to the host as many times as is stated in the Max Retries field described below. If the connection fails, a new attempt will be made after the number of seconds entered in the Retry Interval (s) field described below. |
Retry Interval (s) | Enter the time interval in seconds, between retries. If a connection problem occurs, the actual time interval before the first attempt to reconnect will be the time set in the Timeout field in the Advanced tab plus the time set in the Retry Interval (s) field. For the remaining attempts, the actual time interval will be the number of seconds entered in this field. |
Max Retries | Enter the maximum number of retries to connect. In case more than one connection attempt has been made, the number of used retries will be reset as soon as a file transfer is completed successfully. Note!This number does not include the original connection attempt. |
Source Tab
The Source tab contains configurations related to the remote host, source directories, and source files. The configuration available can be modified through the choice of a Collection Strategy. The following text describes the configuration options available when no custom strategy has been chosen.
The SCP collection agent configuration - Source tab
Setting | Description |
|---|---|
Collection Strategy | If there is more than one collection strategy available in the system a Collection Strategy drop-down list will also be visible. For further information about the collection strategy, see Appendix 4 - Collection Strategies. |
File Information | |
Directory | Enter the absolute pathname of the source directory on the remote host, where the source files reside. The pathname might also be given relative to the home directory of the Username account. |
Filename | Enter the name of the source files on the remote host. Regular expressions according to Java syntax apply. For further information, see http://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html. ExampleTo match all filenames beginning with |
Compression | Select the compression type of the source files. Determines whether the agent will decompress the files before passing them on in the workflow or not.
|
Before Collection | |
Move to Temporary Directory | If enabled, the source files will be moved to the automatically created subdirectory |
Append Suffix to Filename | Enter the suffix that you want to be added to the file name prior to collecting it. Important!Before you execute your workflow, make sure that none of the file names in the collection directory include this suffix. |
Inactive Source Warning (h) | If enabled, when the configured number of hours have passed without any file being available for collection, a warning message (event) will appear in the System Log and Event Area: The source has been idle for more than <n> hours, the last inserted file is <file>. |
After Collection | |
Move to | If enabled, the source files will be moved from the source directory (or from the directory |
Destination | Enter the absolute pathname of the directory on the remote host into which the source files will be moved after the collection. This field is only available if Move to is enabled. |
Prefix and Suffix | Enter the Prefix and/or suffix that will be appended to the beginning and/or the end, respectively, of the source files after the collection. This field is only available if Move to or Rename is enabled. |
Search and Replace | Select this option if you want to apply the Search and Replace function. Select either the Move to or Rename setting.
Search and Replace operate on your entries in a way that is similar to the Unix sed utility. The identified filenames are modified and forwarded to the following agent in the workflow. This functionality enables you to perform advanced filename modifications, as well:
|
Keep (days) | Enter the number of days to keep moved or renamed source files on the remote host after the collection. In order to delete the source files, the workflow has to be executed (scheduled or manually) again, after the configured number of days. Note, a date tag is added to the filename, determining when the file may be removed. This field is only available if Move to or Rename is selected. |
Rename | If enabled, the source files will be renamed after the collection, remaining (or moved back from the directory |
Remove | If enabled, the source files will be removed from the source directory (or from the directory |
Ignore | If enabled, the source files will remain in the source directory after the collection. This option is not available if Move to Temporary Directory is enabled. |
UDR Type | |
Route FileReferenceUDR | Select this check box if you want to forward the data to an SQL Loader agent. See the description of the SQL Loader agent in SQL Loader Agent for further information. |
Advanced Tab
The Advanced tab contains configurations related to a more specific use of the SCP service.
The SCP collection agent configuration - Advanced tab
Setting | Description |
|---|---|
Advanced Settings | |
Port | Enter the port number the SCP service will use on the remote host. |
Timeout (s) | Enter the maximum time, in seconds, to wait for a response from the server. 0 (zero) means to wait forever. |
Accept New Host Keys | If selected, the agent overwrites the existing host key when the host is represented with a new key. The default behavior is to abort when the key mismatches. |
Enable Key Re-Exchange | This option enables or disables automatic re-exchange of session keys during ongoing connections. This can be useful if you have long-lived sessions since you may experience connection problems for some servers if one of the sides initiates a key re-exchange during the session. |
Additional Hosts | |
Additional Hosts | This option allows additional host names or IP addresses to be used during the connection establishment. These hosts are tried, in sequence from top to bottom, if the agents fail to connect to the remote host set in their Connection tabs. Use the Add, Edit, Remove, Up, and Down buttons to configure the host list. |
Security Tab
The Security tab contains configurations related to the Advanced Security Options for SCP. The Configuration available can be modified by enabling the Advanced Security Option check box. If the Advanced Security Option is not enabled, the Cipher Mode will default to aes128-ctr and the HMac Type will default to hmac-sha2-256. If the Advanced Security Option is enabled but the combo box fields are left empty, the Cipher Mode will default to aes128-ctr and the HMac Type will default to hmac-sha2-256.
The SCP collection agent configuration - Security tab
Setting | Description |
|---|---|
Advanced Security Settings | |
Enable Advanced Security Option | If enabled, this will enable the Cipher Mode and HMac Type options below, allowing for advanced security configuration. |
Cipher Mode | Select the algorithms for the Block Cipher Modes. This allows the agent to determine which algorithm for the block cipher to be used when communicating with the servers.
|
HMac Type | Select the encryption methoc for Key Exchange. This allows the agent to determine the method of encryption to be used when the keys are exchanged between the servers and the SCP agent.
|