This section includes a description of the complementary Control File Collection Strategy that is applied in for the Disk, FTP, SFTP, and SCP Collection agents.

Overview

The collection strategy makes it possible to collect files for which a corresponding control file exist. If the control file does not exist, the file is ignored.

The Control File Collection Strategy controls which further configuration options that are available in the Source tab. If no strategy is selected, the default strategy is used.

Collection Strategy - Control File

Note!

The Collection Strategy drop down list will only be visible if there are other collection strategies available in the system, apart from the default collection strategy available.

Setting	Description
Collection Strategy	Select the Control File option in this list.
Directory	Enter the absolute path name of the source directory on the remote host, where the source files reside. The path name may also be entered relative to the home directory of the User Name account.
Include Subfolders	Select this checkbox to also include subfolders within the specified directory.
Filename	Enter the name of the source files on the remote host. Regular expressions according to Java syntax can be used. For further information, see http://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html. Example To match all file names beginning with `TTFILE`, enter: `TTFILE.*`
Compression	Select compression type for the source files. This selection determines if the agent will decompress the files before passing them on in the workflow. No Compression: The agent will not decompress the file(s). Gzip: The agent will decompress the file(s) using gzip.
Position	The control filename consists of an extension added either before or after the shared filename part. Select one of the choices: Prefix or Suffix. Prefix means that the text entered inte the Control File Extension field will be searched for before the shared filename part, and Suffix means that the text entered in the Control File Extension field, will be searched for after the shared filename part.
Control File Extension	The Control File Extension is used to define when the data file should be collected. A data file with filename `FILE` will only be collected if the corresponding control file exists. A possible control filename can be `FILE.ok`. The text entered in this field is the expected extension of the shared filename. The Control File Extension will be attached to the beginning or the end of the shared filename, depending on the selection made in the Position list, above.
Data File Extension	The Data File Extension will only be applicable if Position is set to Suffix. There can be cases where a more strict definition of which files should be collected is needed. This is defined in the Data File Extension field. Consider a data file called `FILE.dat`. If `.dat` is entered in the Data File Extension field the corresponding Control file will be called `FILE.ok` if .ok is entered in the Control File Extension field. Note! Consider a directory containing 5 files: FILE1.dat FILE2.dat FILE1.ok ok.FILE1 FILE1 The Position field is set to Prefix and the Control File Extension field is set to `.ok`. The control file is `ok.FILE1` and `FILE1` will be the file collected. The Position field is set to Suffix and the Control File Extension field is set to `.ok`. The control file is `FILE1.ok` and `FILE1` will be be the file collected. The Position field is set to Suffix and the Control File Extension field is set to `.ok` and the Data File Extension field is set to `.dat`. The control file is `FILE1.ok` and `FILE1.dat` will be the file collected. After collection, the control file is handled in the same way as the collected file is configured to be handled, that is the system should delete/rename/move/ignore it.
Move to Temporary Directory	If this option is selected, the source files will be moved to the automatically created subdirectory `DR_TMP_DIR` in the source directory, before collection. This option supports safe collection when source files repeatedly use the same name.
Inactive Source Warning (h)	If this option is selected, a warning message (event) will appear in the System Log and Event Area when the configured number of hours have passed without any file being available for collection: The source has been idle for more than <n> hours, the last inserted file is <file>.
Move to	If this option is selected, the source files will be moved from the source directory (or from the directory `DR_TMP_DIR` if using Move to Temporary Directory), to the directory specified in the Destination field, after collection. Note! The Destination must be located in the same file system as the collected files at the remote host. Additionally, absolute path names must be defined (relative path names cannot be used).
Rename	If this option is selected, the source files will be renamed after the collection, and remain (or moved back from the directory `DR_TMP_DIR` if using Move to Temporary Directory) in the source directory from which they were collected.
Remove	If this option is selected, the source files will be removed from the source directory (or from the directory `DR_TMP_DIR`, if using Move to Temporary Directory), after the collection.
Ignore	If this option is selected, the source files will remain in the source directory after the collection. This field is not available if Move to Temporary Directory is enabled.
Destination	If the Move to option has been selected, enter the full path name of the directory on the remote host into which the source files will be moved after the collection in this field. If any of the other After Collection options have been selected, this option will not be available.
Prefix and Suffix	If any of the Move to or Rename options have been selected, enter the prefix and/or suffix that will be appended to the beginning and/or end of the name of the source files, respectively, after the collection, in these fields. If any of the other After Collection options have been selected, this option will not be available. Note! If Rename is enabled, the source files will be renamed in the current (source or `DR_TMP_DIR` ) directory. Ensure that you do not assign a Prefix or Suffix , giving files new names that still match the Filename regular expression. That will cause the files to be collected over and over again.
Keep (days)	If any of the Move to or Rename options have been selected, enter the number of days to keep moved or renamed source files on the remote host after the collection in this field. In order to delete the source files, the workflow has to be executed (scheduled or manually) again, after the configured number of days. If any of the other After Collection options have been selected, this option will not be available. Note! A date tag is added to the filename, determining when the file may be removed.
Route FileReferenceUDR	Select this checkbox to route File Reference UDR instead of raw data.