Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 9 Next »

To open the real-time Disk-Deprecated forwarding agent configuration dialog from a workflow configuration, you can do one of the following:

  • right-click the agent icon and select Configuration...
  • double-click the agent icon

Disk_Deprecated Tab

General Tab

Disk-Deprecated forwarding agent - General tab


SettingDescription
Input Type

The agent can act on two input types:

  • bytearray (default)
  • MultForwardingUDR


If nothing else is stated the documentation refers to input of bytearray.

If you select MultiForwardingUDR, the behavior of the agent will be different. For further information, see 15.3.2.2 Disk Forwarding MultiForwardingUDR Input - Real-Time.

File Information

SettingDescription
File Information
Directory

Enter the path of the target directory on the local file system of the EC, where the forwarded files will be stored. The path can be absolute or relative to the $MZ_HOME environment variable.

Create Directory

Select this check box to create the directory path that you specify in Directory.

The directories are created when the workflow is executed.

CompressionSelect Gzip to decompress files before collection and insertion into the workflow. If the collected files are not compressed, select No Compression.

File Closing Criteria

When any of the following criteria is fulfilled, the current file will be closed and an output file will be generated.

The time and volume counters that are evaluated against the criteria are reset each time a batch is closed.

Enter 0 (zero) in a field to disable a criterion.

SettingDescription
Volume (bytes)

Enter a file closing criterion based on the processed UDR volume in bytes. The size of the output file will not exceed the value in this field unless it is set to 1 (one byte).

Volume (UDRs)

Enter a file closing criterion based on the number of processed UDRs.

Timer (sec)Enter a file closing criterion based on time.

File Synchronization Criteria 
When any of the following criteria is fulfilled, the agent will synchronize a disk buffer with the batch data that is stored the in-memory. The synchronized data is stored in a subdirectory named DR_TMP_DIR under the target directory.

If an error occurs that causes the workflow to abort, the synchronized data will be processed at restart of the workflow.
The time and volume counters that are evaluated against the criteria are reset each time synchronization occurs.
Enter 0 (zero) in a field to disable a criterion.
SettingDescription
Volume (bytes)

Enter a synchronization criterion based on the processed UDR volume in bytes.

Volume (UDRs)Enter a synchronization criterion based on the number of processed UDRs.
Interval (sec)Enter the length of synchronization interval in this field.

Advanced Tab

Disk-Deprecated forwarding agent - Advanced tab

Each batch is handled by a separate worker process with its own processing queue. If the queue size or maximum number of concurrent batches is exceeded, the agent immediately sends the received data to error handling by routing an ErrorUDR to the workflow. This UDR contains the original data and error information.

If a worker fails to write or move data, the agent will retry the last operation for a configurable number of times. As a last resort, it will send the data to error handling. Data that is received while the agent is attempting to recover from an error is immediately sent to error handling.

For more information about error handling, see Closing Batches from APL and Error Handling.

Use the settings below to define the worker queue size and how the agent should behave when error conditions occur.

SettingDescription
Concurrent Batches Limit

Enter the number of open batches that can be handled by the agent.

Use the value zero (0) for an unlimited number of batches.

It is recommended to use a non-zero when there is a high throughput of small batches, e g containing a single UDR. This will prevent out-of-memory errors.

Worker Queue Limit

Enter the maximum number of incoming UDRs that can be queued by worker processes.

Error Buffer Limit

The error buffer contains data will be routed to the workflow via ErrorUDR types in case of failures. 

Enter the maximum number of bytearray UDRs that can be stored in the buffer, and in the routed UDR.

Error Buffer Timeout (sec)Enter the maximum time in seconds that worker processes should wait for new data, assuming that the error buffer is not empty, before routing an ErrorUDR to the workflow.
Retries on FailureEnter the maximum number of times worker processes should attempt synchronization on failure, e g when writing to the file stream.
Retries for Moving FilesEnter the maximum number of times that worker processes should attempt to move the synchronized data in DR_TMP_DIR to the target destination. 
Recovery Attempt Interval (sec)Enter the number of seconds between retry attempts. 
Orphan File SuffixEnter a suffix that should be appended to the target file names when the agent recovers temporary files after a workflow restart. 

FileName Template Tab

The names of the created files are determined by the settings in the Filename Template tab. For a detailed description of the Filename Template tab, see 3.1.6 Workflow Template.


  • No labels