Batch-Based Real-Time Agents - Agent Configuration(3.0)

Owned by Former user (Deleted)
Last updated: Apr 20, 2023 by Jenni Wallin
4 min read


This section describes how to configure the agents.

When you select one of the agents, the configurations include the same tabs that are displayed in a batch version of the configuration, with the exception of the Filename Sequence tab, which is not included as it is irrelevant in a real-time workflow. 

For details of the configuration of the agents, see the relevant sections below:

Disk(3.0)

FTP(3.0)

SCP(3.0)

SFTP(3.0)

Additional Configurations for Collection Agents

When the collection agents are included in a real-time workflow, there are also three additional tabs for each of the agent configurations: Decoder, Decompression and Execution. See the example image below.

Decoder Tab

In the Decoder tab you configure the settings related to the decoding of the collected data.

SettingDescription
Decoder

List of the decoders available for selection.

The decoders are named according to the following syntax:
<decoder> (<module>)

The option MZ Format Tagged UDRs indicates that the expected UDRs are stored in one of the built-in formats. If the compressed format is used, the decoder will automatically detect this.

Select this option to make the Tagged UDR Type list accessible for configuration. If you select this option, the Tagged UDR Type list is enabled.

Tagged UDR Type

List of available internal UDR formats stored in the Ultra and Code servers. The formats are named according to the following syntax:

<internal> (<module>)

If the decoder is to reprocess UDRs of an internal format, you must select the Decoder MZ format tagged UDRs to enable this list. Once enabled, you can select the internal format.

Full Decode

If you select this check box, the UDR is fully decoded before it is inserted into the workflow. This may have a negative impact on performance since all fields may not be accessed in the workflow, making decoding of all fields unnecessary.

If disabled (default), the amount of work required for decoding is minimized, using a "lazy" method of decoding sub fields. This means the actual decoding work may not be done until later in the workflow, when the field values are accessed for the first time.

Corrupt data (that is, data for which decoding fails) might not be detected during the decoding stage and may cause the UDR to be discarded at a later processing stage.

Decompression Tab

In the Decompression tab you specify if you want to decompress the files or not.

SettingDescription
Compression Type

Select the required decompression algorithm:

  • No Compression: The agent will not decompress the files. This is the default setting.
  • LZO: The files are compressed using LZO compression. If you select this option, the Command Line field is displayed and populated by default with the command lzop -d -c -q. This command allows you to use the lzop unix command to decompress standard input with the LZO algorithm and write the output on standard output. You can enter an alternative lzop command if required. For further information on the lzop command, see https://www.lzop.org/lzop_man.php.

Execution Tab

In the Execution tab, you configure how often the workflow will be executed, how decoding errors will be handled and what action is to be taken when Cancel Batch messages are called. For furher information on Cancel Batch, see Batch-Based Real-Time Agents - Transaction Behavior(3.0).

Note!

Cancel Batch messages are sent:

  • If a decoding error occurs, and you have selected the Cancel Batch option in the agent configuration
  • If you have selected for files to be decompressed in the Decompression tab of agent configuration and it fails, e g because a file is corrupt or is not compressed.
SettingDescription
Run OnceSelect this option if you want the workflow to run once.
Repeat Every X Seconds

Select this option if you want the workflow to be run repeatedly with an interval of a specific number of seconds. The default value is 5 seconds.

If you have selected this option, when the agent encounters an error, the workflow does not abort. The error is reported in the System Log and the agent retries at the next repeat.

Decoding Error Handling

Select one of the error handling options to control how to react upon decoding errors:

  • Cancel Batch - Processing of the current batch is stopped and skips to the next batch.

  • Route Raw Data - Route the remaining, undecodable, data as raw data. This option is useful if you want to implement special error handling for batches that are partially processed.

Abort Immediately

If enabled, the workflow immediately aborts on the first Cancel Batch message from any agent in the workflow. The erroneous data batch is kept in its original place and must be moved/deleted manually before the workflow can be started again.

Abort After X Consecutive Cancel Batch

If enabled, the value of X indicates the number of allowed Cancel Batch calls, from the collection agent before the workflow will be aborted. The counter is reset between each successfully processed data batch. Thus, if 5 is entered, the workflow will abort on the 6th data batch in a row that is reported erroneous.

Never Abort

The workflow will never abort. However, as with the other error handling options, the System Log is always updated for each Cancel Batch message.

Additional Configurations for Forwarding Agents

When the forwarding agents are included in a real-time workflow, there is also one additional tabs for each of the agent configurations: File Closing Criteria. See the example image below.

SettingDescription

(bytes)

When the file size has reached the number of bytes entered in this field, the file will be closed as soon as the current bytearray has been included, and stored in the storage directory. This means that the file size may actually be larger than the set value since will not cut off any bytearrays. If nothing is entered, this file closing criteria will not be used.

Volume (UDRs)

When the file contains the number of UDRs entered in this field, the file will be closed and stored in the storage directory. If nothing is entered, this file closing criteria will not be used.

Timer (sec)

When the file has been open for the number of seconds entered in this field, the file will be closed and stored in the storage directory. If nothing is entered, this file closing criteria will not be used.