Batch-Based Real-Time Agents - Agent Configuration(3.1)

Owned by Jenni Wallin
Last updated: Nov 18, 2024 by Yuri Au Yong


This section describes how to configure the agents.

When you select one of the agents, the configurations include the same tabs that are displayed in a batch version of the configuration, with the exception of the Filename Sequence tab, which is not included as it is irrelevant in a real-time workflow. 

For details of the configuration of the agents, see the relevant sections below:

Disk(3.1)

FTP(3.1)

SCP(3.1)

SFTP(3.1)

Additional Configurations for Collection Agents

When the collection agents are included in a real-time workflow, there are also three additional tabs for each of the agent configurations: Decoder, Decompression and Execution. See the example image below.

Decoder Tab

In the Decoder tab you configure the settings related to the decoding of the collected data. For more information on how to configure the Decoder tab, see Decoder(3.1).

Decompression Tab

In the Decompression tab you specify if you want to decompress the files or not.

SettingDescription
Compression Type

Select the required decompression algorithm:

  • No Compression: The agent will not decompress the files. This is the default setting.
  • LZO: The files are compressed using LZO compression. If you select this option, the Command Line field is displayed and populated by default with the command lzop -d -c -q. This command allows you to use the lzop unix command to decompress standard input with the LZO algorithm and write the output on standard output. You can enter an alternative lzop command if required. For further information on the lzop command, see https://www.lzop.org/lzop_man.php.

Execution Tab

In the Execution tab, you configure how often the workflow will be executed, how decoding errors will be handled and what action is to be taken when Cancel Batch messages are called. For furher information on Cancel Batch, see Batch-Based Real-Time Agents - Transaction Behavior(3.1).

Note!

Cancel Batch messages are sent:

  • If a decoding error occurs, and you have selected the Cancel Batch option in the agent configuration
  • If you have selected for files to be decompressed in the Decompression tab of agent configuration and it fails, e g because a file is corrupt or is not compressed.
SettingDescription
Run OnceSelect this option if you want the workflow to run once.
Repeat Every X Seconds

Select this option if you want the workflow to be run repeatedly with an interval of a specific number of seconds. The default value is 5 seconds.

If you have selected this option, when the agent encounters an error, the workflow does not abort. The error is reported in the System Log and the agent retries at the next repeat.

Decoding Error Handling

Select one of the error handling options to control how to react upon decoding errors:

  • Cancel Batch - Processing of the current batch is stopped and skips to the next batch.

  • Route Raw Data - Route the remaining, undecodable, data as raw data. This option is useful if you want to implement special error handling for batches that are partially processed.

Abort Immediately

If enabled, the workflow immediately aborts on the first Cancel Batch message from any agent in the workflow. The erroneous data batch is kept in its original place and must be moved/deleted manually before the workflow can be started again.

Abort After X Consecutive Cancel Batch

If enabled, the value of X indicates the number of allowed Cancel Batch calls, from the collection agent before the workflow will be aborted. The counter is reset between each successfully processed data batch. Thus, if 5 is entered, the workflow will abort on the 6th data batch in a row that is reported erroneous.

Never Abort

The workflow will never abort. However, as with the other error handling options, the System Log is always updated for each Cancel Batch message.

Additional Configurations for Forwarding Agents

When the forwarding agents are included in a real-time workflow, there is also one additional tabs for each of the agent configurations: File Closing Criteria. See the example image below.

SettingDescription

(bytes)

When the file size has reached the number of bytes entered in this field, the file will be closed as soon as the current bytearray has been included, and stored in the storage directory. This means that the file size may actually be larger than the set value since Usage Engine will not cut off any bytearrays. If nothing is entered, this file closing criteria will not be used.

Volume (UDRs)

When the file contains the number of UDRs entered in this field, the file will be closed and stored in the storage directory. If nothing is entered, this file closing criteria will not be used.

Timer (sec)

When the file has been open for the number of seconds entered in this field, the file will be closed and stored in the storage directory. If nothing is entered, this file closing criteria will not be used.