Decoder(4.0)

The Decoder agent converts raw data to internal UDRs, based on information from a UFDL decoder definition or one of the built-in formats; MZ Tagged, JSON or CSV.

This is a processing agent for Batch workflow configurations.

The Decoder agent's configuration contains the following settings:

SettingDescription

Decoder

List of available decoders created in the Ultra Format Editor, as well as the default built-in decoders:

  • CSV Format
  • JSON Format
  • MZ Tagged Format

Different options will be available depending on which decoder you select.

On Error

This setting controls how the workflow should react upon decoding errors.

  • Cancel Batch - The entire batch is cancelled. This is the default behavior.

  • Route Raw Data - The remaining, undecodable, data is routed as raw data. This option is useful if you want to implement special error handling for batches that are partially processed.

Full Decode

This option is only available when you have selected a decoder created in the Ultra Format Editor.

If you have selected this option, the UDR will be fully decoded before being sent out from the decoder agent. This option may have a negative impact on performance, since not all fields may be accessed in the workflow, making decoding of all fields in the UDR unnecessary. If it is important that all decoding errors are detected, you must select this option.

If you have not selected this option, the amount of work needed for decoding is minimized using "lazy" decoding of field content. This means that the actual decoding work might not be done until later in the workflow, when the field values are accessed for the first time. In this case, corrupt data (that is, data for which decoding fails) might not be detected during the decoding stage, but can cause a workflow to abort at a later processing stage.

MZ Tagged Specific Settings

Tagged UDR type

Here you can select any of the available internal UDR formats stored in the Ultra and Code servers that you want the Decoder to send out.

JSON Specific Settings
UDR TypeSelect the UDR type you want the Decoder to send out. You can either select one of the predefined UDRs or the DynamicJsonUDR, which allows you to add a field of type any for including payload.
Unmapped FieldsIf you have selected the DynamicJsonUDR as UDR Type, you can select the option data in this field in order to include payload. If you have selected another UDR type that contains an any, or a map field, you can select to put any unmapped fields into the field you select in this list. All fields of any or map type in the selected UDR type will be available. If set to (None), any unmapped fields will be lost.
Schema PathEnter the path to the JSON schema you want to use in this field.
CSV Specific Settings
UDR TypeSelect the UDR type you want the Decoder to send out. You can either select one of the predefined UDRs or the DynamicCsvUDR if the CSV format is not known.
FormatSelect the CSV format you want to use; Unix, Mac, Windows, or Excel, or select to define your own customized format. If you select Custom, the following four settings will be enabled.
DelimiterEnter the delimiter character(s) for the fields in the CSV.
Use QuoteSelect this option if quotes are used in the CSV.
QuoteIf Use Quote is selected, enter the type of quotes used in the CSV.
LIne BreakEnter how line breaks are stated in the CSV.

Note!

The use and settings of private threads for an agent, enabling multi-threading within a workflow, are configured in the Thread Buffer tab that is available in batch processing and batch forwarding agents.

Input/Output Data

Input Data

bytearrays

Output Data

Selected predefined UDRs.

Transaction Behavior

The transaction behavior describes what commands the Decoder agent emits and retrieves that will alter the state of the file that is being processed.

Emits

The Decoder agent emits the following commands:

Command

Description

Cancel Batch

This command is emitted when the Decoder agent fails to decode the received data.

Retrieves

The Decoder agent retrieves the following commands from other agents:

Command

Description

End Batch

Unless this has been generated by a Hint End Batch message, the Decoder agent evaluates that all the data in the batch was decoded. When using a constructed or blocked decoder, the decoder does additional validation of the structural integrity of the batch.

Debug Events

The Decoder agent produces the following debug events:

  • Splitting batch

    Emitted when the Hint End Batch event occurs.