is capable of handling various Usage Detail Record (UDR) formats provided that their structure is known. Vendor specific UDR formats are often complex and hard to work with in agents that will view or manipulate data. Therefore, converts the external UDR formats to an internal representation, and vice versa when UDRs are leaving the system.

In order to make this conversion, both internal and external format definitions have to be defined. When the definitions exist, there must be a mapping description that maps each external field of interest to an internal field.

Ultra does not require a one-to-one relationship between external and internal field names. Normally, only fields of interest, or those that are needed when leaving the system, are declared in the internal format definition. Often the internal definition holds additional fields that are initially empty and then populated on the way through a workflow.  does not constrain how these fields are used.

 introduces a language called Ultra Format Definition Language (UFDL) that is used when describing UDR formats and mappings.

Apart from describing external formats, internal formats and mappings, the UFDL requires separate decoder and encoder specifications. A decoder or encoder refers to one or several mappings that are used for the actual translation.

Format Storage

In order to operate on UDRs, there must be a compiled version (Java class) of the internal format definition available in the Code or Ultra server. Such Java classes are introduced into the servers in two different ways:

1. Format definitions entered in the Ultra Format Editor are compiled and inserted in the Ultra server when saved.
    

2. Precompiled format definitions (for instance; Radius, NetFlow, and SNMP protocols) are inserted in the Code server during package commit.

A format can be recompiled while workflows using it are running. In this case, the changes to the format will take effect the next time the workflow is activated.

All Java classes generated by the Ultra server are kept in the system. This is to make UDRs, for instance stored in Data Veracity, using the old format version reprocessable. 

Built-in Formats

There four different built-in external formats that can be used between workflows; MZ Tagged, MZ Tagged (compressed), JSON, and CSV. These can be used by the Decoder and Encoder agents as well as by the real-time collection agents for Inter Workflow, TCP/IP and GTP'.

MZ Tagged Formats

The MZ Tagged format is available in all supported agents, while the MZ Tagged (compressed) format is only available in the Encoder agent. Using the compressed format will reduce the size of the UDRs significantly. However, since compression will require more CPU, you should consider the trade off between I/O and CPU when choosing encoder. No Decoder or Encoder needs to be defined in Ultra to handle these formats. 

The MZ Tagged formats contain header data that is based on the time zone settings of the host on which it was generated. This is important to consider when you compare output from identical workflows that are running on hosts with different time zone settings. The binary data may differ due to the header content.

JSON Format

The JSON Format is available in the Decoder, Inter Workflow, TCP/IP, and GTP' agents. With this built-in format you can also choose to use a DynamicJsonUDR where you can add the payload to the UDR. The format is validated according to the specified JSON schema.

CSV Format

The CSV Format is available in the Decoder, Inter Workflow, TCP/IP, and GTP' agents. The following predefined CSV formats are supported; Unix, Mac, Windows, Excel, but you can also create your own custom format.

Decoding of an UDR

The Ultra engine is designed to decode only the necessary parts of an Usage Detail Record, UDR, to keep workflow performance as high as possible. By default (that is, provided that  Full Decode is not selected), the decoder will only decode enough to calculate the size of the UDRs. The UDR content will be decoded as the field values are accessed. Hence, if there is a decoding error, this may not be discovered by the decoder however by the agent accessing a specific field.