UFDL uses the following building blocks:

As an example, consider a incoming call detail record containing information on the type of call, who made the call and which number was dialed. That means that the record contains three fields. Two fields and new termination marks (;) for each field in the record in order to conform to the expected output format must be added.

Concerning both incoming and outgoing external formats, each record (the last field in the record) is terminated by a newline character.

Image Added

As can be seen in the image above, An example of two external format definitions, an external specification is used to describe the raw data formats.

The UDR format used by the processing agents is generated through the target_internal specification. This format is automatically generated based on the incoming external data and includes any fields in the internal specification of the in-map.

In this case, the generated internal format consists of the fields from the external incoming format with two additional fields. Hence, we introduce an internal containing the new fields only.

Image Added

Finally, the in_map and out_map definitions are used when creating decoder and encoder definitions to be used.

UFDL code of the example is described as follows:

external recA { 
    int      type : static_size(1); 
    ascii Anum : static_size(8); 
    ascii Bnum : terminated_by(0xA);
};

internal recA_int { 
   int field1; 
   int field2; 
};

external recB { 
   int   type   : terminated_by(";"); 
   ascii Anum   : terminated_by(";"); 
   ascii Bnum   : terminated_by(";"); 
   int   field1 : terminated_by(";"); 
   int   field2 : terminated_by(0xA); 
}; 

in_map A_map : external(recA), 
               internal(recA_int), 
               target_internal(recA_TI) { 
                 automatic; 
               };

decoder decode_A : in_map(A_map);

out_map B_map : internal(recA_TI), 
                external(recB) { 
                   automatic; 
                };
encoder encode_B : out_map(B_map);

External

Network elements delivering data to the system produce records in various physical formats. An external format describes the physical structure of the incoming data, as well as the data delivered by the system. It can be any type of ASCII, XML, or binary data. Typically, one external block per record type is defined. File headers and trailers are also considered to be records.

The external formats are described through a number of different methods with different syntax. The external formats, currently supported by the Image Added Platform, are:

Records of different format types may exist in the same record. An example of this is a BER (ASN.1) record that contains a field whose format is a sequential record (specified through an external block). The image below, A definition of a sequential external record, illustrates the syntax for a sequential record type.

Image Added

There are two ways to define an internal UDR format in UFDL. Either by using the automatic generation through an in_map specification, or through an internal declaration block. For more complex formats it is often impractical to manually write an internal format if all that is desired is a direct mapping from the fields in the external format. In these cases the target_internal specification in the in_map is used. An additional internal block can be specified in the in_map if additional processing fields are needed.

The in_map manages the mapping of external data into the internal UDR structure. An external must be defined, and one or both of the internal and target_internal options must be used to specify the internal format to map to.

For information about how to specify an internal format, see Internal Formats. For information about in_map specifications, see In-maps.

internal    AXE_DAM { 
  string ANumberAreaCode; 
  string ANumberClass : optional; 
  string BR_IncomingPQR; 
  string BR_NumberClass; 
  intLen ANumber; 
  intLen BRNumber; 
  intLen BSNumber; 
  string PreProcRecordClass; 
 }; 

The target_internal is the format used by the processing agents. It can be a direct representation of the external incoming format, or a combination of the external and additional fields from an internal definition. An internal definition is needed in case any changes to the default mapping from the external format are required or if several externals are mapped to the same UDR.

To understand the structure of the generated target_internal format, it is necessary to understand how the automatic mapping of an in_map is performed. When automatically mapping an external format (that is, using the automatic keyword), all external fields are mapped to an internal field unless the external format specifies a different automatic mapping behavior (for example through the external_only keyword).

The automatic in-map generation performs the following steps for each external field:

If the automatic mapping algorithm requires new fields, a new internal type is first created. This type gets the name specified in the target_internal, and is a subtype of the internal type (if specified). Decoders using this in_map then produce records of that internal type.

The external field default mapping type depends on the external format specification and if the external field is of a constructed type (a list or record type), the automatic mapping logic is applied recursively to any sub-formats.

Other than automatic mappings, it is possible to specify explicit type mappings, where the corresponding external and internal fields are specified directly.

Image Added

In the image above, a resulting target_internal, a combination of explicit (the e: and i: specifications), and automatic mapping are used to create the resulting target_internal CDR_TI.

Event Types

Event format types are special internal types that can be sent to the Event Server. Events store additional internal information used during the event processing. For performance reasons, and in order to avoid confusion, event types should only be used as  “message UDRs”  dispatched to the Event Server. In other words, do not use event UDRs to transport ordinary UDR data.

Event types are declared in the same way as internal types, except for the keyword event, which is used instead of internal. For further information, see Internal Formats.

The following code results in the events shown in the image below, Event format showing added fields:

event AMA_Event {

      int  anum;
      int  bnum;
      ...
}   

Image Added

For further information about Event handling, see the /wiki/spaces/MD82/pages/3768690.

Decoder 

A Decoder defines what external records to accept, and describes any additional file structure information, such as whether file blocking is used in the input data.

A decoder definition consists of in_maps or other decoders. The most basic decoder definition uses the single in-map argument. In the following example, myDecoder appears in the configuration of a Decoder agent:

decoder myDecoder : in_map( my_in_map ),
block_size(2048), 
terminated_by(0x00);

The image below, Two ways of defining a decoder, introduces some validation logic to the incoming stream of data.

Image Added

The first case, TTFile1, requires a batch to start with a header, end with a trailer, and contain any number of UDRs in between. The second decoder definition (TTFile2) ignores the order and occurrence of arriving records. For further information about decoder specifications, see Decoders.

Out-map

An out-map defines how the internal data is mapped to the outgoing format. It requires an internal and an external definition as arguments and maps fields either automatically or through explicit field mappings.

Image Added

In the image above, The minimum required to define an out-map, it is shown how an external Access_Reject_Ext is created using the internal Access_Reject_Int. The automatic keyword map all fields with the same name, while field names present in the internal, but not the external, are ignored. For further details about how to specify an out_map see Out-maps.

Encoder

Encoders define the rules for mapping internal UDRs to external data structures (raw data).

An encoder must be defined in order to make any out_map usable, that is, selectable from the dialog. The most basic definition has one out_map as its only argument. In the following example, myEncoder appears in the configuration dialog of an Encoder agent:

encoder myEncoder : out_map( my_out_map );