Building Blocks
UFDL uses the following building blocks:
Block | Description |
---|---|
External formats | An external format describes the physical layout of the raw data. It can be declared in multiple ways such as through an |
Internal formats | An internal format describes the structure of the internal, working UDR type. These formats are used in workflow activities (such as APL processing) and are declared through an |
In-map | A mapping from an external type to an internal type. This is the basic component used to describe a decoder, and it is declared through the Note! Mapping between different Ultra formats may work, but is not supported, for example, between xml format and ASN.1 formats. Instead you need to add an Analysis agent to execute the conversion. |
Out-map | A mapping from an internal type to an external type. This is the basic component used to describe an encoder, and it is declared through the Note! Mapping between different Ultra formats may work, but is not supported, for example, between xml format and ASN.1 formats. Instead you need to add an Analysis agent to execute the conversion. |
Decoder | A decoder uses one or more in-maps to specify a set of decoding rules to be used when decoding input data. Blocking specification as well as other file structure specifications can also be added. |
Encoder | An encoder uses one or more out-maps to specify a set of encoding rules to be used when decoding input data. Blocking specifications can also be added. |
As an example, consider a incoming call detail record containing information on the type of call, who made the call and which number was dialed. That means that the record contains three fields. Two fields and new termination marks (;
) for each field in the record in order to conform to the expected output format must be added.
Concerning both incoming and outgoing external formats, each record (the last field in the record) is terminated by a newline character.
An example of two external format definitions
As can be seen in the image above, An example of two external format definitions, an external specification is used to describe the raw data formats.
The UDR format used by the processing agents is generated through the target_internal
specification. This format is automatically generated based on the incoming external data and includes any fields in the internal
specification of the in-map.
In this case, the generated internal format consists of the fields from the external incoming format with two additional fields. Hence, we introduce an internal
containing the new fields only.
Relation between declared and generated internal formats
Finally, the in_map
and out_map
definitions are used when creating decoder and encoder definitions to be used.
UFDL code of the example is described as follows:
external recA { int type : static_size(1); ascii Anum : static_size(8); ascii Bnum : terminated_by(0xA); }; internal recA_int { int field1; int field2; }; external recB { int type : terminated_by(";"); ascii Anum : terminated_by(";"); ascii Bnum : terminated_by(";"); int field1 : terminated_by(";"); int field2 : terminated_by(0xA); }; in_map A_map : external(recA), internal(recA_int), target_internal(recA_TI) { automatic; }; decoder decode_A : in_map(A_map); out_map B_map : internal(recA_TI), external(recB) { automatic; }; encoder encode_B : out_map(B_map);
External
Network elements delivering data to the system produce records in various physical formats. An external format describes the physical structure of the incoming data, as well as the data delivered by the system. It can be any type of ASCII, XML, or binary data. Typically, one external block per record type is defined. File headers and trailers are also considered to be records.
The external formats are described through a number of different methods with different syntax. The external formats, currently supported by the
Platform, are:Sequential formats: ASCII or binary based on static or dynamic sizes for both records and fields (see External - Sequential Format). The generic external specifications are defined via the
external
block.Ericsson IOG/IN formats: (see External - Ericsson IOG/IN Records). This special format type is described through the
inw
variant of theexternal
block.ASN.1 based formats: A subset of ASN.1 is supported (see External - ASN.1 Formats). These formats can be used to specify either BER or PER encodings.
XML formats: A subset of the standard XML schema syntax is supported and can be used to handle different XML formats. For further information, see XML Schema Support.
Records of different format types may exist in the same record. An example of this is a BER (ASN.1) record that contains a field whose format is a sequential record (specified through an external
block). The image below, A definition of a sequential external record, illustrates the syntax for a sequential record type.
A definition of a sequential external record
Internal and In-Map
There are two ways to define an internal UDR format in UFDL. Either by using the automatic generation through an in_map
specification, or through an internal
declaration block. For more complex formats it is often impractical to manually write an internal format if all that is desired is a direct mapping from the fields in the external format. In these cases the target_internal
specification in the in_map
is used. An additional internal
block can be specified in the in_map
if additional processing fields are needed.
The in_map
manages the mapping of external data into the internal UDR structure. An external
must be defined, and one or both of the internal
and target_internal
options must be used to specify the internal format to map to.
For information about how to specify an internal
format, see Internal Formats. For information about in_map
specifications, see In-maps.
Example - An internal type declaration
internal AXE_DAM { string ANumberAreaCode; string ANumberClass : optional; string BR_IncomingPQR; string BR_NumberClass; intLen ANumber; intLen BRNumber; intLen BSNumber; string PreProcRecordClass; };
An internal type declared through target_internal
is automatically generated based on the fields of the external
and internal
formats.
The target_internal
is the format used by the processing agents. It can be a direct representation of the external incoming format, or a combination of the external and additional fields from an internal definition. An internal definition is needed in case any changes to the default mapping from the external format are required or if several externals are mapped to the same UDR.
To understand the structure of the generated target_internal
format, it is necessary to understand how the automatic mapping of an in_map
is performed. When automatically mapping an external format (that is, using the automatic
keyword), all external fields are mapped to an internal field unless the external format specifies a different automatic mapping behavior (for example through the external_only
keyword).
The automatic in-map generation performs the following steps for each external field:
If the field has an explicit mapping specified, use that mapping (explicit field mapping specifications overrides automatic mapping).
If there is a field in the
internal
format with the same name as in theexternal
, the external format field is mapped to the internal field.Otherwise, a new field is created in the generated internal format with the same name as the external field, and a type is chosen according to the external format (the default mapping type).
If the automatic mapping algorithm requires new fields, a new internal type is first created. This type gets the name specified in the target_internal
, and is a subtype of the internal
type (if specified). Decoders using this in_map
then produce records of that internal type.
The external field default mapping type depends on the external format specification and if the external field is of a constructed type (a list or record type), the automatic mapping logic is applied recursively to any sub-formats.
Other than automatic mappings, it is possible to specify explicit type mappings, where the corresponding external and internal fields are specified directly.
A resulting target_internal
In the image above, a resulting target_internal
, a combination of explicit (the e:
and i:
specifications), and automatic mapping are used to create the resulting target_internal
CDR_TI
.
Event Types
Event format types are special internal types that can be sent to the Event Server. Events store additional internal information used during the event processing. For performance reasons, and in order to avoid confusion, event types should only be used as “message UDRs” dispatched to the Event Server. In other words, do not use event UDRs to transport ordinary UDR data.
Event types are declared in the same way as internal
types, except for the keyword event
, which is used instead of internal
. For further information, see Internal Formats.
The following code results in the events shown in the image below, Event format showing added fields:
event AMA_Event { int anum; int bnum; ... }
Event format showing added fields
For further information about Event handling, see the Desktop User's Guide.
Decoder
A Decoder defines what external records to accept, and describes any additional file structure information, such as whether file blocking is used in the input data.
A decoder
definition consists of in_maps
or other decoders. The most basic decoder definition uses the single in-map argument. In the following example, myDecoder
appears in the configuration of a Decoder agent:
Example - Decoder
The decoder myDecoder
is created by taking one in-map. The incoming data is blocked in 2048 bytes, with the block padding 0x00
.
decoder myDecoder : in_map( my_in_map ), block_size(2048), terminated_by(0x00);
A decoder definition can consist of several in-maps, that is, the data stream received may contain several record type definitions. Another possible definition is combining several decoders, called constructed decoder.
The image below, Two ways of defining a decoder, introduces some validation logic to the incoming stream of data.
Two ways of defining a decoder
The first case, TTFile1
, requires a batch to start with a header, end with a trailer, and contain any number of UDRs in between. The second decoder definition (TTFile2
) ignores the order and occurrence of arriving records. For further information about decoder specifications, see Decoders.
Out-map
An out-map defines how the internal data is mapped to the outgoing format. It requires an internal and an external definition as arguments and maps fields either automatically or through explicit field mappings.
The minimum required to define an out-map
In the image above, The minimum required to define an out-map, it is shown how an external
Access_Reject_Ext
is created using the internal
Access_Reject_Int
. The automatic
keyword map all fields with the same name, while field names present in the internal
, but not the external
, are ignored. For further details about how to specify an out_map
see Out-maps.
Encoder
Encoders define the rules for mapping internal UDRs to external data structures (raw data).
An encoder
must be defined in order to make any out_map
usable, that is, selectable from the dialog. The most basic definition has one out_map
as its only argument. In the following example, myEncoder
appears in the configuration dialog of an Encoder agent:
Example - Encoder
encoder myEncoder : out_map( my_out_map );
Similar to a decoder, an encoder takes one or several out-maps to define the structure of the outgoing data. For detailed information about encoder specifications, see Encoders.