Aggregation, Consolidation and Correlation

The Aggregation agent can correlate, consolidate and aggregate records of different record types simultaneously, in real-time or batch-mode, based on configurable criteria. Among other things, it includes support for the following:

• Automatic creation of session on first UDR arrival

• Association of UDRs, potentially of different record types

• Script execution on association

• Configurable timeout management for sessions

• Session variables, including lists of associated records

• Storage of associated data in either memory, file or Couchbase database

• Sessions stored in a Couchbase database can be replicated to multiple servers

• Transaction safe for batch mode

• Creation of new records based on aggregated information using scripting

An Aggregation agent can simultaneously correlate and aggregate multiple data streams from multiple network elements / network technologies. Any aggregation, consolidation or correlation logic can be configured through the GUI.

An Aggregation profile is used to select the UDR types to correlate, and the fields on which to base the correlation. A rule identifies UDRs based on the content of the chosen UDR fields for each input type. One or more fields can be used to find the matching UDRs.

In the example below, records of type ApacheUDR, Radius, and Cisco Netflow are processed and matched on the fields:

• ApacheUDR.RemoteHostINet (httpd)

• Radius.FramedIPAddress

• V5UDR.SourceIP (Cisco Netflow)

The Aggregation Profile is then selected when configuring the Aggregation agent within this workflow.  

Example of an Aggregation agent in a real-time workflow

Aggregation/Correlation sessions can be viewed through the Aggregation Session Inspection GUI. The sessions can be searched for and released for further down-stream systems for further processing.  

Aggregation Flush Functionality

The APL function aggregationHintFlushSessions is available in Analysis and Aggregation APL code in Batch Workflows. It is used to indicate that the timeout function block, in the APL code for the specified Aggregation Agent, will be executed for all the sessions in the storage that have a timeout value. The timeout block execution will be performed after the drain function block has been executed in the specified Aggregation Agent. This can be useful before system upgrades or maintenance windows where it is desirable to keep temporary data at a minimum.

Couchbase Aggregation

Couchbase can be used to store aggregation session data in real-time workflows. By selecting a mirror profile you may also acquire read only access to sessions in a secondary Couchbase cluster. This is useful in various failover scenarios. The mirror session data can be extracted via APL code. Storing aggregation sessions in Couchbase improves scalability and enables sharing of data across workflows and geographic locations.