Error Correction System Description

Owned by Jenni Wallin
Feb 24, 2023
4 min read

The Error Correction System (ECS) is used for storage of UDRs and files that fail validation. Data in ECS can be tagged to enable identification, sorting and reprocessing. The ECS Forwarding Agent is used to send data to ECS, and the ECS Collection Agent is used for reprocessing stored data in a workflow.  

There are two predefined error cases – Error Codes – for UDRs that do not match any of the configured  session matching criteria for Aggregation, and for duplicate UDRs. A user can also define own Error Codes. 

Below a workflow sending data to ECS is shown.  

Workflow with ECS forwarding agent

ECS Inspector

The ECS Inspector is used to inspect and maintain UDRs and batches located in ECS. Examples of allowed operations:   

  • Define Error Codes and Reprocessing Groups. In order to be possible to collect for reprocessing, data in ECS must be tagged with a Reprocessing Group. 
  • Map Error Codes to Reprocessing Groups for automatic tagging during workflow processing. 
  • Set Reprocessing Group manually. 
  • Set Reprocessing State Manually. 
  • Change data.  
  • Define which data that cannot be changed. 
  • Define filters for easy search and view of data sets. 
  • Delete data. 

ECS Inspector

Error Search and Examination 

The Search ECS UI has a rich set of search criteria. The selections can be saved as a Filter for future use.  

ECS Search criteria 

The result of the search is displayed in the ECS Inspector window, where clicking on a row – which represents one entry / UDR – displays the full UDR content. 

Manual Error Correction 

To manually change a UDR, display the UDR by clicking the row in the ECS Inspector. It is also possible to select several UDRs and make a “bulk” change. There are two ways of making bulk changes: 

  • The Matcher Configuration of one or more fields to find the records that should be edited and then the Assignment Configuration to set the values in one or more fields as shown in the figure.  This option will show the results on each record processed by the edit – the change or the indication of no change. 
  • Manual APL script that is written into a pop-up window. This is typically used when the predefined operations are not enough to cover the edit. 

ECS bulk edit

Any changes to a UDR that is made using the UDR File Editor is logged in the System Log as a user event and includes the user id, date and time of change as well as the ECS ID for the modified UDR. 

Error Classification and Grouping 

Any UDR can be associated with error codes – and optionally any additional message/information – using APL. For instance, if the UDR is validated for user Id, missing or invalid Id can result in Error Codes “INVALID_ID” or “MISSING_ID”. In case the Id is invalid, you may want to add the value of the current id as the optional message/information, which will make it easy to view in the ECS Inspector. 

Error Codes can be automatically assigned to re-processing groups to facilitate corrective actions such as collection from ECS to a correction workflow. 

ECS Search and Update Tuning 

The ECS Search and Update functionality can be tuned:  

  • Maximum number of ECS entries that are displayed in GUI simultaneously (default value 500). 
  • Maximum number of ECS entries that can be updated at one time (default value 15,000,000) with for example Bulk Edit.

Reprocessing of Erroneous UDRs 

Data in ECS can be collected for reprocessing using the ECS Collection Agent. There are two prerequisites for collection of data from ECS; it must belong to a Reprocessing Group, and the state must be “New”. Note that collected data will change state to “Reprocessed” but will not be deleted immediately. The deletion is performed by the automatic Task “ECS Maintenance” (the time interval is configurable). Thus, if a user wants to collect data again, the state has to be changed back to “New” from the ECS Inspector. 

Workflow with ECS collection and forwarding agents 

The example above shows a reprocessing workflow that collects from ECS, applies correction logic and then re-introduces corrected data to the appropriate streams or sends it to ECS again in case the UDRs cannot be corrected.