Data Veracity for Correcting Data

Data Veracity provides a repository of erroneous records and allows users to search, view, modify and reprocess the data. Data Veracity will serve as an alternative to the Error Correction System (ECS), being able to complement and improve upon the existing functionalities. Data Veracity will be able to support larger volumes of data and has a higher performance capability when it comes to searching and filtering compared to the ECS. However, unlike the ECS, Data Veracity will only support erroneous UDRs, as opposed to both erroneous Batch files and UDR.

Data Veracity will contain a new web UI to support data exploration, classification and modification of records. this is accompanied by 3 new agents and APL functions to offload data flows from invalid, corrupt, or incomplete records. The APL functions will work much like the ECS functions to facilitate the automation of detecting, qualifying and correcting the invalid records.

Data Veracity Profile

The Data Veracity Profile contains settings for selecting the database profile, mapping of UDR types to be used, mapping information for MIM and the generation of the SQL script for creating tables in the external database.

Data Veracity is supported for use with the following databases:

• Oracle

• PostgreSQL

The Data Veracity forwarding agent sends erroneous UDRs collected from any source and connects to the external database to insert and commit UDRs and related metadata into a specific table. A standard Data Veracity Forwarding workflow will typically involve a collection agent, an analysis or aggregation agent to perform the UDR filtering, followed by the Data Veracity forwarding agent itself to output the erroneous UDRs.

Open

Example of Workflow with the Data Veracity forwarding agent

The Data Veracity collection agent will collect any erroneous UDRs from the database using the search filters or error codes defined in the Data Veracity web UI. The UDRs to be collected will have either the NEW or UPDATED states in the database table, the state will be changed from NEW to UPDATED after collection. A standard Data Veracity collection workflow will typically involve the Data Veracity collection agent, an analysis agent or an aggregation agent to reprocess the UDR, followed by a standard output agent or a Data Veracity forwarding agent to forward any remaining erroneous UDRs back to the database tables.

Data Veracity Web UI

The web UI for Data Veracity contains 6 functions that allow you to manipulate erroneous data, and manage all aspects of filtering, error codes and repair rules.

Open

Data Veracity Web UI