Data Veracity provides a repository of for erroneous records and allows users to search, view, modify and reprocess the stored data. Data Veracity will serve as is an alternative to the Error Correction System (ECS), being able to complement and improve upon the existing functionalities. Data Veracity will be able to support :
- Data Veracity can handle larger volumes of data and has
...
- higher performance
...
- when
...
- searching
...
- , filtering, and processing data.
- The Error Correction System can store erroneous batches as well, and not only erroneous UDRs. No additional installation is required. For demo and development systems, the integrated Derby database can be used.
Data Veracity will contain a new web UI to support data exploration, classification and modification of records. this is accompanied by 3 new agents and APL functions to offload data flows from invalid, corrupt or incomplete records. The APL functions will work much like the ECS functions to facilitate the automation of detecting, qualifying and correcting the invalid records.
...
The Data Veracity Profile contains settings for selecting the database profile, mapping of UDR types to be used, mapping information for MIM and generation of the SQL script for the generation of tables in the external database.
Data Veracity Profile
Data Veracity is supported for use with the following databases:
...
The Data Veracity forwarding agent will be able to send any erroneous UDRs collected from any particular source and connect commits received UDRs to the external database to insert and commit the UDRs and , including any related metadata into the specific table. A standard Data Veracity Forwarding workflow will typically involve a collection agent, an analysis or aggregation agent to perform the UDR filtering, followed by the Data Veracity forwarding agent itself to output the erroneous UDRs..
Example of Workflow with the Data Veracity forwarding agent
The Data Veracity collection agent will be is used to collect any erroneous UDRs from the database using the search filters or error codes defined in the Data Veracity web UI. The UDRs to be collected will have either the NEW or UPDATED states in the database table, the Only UDRs with the status "NEW" will be collected. The state will be changed from "NEW" to "UPDATED" after collection. A standard Data Veracity collection workflow will typically involve the Data Veracity collection agent, an analysis agent or aggregation agent to reprocess the UDR, followed by a standard output agent or a Data Veracity forwarding agent to forward any remaining erroneous UDRs back to the database tables.
Data Veracity
...
UI
The web UI for Data Veracity will contain 6 UI contains functions for the users to perform and these serve to allow the user to manipulate the erroneous data, manage all aspects of filtering, error codes and repair rules.examination and manipulation of the stored erroneous data.
Data Veracity Web UI