Data Correction

Owned by Former user (Deleted)
Last updated: Sept 27, 2024 by Felicity Borrelli
2 min read

This document describes the Data Correction process, how the records are organized and displayed in the UI, and the different actions that you can take on these records.  

For information about terms used in this document, see Glossary.

Introduction

Data Correction is used to collect and modify invalid records so they can be reprocessed. When a stream processes data, records that fail validation against the rules configured in the Validate Function are sent to Data Correction for further investigation. See Validate for more information.

Data correction and reprocessing

Once a record is available in Data Correction, you can decide to correct, reprocess, or delete the record. Once corrected and reprocessed, records are sent back to the Validate Function and re-validated. If no validation rule fails, the records are processed further. You may enable E-mail notifications to be sent to configured recipients whenever there is a new error in Data Correction.

Logs are generated whenever records are sent and reprocessed from Data Correction. You can analyze the count of the rejected records per stream run using Usage audit. 

A record is stored in Data Correction for 6 months from the time it was last updated.

Deleting a solution, stream, or a Validate Function is impossible when data exists in Data Correction. All relevant records must first be deleted.

User Interface 

The user interface is intuitive and you can view the erroneous records in two ways:

  • Summary view: This view displays errors detected by the streams in the solution and you can open it by clicking on the Data Correction tab.

  • Detail view: This view details a specific error type and enables you to modify the data. You can open this view by clicking an error type in the Summary view.

Subsections

This section contains the following subsections:

Â