The Archiving agent provides comprehensive data management capabilities. This agent, profile and inspector support both archival and retrieval processes.

• Looking up Archived Data: Customers can use the Archive Inspector to look up archived files. This tool is available in the Execution Manager and provides advanced search functionality that allows you to search for files within specified periods.

• Deleting Archived Data: The Archive Cleaner task is responsible for automatic deletion of outdated archives based on the purge criteria set within the Archive profile. You can also trigger clean-up processes manually to ensure that the data does not exceed its intended retention period.

For more information on functionality and configuration, see the Archiving section in the MediationZone software description.

The Amazon S3 forwarding agent provides capabilities to write data to S3.

For best practices around data security and encryption, see the Amazon S3 documentation.

• Looking up Records:

  • Data Veracity User Interface: Open the Data Veracity user interface to search and locate specific UDRs or batches. Use the Search options to filter records based on various criteria such as error codes, insert times, or specific MIM values. With advanced filtering and saved filters, you can streamline recurrent searches and improve efficiency in managing records.

  • View and Inspect Data: Once records are located, you can view details and inspect the contents of UDRs in the user interface.

• Deleting Records:

  • Mark for Deletion: If a record contains sensitive information that should not be retained, mark it for deletion directly in the Data Veracity user interface. This action flags the records but does not remove them from the database.

  • Force Deletion: For immediate removal, use the Force Delete option available to administrators. This option should be used with caution, as it permanently removes the record from the database, ensuring that sensitive data is irrecoverable.

  • Automated Cleanup via Maintenance Task: Configure the DataVeracity_Maintenance system task to automatically remove records that have been marked for deletion. This task can be triggered based on specific conditions and time frames, adhering to data retention policies.

• Managing Access and Security:

  • Restricted Fields and Data Masking: Ensure sensitive fields are restricted and masked to prevent unauthorized access and modification. With Data Veracity you can configure permissions to restrict who can view or edit sensitive data.

  • Permission Controls: Use the Access Controller to assign and manage permissions for different user levels, ensuring that only authorized personnel can execute sensitive operations like data deletion or modification.

• Auditing and Compliance:

  • Audit Trails: Keep track of who accessed and modified the Data Veracity records. Ensure all actions on sensitive data are logged to support compliance audits and internal controls.

• External Databases:

See the specific documentation provided by each database platform provider. Implement encryption and secure data handling practices as recommended:

• PostgreSQL: PostgreSQL Documentation

• Oracle Database: Oracle Security Guides

• SAP HANA: https://help.sap.com/docs/

For more information on functionality and configuration, see the Data Veracity section in the MediationZone documentation.

• Looking Up Records:

  • Search for Records: Use the ECS Inspector's Search function to find specific UDRs or batches. You can filter searches based on various criteria such as workflow, agent, error code, or insert period. Advanced search options allow for more detailed filtering, including the use of wildcards and intervals for specified fields.

  • Using Saved Filters: For frequent searches, you can save your search settings as filters for quick future access. This is particularly useful for regularly monitoring or auditing records that meet specific criteria.

• Deleting Records:

  • Selective Deletion: After locating the records using the Search function, you can selectively delete entries. Ensure that the records are in a reprocessed state, as only reprocessed entries can be safely removed without affecting ongoing processes.

  • Bulk Deletion: For managing larger volumes of data, the Bulk Edit feature allows you to delete multiple records simultaneously. This tool is ideal for removing batches of sensitive information efficiently.

  • Scheduled Cleanup: Use the ECS Maintenance System Task to set up automatic purging of outdated or unnecessary records. This can be configured to remove records based on their reprocessing state and can be tailored to ensure compliance with data retention policies.

• Secure Management:

  • Restrict Field Edits: To prevent unauthorized modifications, configure restricted fields to protect sensitive data within UDRs from being edited.

  • Monitor and Audit: Regularly review the ECS Maintenance logs and use the statistics reports to ensure that deletions and data management practices comply with organisational security policies.

For more information on functionality, see th Error Correction System section in the MediationZone documentation.

• Using SQL Processing Agent for Lookup Operations:

  • Query Configuration for Data Retrieval: Configure the SQL Processing Agent to perform selective queries that identify sensitive data needing review or deletion. This involves setting up SQL queries within the agent to filter and retrieve only the data matching specific security or compliance criteria.

• Configuring the SQL Forwarding Agent for Data Deletion:

  • SQL Statement Configuration: Use the SQL Forwarding Agent to execute SQL DELETE commands. In the agent’s configuration dialog, enter SQL statements specifically designed to target and delete sensitive records. Example: DELETE FROM customer_records WHERE customer_id = $(UDR.CustomerID);.

  • Dynamic Data Handling: Leverage MIM values and UDR field variables in your SQL statements to ensure that only the intended records are targeted for deletion, enhancing security and precision.

  • Stored Procedures Support: For complex deletion requirements, configure the agent to call stored procedures that encapsulate deletion logic. This approach enhances security by abstracting the deletion logic into the database layer. Example SQL call: CALL secure_delete_procedure($(UDR.CustomerID));.

For more information on functionality and configuration, see the SQL Agents section in the MediationZone documentation..