9.19.1 Duplicate Batch Detection Agent Overview

The Duplicate Batch Detection agent allows duplication control on passed data batches. Each data batch will be tested against already stored meta data from batches, to see if they are considered to be duplicates.

Meta data for data batches are kept for a configurable number of days. If the meta data of a batch has been removed, a duplicate of this batch can no longer be detected.

If any duplicates are detected, a message is logged to the System Log, and the duplicate batch is cancelled, which may cause the workflow to abort. For further information, see 6.14 System Log.

To monitor the duplicate batches the Duplicate Batch Inspector may be used. for further information, see 9.19.6 Duplicate Batch Inspector.

It is only appropriate to use the agent after agents that may create duplicate batches. Normally this is a file based collection agent. Several workflows may utilize the same Duplicate Batch profile. In this case, their batches will be mutually compared.