...
| Info |
|---|
| By default, when working with files, Usage Engine Cloud Edition will either successfully process an entire input file, or not process the input file at all. This is called transactional processing. This transactional processing helps ensure that data is neither lost nor duplicated. |
| Anchor | ||||
|---|---|---|---|---|
|
If you are processing many small files, the performance overhead of having one transaction per file can reduce performance.
It is possible to batch multiple files into a single transaction when collecting files with the Amazon S3 collector.
In the configuration:
- Set the Transaction Batch Size to a value greater than 1 to process multiple files per transaction.
...
| Note | ||
|---|---|---|
| ||
| The number of stream replicas must be chosen when you create your stream. It cannot be changed once set. This is because of persistent states in Cloud Edition such as aggregation sessions and de-duplication information is stored separately per replica. |
...