This section page describes how to tune the Aggregation agent with file storage.
Aggregation Cache
The Aggregation agent can store sessions on the file system, using a storage server, but also in a cache. The maximum size of the cache will be determined by the Max Cached Sessions parameter in the Aggregation profile (see Aggregation Profile in 9.2.6.1 Performance Tuning with File Storage) and and the average size in memory of a session. It is difficult to estimate the exact memory consumption through testing but the following should be considered when implementing an Aggregation workflow:
Try to keep the session data small. Specifically, do not use large maps or lists in the sessions. These will take up a lot of memory.
If memory issues are encountered, try decreasing the Max Cached Sessions . In order to find out if the cache size is overdimensioned, you can study the memory of the Execution Context that is hosting the workflow in System Statistics. For information about System Statistics, see 6.16 System Statistics.
To avoid a large aggregation cache causing out of memory errors, the aggregation cache detects that the memory limit is reached. Once this is detected, sessions will be moved from the memory cache to the file system.
...
| Tip | ||
|---|---|---|
| ||
To speed up the start of workflows that run locally (on the EC), set the Execution Context property By doing so, the aggregation cache will be kept in memory for up to 10 minutes after a workflow has stopped. This in turn enables another workflow, that runs within a 10 minute interval after the first workflow has stopped, and that is configured with the same profile, to use the very same allocated cache. Note that since the cache remains in memory for up to 10 minutes after a workflow stopped executing, other workflows using other profiles might create caches of their own during this time. The memory space of the respective aggregation caches will add up in the heap. If the EC at a certain point runs out of memory, performance deteriorates as cache is cleared and, as a result, sessions have to be read from and written to disk. The profile session cache functionality is only enabled in batch workflows where the Aggregation profile is not set to read-only, and the storage is placed locally to the EC. |
Memory Handling in Real-Time
...
| Warning | ||
|---|---|---|
| ||
In real-time, when memory caching without any file storage, i e Storage Commit Interval is set to zero, make sure that you carefully scale the cache size to avoid losing a session due to cache over-runs. An over-run cache is recorded by the system event in the System Log. For further information, see 9.2.8 Aggregation Session Inspector. |
While the aggregation cache will never cause the EC to run out of memory, it is still recommended that you set the Max Cached Sessions low enough so that there is enough space for the full cache size in memory. This will increase system performance.
Multithreading
If you have many sessions ending up in timeout, you can improve the performance by enabling multithreading, i e use a thread pool, for the timeout function block in the Aggregation agent. When multithreading is enabled, the workflow can hand over sessions to the pool via the queue without having to wait for the read operations to complete, since the threads in the thread pool will take care of that. With many threads, the throughput of read operations completed per second can be maximized.
...