Performance Tuning with Redis Storage(4.3)

This section describes how to tune the Aggregation agent with Redis storage.

Unless stated otherwise, the properties referred to in this section are set in the Advanced tab in the Aggregation profile.

Connections to Thread Pool

There are by default eight threads per workflow in the Redis thread pool that are used for Aggregation storage handling. Each thread corresponds to a Redis connection. You can increase the number of threads by setting mz.redis.agg.nr.of.threads. This may improve throughput performance, depending on the number of available CPUs.

Timeouts

There are by default two timeout threads per workflow that periodically check the Redis aggregation storage for timed-out sessions. You can control how often this check is performed by setting mz.redis.agg.timeoutwait.sec. The default value is 1 second.

You can also increase the number of threads that perform this check by setting the property mz.redis.agg.timeout.threads. Setting a higher value than the default may speed up the detection of timeouts. However, the number of CPUs is a limiting factor.

Timeout data is stored in the Aggregation sessions but also in a configurable number of "sorted sets", enabling faster iteration in the timeout threads. When you have a large number of sessions and workflows, you may want to increase the number of sorted sets to reduce resource contention and thereby timeout latency. To do this, increase the default value of mz.redis.agg.timeout.sets. By default, there are 100 sets. 

By default, the Aggregation agent ensures the consistency of the stored sessions and sorted sets, preventing the loss of timed-out sessions. In cases where transaction safety is not critical, you may increase the performance of timeout threads by disabling the consistency checks. To do this, set mz.redis.agg.slow.timeout.enable to false.

You use the Operation Timeout (ms) setting in the Connectivity tab of the assigned Redis profile to control the timeout of Redis "CRUD" operations, i e create, read, update, and delete. Setting a lower value than the default 1000 ms may have a positive impact  on throughput performance. However, if the value is set too low, indicated by a large number of operation timeouts  errors in the EC/ECSA logs, a lower throughput can be expected.

Session Storage Format

The aggregation sessions are stored in JSON format. However, some of the data within the JSON strings can be stored in binary format instead of plain text (default). You can change the stored format by setting the property json_serializer.format. The valid values are:

• MZ-BIN - The session data is serialized into JSON strings with binary content.

• JSON - The session data is serialized into JSON strings with plain text content.
  
  

{ 
  "drType": "MZ-BIN", 
  "drFormatVersion": 2, 
  "data": "Af+cAAAAR2NvbS5tZWRpYXRpb256b25lLnVsdHJhLmluLmFnZ3Jl 
  Z2F0aW9uX2NvbW1vbl9zZXNz\naW9uX3Nlc3Npb25fMTk1MzI3MTEwAAABT 
  C19yJgAAAAAAAAAAAEAAAAGAQAAAAExAQAAAAExAQAA\nAAExAAAABgEAAA 
  AHY29uc3VtZQ==", 
  "SessionTimeout": 1426692360344, 
  "initialized": true 
 }