Versions Compared

Old Version 1

changes.mady.by.user Jenni Wallin

Saved on

compared with

New Version 2

changes.mady.by.user Former user

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

For further information on performance tuning with Couchbase storage, see Tuning & Performance in https://developer.couchbase.com/documentation/server/5.1/introduction/intro.html.

Queries and Indexes

When Couchbase is selected as the storage type in an Aggregation profile, a bucket is automatically created during execution of a workflow. The bucket is named according to the configuration of the assigned Couchbase profile. The bucket is populated with documents that contain the aggregation session data. This makes it possible to index the timeout information of aggregation sessions in Couchbase.

...

For more information about queries and indexes, see https://developer.couchbase.com/documentation/server/5.1/indexes/indexing-overview.html.

Timeout

There are by default, two timeout threads per workflow that periodically check the Couchbase aggregation storage for timed out sessions. You can control how often this check is performed by setting  mz.cb.agg.timeoutwait.sec . The default value is 10 seconds.

...

Queries operate over a different protocol than CRUD operations and have a separate timeout property named view.timeout in the Advanced tab of the Couchbase profile. The default value is set to 75000 (ms). It is generally not recommended to decrease this value. However, if you frequently receive the error Failed to iterate through timeout sessions in the EC/ECSA logs, increasing this value may have a positive impact on throughput performance.

Excerpt

Session Storage Format

The aggregation sessions are stored in JSON format. However, some of the data within the JSON strings can be stored in binary format instead of plain text (default). You can change the stored format by setting the property mz.cb.agg.json_serializer.format. The valid values are:

  • MZ-BIN - The session data is serialized into JSON strings with binary content.
  • JSON - The session data is serialized into JSON strings with plain text content.
Info
titleExample - Binary Format
Code Block
languagetext
themeEclipse
{ 
  "drType": "MZ-BIN", 
  "drFormatVersion": 2, 
  "data": "Af+cAAAAR2NvbS5tZWRpYXRpb256b25lLnVsdHJhLmluLmFnZ3Jl 
  Z2F0aW9uX2NvbW1vbl9zZXNz\naW9uX3Nlc3Npb25fMTk1MzI3MTEwAAABT 
  C19yJgAAAAAAAAAAAEAAAAGAQAAAAExAQAAAAExAQAA\nAAExAAAABgEAAA 
  AHY29uc3VtZQ==", 
  "SessionTimeout": 1426692360344, 
  "initialized": true 
 }
Info
titleExample - Plain text format
Code Block
languagetext
themeEclipse
{ 
  "drType": "JSON", 
  "drFormatVersion": 2, 
  "data": { 
  "Type": "udr", 
  "StorableId": "aggregation_common.session.session", 
  "TypeName": "aggregation_common.session.session", 
  "Version": 1, 
  "Content": { 
  "CRCValue": 0, 
  "v_bnum": "1", 
  "SessionID": null, 
  "initialized": true, 
  "SessionTimeout": 1426691920461, 
  "v_response": "consume", 
  "v_anum": "1", 
  "v_total_duration": 5 
  } 
  }, 
  "SessionTimeout": 1426691920461, 
  "initialized": true 
 }

The Aggregation agent can read stored session data in both formats, regardless of the selected value.

In order to obtain the best possible performance in the Aggregation agent, you should use the binary format.

Tip
titleHint

You can use an Aggregation agent with Force Read Only selected to read the stored sessions. You can then encode the binary content to JSON with the APL function jsonEncodeUdr. This is useful for debugging, or when you want to view the sessions in an external system. For more information about jsonEncodeUdr, see the /wiki/spaces/MZD70/pages/5145301.

Replication and Persistence

You can use the properties mz.cb.awaitPersistenceTo and mz.cb.awaitReplicationTo in the Advanced tab of the selected Couchbase profile to minimize the risk of data loss. However, setting a higher value than the default 0 will reduce the throughput performance.

Automated Index Updates

Note
titleNote!

This section only applies if you are using view instead of queries.

...