Couchbase (4.3)
- Jenni Wallin
A Couchbase profile is used to read and write bucket data in a Couchbase database and can be accessed by workflows using Aggregation, Distributed Storage, or PCC. The profile should not be used to operate on data that has been inserted or updated by external third-party software.
As a client of Couchbase, the profile operates in synchronous mode. When sending a request to Couchbase, the profile expects a server response, indicating success or failure, before proceeding to send the next one in the queue.
The Couchbase profile is loaded when you start a workflow that depends on it. Changes to the profile become effective when you restart the workflow.
Note!
Created or updated Couchbase profiles that are used for PCC do not become effective until you restart the ECs.
Configuration
External References
| Setting | Description |
|---|---|
External References | Select this button to Enable External References in an agent profile field. This can be used to configure the use of the following fields, and the respective external reference keys available:
For further information, see External Reference (4.3). |
In a Couchbase profile, there are three tabs: Connectivity, Management, and Advanced.
Connectivity Tab
The Connectivity tab is displayed by default.
Note!
Refer to /wiki/spaces/3PP/pages/1671475 for supported Couchbase versions.
Note!
From Couchbase version 5.0, buckets no longer use bucket-level passwords and you must access the bucket using a user and password that you must create in the Couchbase Web Console if it does not already exist.
The following settings are available in the Connectivity tab:
| Setting | Description |
|---|---|
Bucket Name | Enter the bucket that you want to access in Couchbase in this field. |
| Bucket User | Enter the user who has access rights to the bucket. This field supports parameterization using ${} syntax, see Profiles for more information on how parameterization works. |
User Password | Enter the password for the user who has access to the bucket. If you want to set a parameter, select the Parameterized checkbox and enter the parameter name using ${} syntax, see Profiles for more information on how parameterization works (in this mode the regular User Password field is disabled). |
Admin User | Enter the user with admin credentials to manage the Couchbase cluster. |
| Admin Password | Enter the password for the admin user. |
Operation Timeout (ms) | Enter the number of milliseconds after which you want Couchbase "CRUD" operations, that is, create, read, update, and delete, to timeout. Setting a lower value than the default 1000 ms may have a positive impact on throughput performance. However, if the value is set too low, indicated by a large number of operation timeouts errors in the EC/ECSA logs, a lower throughput can be expected. As mentioned, this value is used for timing out CRUD operations in Couchbase. This means that |
Retry Interval Time (ms) | Enter the time interval, in milliseconds, that you want to wait before trying to read the cluster configuration again after a failed attempt. In other words, in case a request sent to Couchbase returns an unsuccessful answer, the number specified here is the time that |
Max Number Of Retries | Enter the maximum number of retries. This is the number of retries |
Enable Security | Select this checkbox to connect to Couchbase cluster with SSL. If you want to set a parameter, select the Parameterized checkbox and enter the parameter name using ${} syntax, see Profiles for more information on how parameterization works (in this mode the regular Enable Security checkbox is disabled). The parameter needs to be set as true or false. |
| Security Profile | Click Browse to select a security profile with certificate and configuration to use, if you prefer to use a secure connection. Refer to Security (3.2) for more information. To create the keystore for Couchbase Server documentation, refer to Configure Client Certificate in the appropriate version. |
Use parameterized Cluster node list | Select this checkbox to set Cluster Nodes as a parameter instead. To set a parameter, you must enter the parameter name using the ${} syntax. See Profiles for more information on how parameterization works. |
Cluster Nodes | In this section, add IP addresses/hostnames of at least one of the nodes in the cluster. This address information is used by the Couchbase profile to connect to the cluster at workflow start and to retrieve the IP addresses of the other nodes in the cluster. If the first node in the list cannot be accessed, the Couchbase profile will attempt to connect to the next one in order. This is repeated until a successful connection can be established. Hence it is not necessary to add all the nodes, but it is good practice to do so for a small cluster. For example, if there are just three nodes, you should add all of them. |
Hint!
The Operation Timeout, Retry Interval Time, and Max Number Of Retries settings together with the Advanced tab setting mz.cb.lock.timeout.in.secs, work jointly. To understand how, see the following explanation.
If you for example use the lookup function with transaction, a LOCK ERROR means that the lookup failed because there was already a transaction lock held on the object that the lookup was trying to get from Couchbase. That is, the lookup was not able to get a lock on the object.
What is most important here is how the communication between and Couchbase works.
In case you use the lookup function it works like this:
- The lookup function with transaction ID tries to get an object from Couchbase with a lock.
sends a request to Couchbase to get an object and lock it.
- If the object exists and is not locked, Couchbase creates a lock and sends the answer back with a successful result code and the data. Alternatively, if the object is locked, Couchbase immediately sends an answer with the result LOCK_ERROR. Couchbase does not wait to answer until the lock is released.
This is why the Operation Timeout parameter does not play a crucial role here.
The Operation Timeout parameter is only used when Couchbase is stalling and does not send an answer for a long time.
Assuming that the object is locked and received answer with LOCK_ERROR,
will wait for the amont of time specified in Retry Interval Time and then retry the request towards Couchbase.
This process is repeated until Max Number Of Retries is exhausted or the answer contains a successful result code and the object is retrieved from Couchbase.
So, if you for example set Retry Interval Time to 100 ms and Max Number of Retries to 10, you get 100 ms * 10 retries = 1000 ms. However, from a MZ perspective, the timeout of 1000 ms is just for a single lookup attempt and not for the whole lookup operation.
To clarify, if you get a failed lookupMany with LOCK_ERROR in the system log, it means that the lookup failed because it tried to get an object with a lock but failed after Max Number Of Retries. That is, some other process (or thread) was holding a lock on that particular object for longer than 1000 ms. This is possible if the Advanced tab setting mz.cb.lock.timeout.in.secs is larger than 1000 ms.
In other words, you need to decide what is more important - that the operation succeeds or that the operation is fast.
Management Tab
The Management tab contains Cluster Management and Monitoring settings.
The following settings are available in the Management tab:
| Setting | Description |
|---|---|
Admin User Name | If you want to create a new bucket that does not exist in your Couchbase cluster, enter the user name that you stated when installing Couchbase in this field. |
Admin Password | If you want to create a new bucket that does not exist in your Couchbase cluster, enter the password that you stated when installing Couchbase in this field. |
Bucket Size (MB) | Enter the size of the bucket you want to create, in MB. |
Number of Replicas | Enter the number of replicas you want to have. |
If the bucket you want to access in Couchbase does not exist, the Couchbase profile can be used for creating the bucket in runtime for you, provided that the Admin User Name and Admin Password for your Couchbase cluster have been entered in the Management tab. If the bucket you want to access already exists in your cluster, these two fields do not have to be filled in.
Advanced Tab
In the Advanced tab you can configure additional properties.
It is recommended that you change these properties when using the Couchbase profile in Aggregation. For more information about using the Couchbase profile in Aggregation, see Performance Tuning with Couchbase Storage in Aggregation Agent - Real-Time.
See the text in the Properties field for further information about the properties that you can set.
This field supports parameterization using ${} syntax. For more information on parameterization see Profiles.
Logging Couchbase Statistics
You can use the property mz.cb.statistics.class if you want to log Couchbase statistics.
The Couchbase statistics that you can log include the following:
- Maximum answer time
- Average answer time
- Number of failed CRUDS
- Maximum lock time
- Average lock time
- Number of failed locks
- Number of timed-out locks
Support for Scope and Collection
Refer to the following properties in the Advanced tab to configure Scope and Collection.
- Couchbase scope name: The default value is "_default".
mz.cb.bucket.scopeName _default - Couchbase collection name: The default value is "_default".
mz.cb.bucket.collectionName _default
Note!
When naming scopes and collections, ensure to follow the standards mentioned in Naming for Scopes and Collections.
Ensure to allocate the correct resources to the scope or collection within the Couchbase bucket. For instance, if you have 5 buckets, each with its own set of resources, and you want to consolidate them into a single bucket using scopes and collections while removing the other 4, ensure that you allocate resources to the scopes and collections in line with the resource allocation used for the individual buckets previously.