diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index 8133ad9abe..9cc098dd8a 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -82,6 +82,10 @@ include::third-party:partial$nav.adoc[] **** xref:learn:clusters-and-availability/xdcr-filtering.adoc[XDCR Advanced Filtering] **** xref:learn:clusters-and-availability/xdcr-conflict-resolution.adoc[XDCR Conflict Resolution] **** xref:learn:clusters-and-availability/xdcr-with-scopes-and-collections.adoc[XDCR with Scopes and Collections] + **** xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning] + **** xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc[XDCR Conflict Logging] + ***** xref:learn:clusters-and-availability/xdcr-viewing-conflict-logs.adoc[Viewing Conflict Logs] + **** xref:learn:clusters-and-availability/xdcr-active-active-sgw.adoc[XDCR Active-Active with Sync Gateway] *** xref:learn:clusters-and-availability/groups.adoc[Server Group Awareness] * xref:learn:security/security-overview.adoc[Security] ** xref:learn:security/authentication.adoc[Authentication] diff --git a/modules/introduction/partials/new-features-80.adoc b/modules/introduction/partials/new-features-80.adoc index d875f9a19f..f4bb1ded02 100644 --- a/modules/introduction/partials/new-features-80.adoc +++ b/modules/introduction/partials/new-features-80.adoc @@ -113,6 +113,11 @@ Additional information sent by clients at connection time can be found in the lo https://jira.issues.couchbase.com/browse/MB-61048[MB-61048]:: Once faulty remote cluster credentials are fixed, XDCR will now be able to more quickly restart replications that depend on the repaired references. +https://jira.issues.couchbase.com/browse/MB-58989[MB-58989]:: +XDCR detects and logs concurrent conflicts that occur in different clusters but on the same document version due to independent modifications by locally connected applications during Active-Active replication. +The conflict logs are for your information only. +For more information, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc[XDCR Conflict Logging Feature]. + [#section-new-feature-800-query-service] === Query Service diff --git a/modules/learn/assets/images/clusters-and-availability/xdcr-active-sgw-after-upgrade.png b/modules/learn/assets/images/clusters-and-availability/xdcr-active-sgw-after-upgrade.png new file mode 100644 index 0000000000..ccb0c7d8b7 Binary files /dev/null and b/modules/learn/assets/images/clusters-and-availability/xdcr-active-sgw-after-upgrade.png differ diff --git a/modules/learn/assets/images/clusters-and-availability/xdcr-active-sgw-before-upgrade.png b/modules/learn/assets/images/clusters-and-availability/xdcr-active-sgw-before-upgrade.png new file mode 100644 index 0000000000..9f49db8e86 Binary files /dev/null and b/modules/learn/assets/images/clusters-and-availability/xdcr-active-sgw-before-upgrade.png differ diff --git a/modules/learn/pages/clusters-and-availability/xdcr-active-active-sgw.adoc b/modules/learn/pages/clusters-and-availability/xdcr-active-active-sgw.adoc new file mode 100644 index 0000000000..d192c23041 --- /dev/null +++ b/modules/learn/pages/clusters-and-availability/xdcr-active-active-sgw.adoc @@ -0,0 +1,99 @@ += XDCR Active-Active with Sync Gateway +:description: pass:q[You can use XDCR with Sync Gateway mobile clusters in a bi-directional, active-active replication, but you must make sure that both the Server and the Sync Gateway versions support this option. Otherwise, using XDCR with Sync Gateway buckets in a bi-directional replication can cause data corruption.] + +[abstract] +{description} + +[#xdcr-active-active-sgw-intro] +== Introduction + +NOTE: *To set up XDCR bi-directional replication with Sync Gateway (SGW), you need to have at least a Server 7.6.6 version and SGW 4.0.0 version. +However, Sync Gateway 4.0.0 is a future release version. + +In the versions earlier than Server 7.6.6 and Sync Gateway (SGW) 4.0.0*, only an active-passive setup was supported with both XDCR and SGW. +XDCR Active-Active replication with Sync Gateway for XDCR-Mobile interoperability configuration was introduced in the Server 7.6.6 version, where you can configure an active-active XDCR setup with Sync Gateway (SGW) and mobile applications both on the XDCR source and target clusters. + +[IMPORTANT] +==== +Here are a few limitations to the _XDCR Active-Active with Sync Gateway_ feature. + +* If you use the _user created extended attributes (user xattrs)_ in your documents, and you have more than 10 user xattrs in a document, then you cannot use the feature _XDCR Active-Active with Sync Gateway_. +This is due to an internal limitation of managing extended attributes in a document. +If you try to use the feature _XDCR Active-Active with Sync Gateway_ when you have more than 10 user xattrs in your document, the XDCR replication **silently skips** replicating that document. +As a result, the data in the replication-skipped document will not be consistent between the target and source clusters. +The only way you will know this skip occured is because the Prometheus stat `subdoc_cmd_docs_skipped` will be incremented and the document will _not_ be consistent between the target and source. +* If you use Eventing service functions that update documents in XDCR-replicated buckets (Eventing source bucket mutations), ensure your functions do not cause continuous replication loops. +In bi-directional active-active XDCR environments, Eventing functions that trigger document updates can lead to "ping-pong" replication unless you implement logic to prevent infinite loops. +Always add safeguards to avoid redundant updates and unwanted replication behavior in bi-directional setups. For more information, see xref:sync-gateway:xdcr-active-active-eventing.adoc[XDCR Active-Active and Eventing]. +==== + +You can configure XDCR Active-Active with Sync Gateway for XDCR-Mobile interoperability using one of the following methods: + +* xref:learn:clusters-and-availability/xdcr-active-active-sgw.adoc#xdcr-active-active-sgw-greenfield-deployment[Greenfield deployment]: Set up a new active-active configuration with both XDCR and SGW. +* xref:learn:clusters-and-availability/xdcr-active-active-sgw.adoc#xdcr-active-active-sgw-upgrade[Upgrading an existing setup]: Convert an existing active-passive XDCR-SGW configuration to an active-active XDCR-SGW setup. + +NOTE: When using the feature _XDCR Active-Active with Sync Gateway_, where Sync Gateway version is 4.0* or a later version and Server is 7.6.6 or a later version, the replication target XDCR inbound user must have the RBAC roles, xref:learn:security/roles.adoc#xdcr-inbound[XDCR Inbound] role and xref:learn:security/roles.adoc#data-writer[Data Writer] role. + +[#xdcr-active-active-sgw-prerequisites] +== Prerequisites + +Set the bucket property `enableCrossClusterVersioning` to use the setting `mobile=Active` during the processes xref:manage:manage-xdcr/create-xdcr-replication.adoc[Create a Replication] in xref:manage:manage-xdcr/xdcr-management-overview.adoc[Manage XDCR]. +To enable the bucket property `enableCrossClusterVersioning` using the REST API, see xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc#modify-enablecrossclusterversioning[Modify the bucket property enableCrossClusterVersioning] or xref:rest-api:rest-bucket-create.adoc#example-enablecrossclusterversioning-edit[Example: Turning on enableCrossClusterVersioning, when Editing]. + +[#xdcr-active-active-sgw-greenfield-deployment] +== Greenfield Deployment + +To configure a new active-active XDCR with Sync Gateway setup, do the following: + +. Create two clusters on Server 7.6.6 or a later version with _all_ the nodes of the clusters. For example, cluster A and cluster B (or you can upgrade the existing Server clusters to 7.6.6 or a later version). +. Create buckets, for example, B1 and B2 in cluster A and cluster B respectively, between which XDCR will be set up. Now, do the following: +.. Enable the ECCV setting on B1. All the mutations in B1 will have a new metadata called HLV. +.. Enable the ECCV setting on B2. All the mutations in B2 will have a new metadata called HLV. ++ +NOTE: ECCV refers to the bucket property `enableCrossClusterVersioning`. If there are more than two buckets in the replication topology, you must enable ECCV for all those buckets. ++ +. Create an XDCR from B1 to B2 by setting `mobile=Active`. Also, create an XDCR from B2 to B1 by setting `mobile=Active`. ++ +For information about creating an XDCR by setting `mobile=Active` through the REST API, see xref:rest-api:rest-xdcr-create-replication.adoc[Creating a Replication]. ++ +For information about creating an XDCR by setting `mobile=Active` from the UI, see xref:manage:manage-xdcr/create-xdcr-replication.adoc#create-an-xdcr-replication-with-the-ui[Create an XDCR Replication with the UI]. +. Configure SGW 4.0.0* version on each cluster, cluster A and cluster B. + +This setup can handle application traffic on both buckets B1 and B2 of the respective clusters along with SGW import into both the buckets simultaneously. + +[#xdcr-active-active-sgw-upgrade] +== Upgrading an existing setup + +You can convert an existing active-passive XDCR-Sync Gateway (SGW) setup into an active-active XDCR-Sync Gateway setup. + +For illustration, there are two clusters, A and B. An SGW is connected to cluster A and this cluster is active. +Cluster B is passive with XDCR setup from bucket B1 in cluster A to bucket B2 in cluster B. +The current application traffic should be only on bucket B1 of cluster A. + +.Replication before upgrade: XDCR Active-Passive with SGW +image::clusters-and-availability/xdcr-active-sgw-before-upgrade.png[,720,align=left] + +. Upgrade both clusters A and B with _all_ the nodes of the clusters to Server 7.6.6 or a later version. +. Enable ECCV on bucket B1. All the mutations in B1, after this point of time, will have a new metadata called HLV. ++ +NOTE: ECCV refers to the bucket property `enableCrossClusterVersioning`. If there are more than two buckets in the replication topology, you must enable ECCV for all those buckets. ++ +. Enable ECCV on bucket B2. All the mutations in B2, after this point of time, will have a new metadata called HLV. ++ +NOTE: If there are more than two buckets in the replication topology, you must enable ECCV for all those buckets. ++ +. Update the replication settings to `mobile=Active` of the already existing XDCR from B1 to B2. ++ +You can use the REST API or the XDCR UI to update an existing replication. For information about using the REST API to modify the replication settings for an existing replication, see xref:rest-api:rest-xdcr-adv-settings.adoc#change-existing-replication-with-mobile-active[Change Settings for an Existing Replication to Set mobile=Active] in xref:rest-api:rest-xdcr-adv-settings.adoc[Managing Advanced Settings]. ++ +. Create an XDCR from B2 to B1 with the replication settings as `mobile=Active`. +. Upgrade SGW on cluster A to the version 4.0.0*. +. Connect SGW version 4.0.0* to cluster B. +. Enable application active traffic on cluster B. + +This setup can handle application traffic on both buckets B1 and B2 of the respective clusters along with SGW import into both the buckets simultaneously. + +This is an illustration of the final configuration: + +.Replication after upgrade: XDCR Active-Active with SGW +image::clusters-and-availability/xdcr-active-sgw-after-upgrade.png[,720,align=left] diff --git a/modules/learn/pages/clusters-and-availability/xdcr-conflict-logging-feature.adoc b/modules/learn/pages/clusters-and-availability/xdcr-conflict-logging-feature.adoc new file mode 100644 index 0000000000..73e238a05d --- /dev/null +++ b/modules/learn/pages/clusters-and-availability/xdcr-conflict-logging-feature.adoc @@ -0,0 +1,585 @@ += XDCR Conflict Logging +:description: pass:q[XDCR detects and logs concurrent conflicts that occur in different clusters but on the same document version due to independent modifications by locally connected applications during Active-Active replication. The conflict logs are for your information only. The best practice in Active-Active systems is that application environments must be designed to avoid conflicts.] +:page-toclevels: 3 + +[abstract] +{description} + +During XDCR Active-Active replication, a conflict can occur if different applications connected locally to 2 different clusters, independently modify the same version of the document at around the same time. +These simultaneous modifications happen within the convergence window*, so the changes made in one document are not reflected in the history of the other. + +If conflict logging is enabled, XDCR logs these conflict events in a conflict log collection, specified by you, for your information. +The conflict logging does not change conflict resolution. + +XDCR performs conflict resolution and replicates the documents whether conflicts are logged or not. +Conflict logging is informational only, so that you can review the conflict log message documents and the copies of the documents that were in conflict, to help you design application systems that avoid conflicts. + +_*A convergence window is a period during which multiple actors, such as users, applications, or clusters, can modify the same document in an active-active topology, before the changes have fully converged across all clusters._ + +== XDCR Conflict Logger Overview + +* Conflict logging is asynchronous and done on a best-effort basis, meaning, not all conflicts may be logged. + +* Conflicts are expected to be rare during XDCR replication. +If there is an upsurge of conflicts, so that logging conflicts may cause replication issues, XDCR stops logging conflicts temporarily to prioritize replication. + +* If any error occurs, due to resource issues or a delay in writing to the conflict log collection, XDCR may pause or stop the logging process, report the error type for the pipeline, and continue with replication so that there is no High Availability issue. + +* xref:learn:clusters-and-availability/xdcr-conflict-resolution.adoc[Conflict Resolution] process remains unchanged. + +IMPORTANT: Even if conflict logging fails, XDCR continues with conflict resolution to make sure of High Availability. +Conflict resolution is not changed by conflict logging. +Conflict logging is informational only. + +[#using-of-xdcr-conflict-logging] +== Using the XDCR Conflict Logging Feature + +To log conflicts during an XDCR, follow these steps: + +. As a prerequisite, to enable the conflict logging feature, xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[Enable the Cross Cluster Versioning] setting on each bucket of the XDCR topology. +Do this by setting the bucket property `enableCrossClusterVersioning` to `true` on all the buckets in the XDCR topology clusters. +As a result, additional metadata called, xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc#hlv-data-maintained-in-xattr[Hybrid Logical Vector (HLV)], also known as version vector, is added to the documents and updated as the documents replicate. ++ +To enable the bucket property `enableCrossClusterVersioning`: + +* Using REST API, see xref:rest-api:rest-bucket-create.adoc#example-enablecrossclusterversioning-edit[Example: Turning on enableCrossClusterVersioning, when Editing]. + +* From the UI, see xref:manage:manage-buckets/edit-bucket.adoc#edit-bucket-for-eccv[Edit a Bucket to Enable Cross Cluster Versioning]. ++ +[NOTE] +==== +* Once `enableCrossClusterVersioning` is enabled, it cannot be disabled. +As a result, HLV also cannot be disabled. + +* The conflict logging feature detects and logs conflicts only when the property `enableCrossClusterVersioning` is set to `true` for all the buckets in the replication topology. +Make sure you set `enableCrossClusterVersioning` to `true` in the bucket properties when adding new buckets or new clusters to the existing replication topology. +==== + +. When creating the XDCR replication, configure and enable conflict logging. +You configure conflict logging by specifying the conflict log collections to store the conflict logs and documents, and the conflict logging rules. +You must specify at least 1 conflict log collection, which is the default. ++ +When you do not specify any conflict logging rules, all conflicts detected during the replication are logged into the default conflict log collection. +The conflict logging rules allow granular configuration of conflict log collections for specific scopes and collections in the replication. +You can override the default log collection by either specifying which scopes and collections must have their conflicts logged, or assigning different conflict log collections for each. ++ +For detailed information about how to enable and configure the conflict logging feature while creating or updating a replication, see the following: + +** xref:manage:manage-xdcr/create-xdcr-replication.adoc#xdcr-ui-settings-for-conflict-logging +[Replication Settings for XDCR Conflict Logging], to set from UI. + +** xref:rest-api:rest-xdcr-create-replication.adoc[Creating a Replication] by xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#configure-conflictlogging-settings[Enabling and Configuring Conflict Logging], to set through REST API. + +. When XDCR logs a conflict, it creates 3 related documents in the conflict log collection specified by you. +The 3 documents together is referred to as the xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-record[Conflict Record]. +As the conflict log collections are regular collections, you can programmatically access and review the documents. +For examples about different ways to query or process the conflict log information, see xref:learn:clusters-and-availability/xdcr-viewing-conflict-logs.adoc[Viewing Conflict Logs]. + +[#conflict-collection] +== Conflict Log Collection + +A conflict log collection, or conflict collection, is the collection you specify to store conflict event messages and conflicting documents when you configure conflict logging for XDCR replication. +Each conflict event creates 3 documents in the conflict log collection; 1 conflict event message document and 2 conflicting documents. +Together, these 3 documents form a xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-record[Conflict Record]. + +For information about configuring a collection to log conflicts, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#configure-conflictlogging-settings[Enabling and Configuring Conflict Logging]. + +You can specify a regular collection as a conflict log collection in the cluster, where you're creating the replication. +You can also name the source collection, which is a part of the replication process, as a conflict log collection. +The conflict record documents are not replicated. + +After the XDCR conflict logger creates the conflict log collection, you can manage the conflict record documents in them. +As these documents are also copies of your application documents, you must use RBAC to make sure appropriate access to the documents. +Make sure to monitor the disk storage use for the conflict log collections and delete documents when they're no longer needed. + +NOTE: XDCR adds a system extended attribute (xattr) called `_xdcr_conflict` into each of the 3 documents in the xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-record[Conflict Record], which prevents XDCR from replicating these conflict log documents. + +[#conflict-record] +=== Conflict Record + +A conflict record consists of the following 3 documents: + +* *Conflict Record Document (CRD)*: This is a JSON document that records details about the conflict event along with the `docId` of the affected document and the CRD creation timestamp in the ISO 8601 format. +The CRD document ID is in the format `crd__`. + +* *Source document*: This is a copy (body and xattrs) of the source document that caused the conflict. +The Source document ID is in the format `src__`. +You can find this document ID in the `srcDoc.id` value of the CRD document. + +* *Target document*: This is a copy (body and xattrs) of the target document that caused the conflict. +The Target document ID is in the format `tgt__`. +You can find this document ID in the `tgtDoc.id` value of the CRD document. + +In the description of the conflict record documents, the source and the target mean replication direction. + +The generated `` in the ID of the conflict record documents ensures that each conflict event, for the same document or the same document ID, is logged uniquely. + +XDCR adds a system extended attribute (xattr) called `_xdcr_conflict` into each of the 3 documents in the Conflict Record, which prevents XDCR from replicating these conflict log documents. + +For query examples to view the conflict record documents, see xref:learn:clusters-and-availability/xdcr-viewing-conflict-logs.adoc[Viewing Conflict Logs]. + +[#conflict-record-document-format] +=== Conflict Record Document Format + +This is an example of the Conflict Record Document. + +[source,json] +---- +{ + "docId": "ae03ecde-203c-4a3b-a6d0-36b60f627c8a", + "id": "crd_1749454000005461807_e2f416ed317beb4f16a3587656b3d3450e1bec893cabebd3b9bb857d63824da4", + "replId": "d976db26a5ce512a243db65a3026073f/bucket1/bucket1-nhuUKw2mOVPRiDvcoF4hJg==", + "srcDoc": { + "bucketUUID": "fada12469f1027ecb078d2c19a2802fb", + "cas": 1749453999768928300, + "clusterUUID": "1bc375a26b7f822ebe91288c58a6c614", + "collection": "_default", + "expiry": 0, + "flags": 0, + "id": "src_1749454000005461807_e2f416ed317beb4f16a3587656b3d3450e1bec893cabebd3b9bb857d63824da4", + "isDeleted": false, + "nodeId": "10.0.32.102", + "revSeqno": 2, + "scope": "_default", + "xattrs": { + "_mou": "", + "_sync": "", + "_vv": "" + } + }, + "tgtDoc": { + "bucketUUID": "6cd3a35ba2d337d84d4fcd30c3a46fb9", + "cas": 1749453999840231400, + "clusterUUID": "d976db26a5ce512a243db65a3026073f", + "collection": "_default", + "expiry": 0, + "flags": 0, + "id": "tgt_1749454000005461807_e2f416ed317beb4f16a3587656b3d3450e1bec893cabebd3b9bb857d63824da4", + "isDeleted": false, + "nodeId": "10.0.25.110", + "revSeqno": 2, + "scope": "_default", + "xattrs": { + "_mou": "", + "_sync": "", + "_vv": "{\"cvCas\":\"0x00003dca244f4718\",\"src\":\"ntR6bcR9VHcjcPpQ5osEiw\",\"ver\":\"0x00003dca244f4718\"}" + } + }, + "timestamp": "2025-06-09T07:26:40.005Z" +} +---- + +Some of the key-value pairs in CRD are as follows: + +|=== +| Key | Value + +| `timestamp` +| Date and time of the event in the UTC ISO 8601 format: +YYYY-MM-DDThh:mm:ss:SSSZ + +The timestamp is the wall clock of the node that noted the conflict. + +| `id`/`srcDoc.id`/`tgtDoc.id` +| CRD document ID prefixed with `crd_`. + +A copy of conflict source document ID prefixed with `src_`. + +A copy of conflict target document ID prefixed with `tgt_`. + +| `docId` +| document id of the actual document in conflict. + +| `replId` +| Replication ID. + +| `srcDoc` and `tgtDoc` +These JSON Objects have the IDs of the source and target document copies in the conflict collection. +The document IDs are indicated by `srcDoc.id` and `tgtDoc.id`. + +| `nodeId` +| Node identification, which can be the Hostname or IP address and a port. + +| `bucketUUID` +| Source/target bucket unique ID. + +| `cas` +| Source/target document CAS. + +| `clusterUUID` +| Source/target cluster unique ID. + +| `isDeleted` +| True or false for - is the source/target document deleted? + +| `scope` and `collection` +| Name of the scope and collection where the conflict source/target documents exist. + +| `revSeqno` +| Source/target document’s revision sequence number. + +| `expiry` +| Source/target document’s expiry time. + +| `flags` +| Source/target document’s flag. + +| `xattrs` +| System xattributes. + +| `_vv` +| Document version vector. +HLV metadata. +|=== + +[#conflict-logger] +=== Conflict Logger + +The XDCR Conflict Logger logs conflicts continuously and is designed to minimize the impact on replication performance and system resources. + +Conflict logging is performed on a best-effort basis. +If system resources are limited or errors occur while writing to the conflict log collection, XDCR prioritizes replication and High Availability. +In these cases, temporarily the conflict logging may be skipped or the conflict logs are dropped. + +By default, the Conflict Logger is tuned to handle a low percentage of conflicts. + +`xdcr_clog_status` is a Prometheus stat, which indicates the status of the Conflict Logger. +The status can be one of the following: + +* *Hibernated*: The logger is temporarily disabled due to persistent errors or resource constraints. ++ +NOTE: Hibernation prevents the degradation of the replication performance. Logging is re-enabled after a set interval, or after the errors are manually resolved and the logger is manually restarted. + +* *Running*: The logger is actively processing conflict events. + +* *DNE*: The logger does not exist; conflict logging is not enabled. + +[#tunable-replication-settings] +== Conflict Log Settings + +By default, the xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-logger[Conflict Logger] is tuned to handle a low percentage of conflicts. + +You can retrieve the conflict logging settings for a replication using the REST API `GET /settings/replications/`. +For more information, see xref:rest-api:rest-xdcr-adv-settings.adoc#rest-api-conflict-logging-replication-tunables[Retrieve Conflict Logging Settings]. +The conflict logger settings start with the prefix `cLog`. + +NOTE: Couchbase recommends that you must not adjust the conflict logger settings unless advised to do so by the Couchbase Support. + +The following table explains the conflict logger settings. +The `conflictLogging` setting, used to enable and configure logging of conflicts, is explained in xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#configure-conflictlogging-settings[Enabling and Configuring Conflict Logging]. + +|=== +| Setting | Description | Default + +| `conflictLogging` +| The main setting which when non-empty and valid enables logging of conflicts. + +For more information, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflictlogging-key[Replication Setting conflictLogging key]. +| + +| `cLogSetTimeoutMs` +| Sets the timeout for SetMeta operation for conflict writing. +| 5000ms + +| `cLogPoolGetTimeoutMs` +| Timeout for getting connection from connection pool +| 5000ms + +| `cLogNetworkRetryCount` +| Number of attempts for a single doc when there is a network error +| 5 + +| `cLogNetworkRetryIntervalMs` +| Time interval to sleep between attempts to write the conflict doc on network error +| 2000ms + +| `cLogWorkerCount` +| Number of workers used by the conflict logger, which actually write conflict documents. +| 20 + +| `cLogQueueCapacity` +| Queue capacity for the conflict logger. +| 6000 + +| `cLogMaxErrorCount` +| Max number of errors encountered by conflict logger in a “Error Window” after which the conflict logger is hibernated. + +For information about hibernation, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-logger[Conflict Logger]. +| 10 + +| `cLogErrorTimeWindowMs` +| This is the time window in which the number of errors in the conflict logger will be checked. +| 120000ms + +| `cLogReattemptDurationMs` +| This is the duration for which the conflict logger will be hibernated and after this the logger will be active again. + +For information about hibernation, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-logger[Conflict Logger]. +| 600000ms +|=== + +[#configure-conflictlogging-settings] +== Enabling and Configuring Conflict Logging + +The XDCR Conflict Logging feature can be configured and enabled per replication using the advanced replication setting `conflictLogging`. +You can use the `conflictLogging` key to mention the storage location for conflict records, and to enable or disable conflict logging. + +It allows you to define a default conflict collection and, optionally, granular logging rules for scopes and collections via the `loggingRules` object. +By configuring `conflictLogging`, you control whether conflicts are logged, and if so, in which bucket and collection the conflict records are saved. +For more information, see the examples and formats below. + +To log conflicts, when creating an XDCR replication use the following settings, which accepts a key and a corresponding JSON object as the value. + +* xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflictlogging-key[`conflictLogging`] is the key. +* xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#logging-rules[`loggingRules`] is a JSON object as the value. + +You can also update an existing replication's conflict logging settings by updating the `conflictLogging` value, which is a part of the xref:rest-api:rest-xdcr-adv-settings.adoc[Manage Advanced Settings] in XDCR replication. + +To configure the key and value: + +* From UI, see xref:manage:manage-xdcr/create-xdcr-replication.adoc[Create a Replication]. For reference, see xref:manage:manage-xdcr/replicate-using-scopes-and-collections.adoc[Replicate Using Scopes and Collections]. + +* Using REST API, when xref:rest-api:rest-xdcr-create-replication.adoc[Creating a Replication], you need to configure the xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflictlogging-key[conflictLogging key] and if necessary, the xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#logging-rules[loggingRules]. + +When configuring conflict logging, you must always specify a default conflict log collection. +All conflicts from the replication are logged into the default conflict log collection. +If `loggingRules` are specified, then the `loggingRules` override the default setting. + +[#conflictlogging-key] +=== Replication Setting conflictLogging key + +The types of values for the `conflictLogging` key are as follows: + +* xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#enabled-for-all-conflicts[Enabled for all conflicts] + +* xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#disabled-conflict-logging[Disabled] + +* xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#enabled-with-special-logging-rules[Enabled with logging rules] + +* xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#disabled-with-stored-logging-rules[Disabled with logging rules configured] + +NOTE: The conflict log collections, or conflict collections, can be any collection in the cluster, including a collection in the bucket that's being replicated. +However, the conflict logs are never replicated. +XDCR adds a system extended attribute or xattr called `_xdcr_conflict` into each of the 3 documents in the xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-record[Conflict Records], which prevents these documents, the conflict logs, from being replicated. + +The following examples show how to use `conflictLogging` with the `curl` command option `--data-urlencode`. +This demonstrates how to specify the conflict logging settings when creating or updating an XDCR replication using the REST API. + +[#enabled-for-all-conflicts] +==== Enabled for all conflicts + +To log all conflicts to the same collection for the entire replication: + +. Enable conflict logging by setting `disabled` to `false`. + +. Designate a default conflict collection by specifying the conflict log bucket and the conflict log collection in `bucket` and `collection`. + +NOTE: You can designate any regular collection as the conflict collection, including collections that are a part of the replication. + +*Format*: +[source,json] +---- +--data-urlencode conflictLogging= '{ + "disabled": false, + "bucket": [bucketName], + "collection": [scope].[collection], + }' +---- + +*Example*: The conflict logging is enabled and the default conflict collection is `conflictlogsbucket.myappscope.log1collection`. +All conflicts in the replication will be logged to the default conflict collection. + +[source,json] +---- +--data-urlencode conflictLogging= '{ + "disabled": false, + "bucket": "conflictlogsbucket", + "collection": "myappscope.log1collection" + }' +---- + +[#disabled-conflict-logging] +==== Disabled + +To disable conflict logging and clear all the `conflictLogging` configuration values, set `--data-urlencode conflictLogging='{}'`. + +[#enabled-with-special-logging-rules] +==== Enabled with logging rules + +To specify different conflict collection rules for different scopes and collections in your replication, define `loggingRules` in the `conflictLogging` settings. + +You can use `loggingRules` to specify, for any scope or collection in the replication, whether the conflicts from that scope or collection must be logged to a different conflict collection or not. +If you do not specify in the logging rules, then by default, the conflicts from all collections are logged to the default conflict collection. + +*Format*: +[source,json] +---- +--data-urlencode conflictLogging= '{ + "disabled": false, + "bucket": [bucketName], + "collection": [scope].[collection], + "loggingRules": {[logging_rules]}, + }' +---- + +*Example*: The conflict logging is enabled. +The default conflict collection is the same as in the previous example, however, `loggingRules` is added. +The `conflictLogging` setting defines the following rules: + +* Conflict logs for the collection `agents.partners` must be logged to a different collection called `conflictlogsbucket2.myappscope.partnerlogs` + +* Conflicts must not be logged for the collection `agents.retailers`. + +* Conflict logs for the `inventory` scope must be logged to a collection called `conflictlogsbucket2.myappscope.inventorylogs`. + +* Conflict logs for the scopes and collections unaffected by `loggingRules` are stored in the default conflict log collection `conflictlogsbucket1.myappscope.log1collection`. + +[source,json] +---- +--data-urlencode conflictLogging= '{ +"disabled": false, +"bucket": "conflictlogsbucket1", +"collection": "myappscope.log1collection" +"loggingRules": { + "agents.partners": { + "bucket": "conflictlogsbucket2", + "collection": "myappscope.partnerlogs" + }, + "agents.retailers": null, + "inventory": { + "bucket": "conflictlogsbucket2", + "collection": "myappscope.inventorylogs" + } + } +}' +---- + +[#disabled-with-stored-logging-rules] +==== Disabled with logging rules configured + +You can configure `conflictLogging` (specify the default conflict collection and the `loggingRules`) but not enable `conflictLogging` by setting `disabled` to `true`. + +NOTE: You can update the replication and set the `conflictLogging` status `disabled` to `true` or `false` at any time. + +When you set `"disabled": true` and have specified other configurations, then the specified values are retained. + +*Format*: + +[source,json] +---- +--data-urlencode conflictLogging= '{ + "bucket": [bucketName], + "collection": [scope].[collection], + "loggingRules": {[logging_rules]}, + "disabled": true, +}' +---- + +*Example*: Conflict logging is disabled, but the default conflict collection and the logging rules are retained. + +[source,json] +---- +--data-urlencode conflictLogging= '{ + "disabled": true, + "bucket": "conflictlogsbucket1", + "collection": "myappscope.log1collection", + "loggingRules": { + "agents.partners": { + "bucket": "conflictlogsbucket2", + "collection": "myappscope.partnerlogs" + }, + "agents.retailers": null, + "inventory": { + "bucket": "conflictlogsbucket2", + "collection": "myappscope.inventorylogs" + } + } +}' +---- + +[#logging-rules] +=== loggingRules in conflictLogging + +Use `loggingRules` to override the xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#enabled-for-all-conflicts[default conflict logging settings]. + +NOTE: Conflict logging rules can be saved without enabling the Conflict Logging setting, however, the logging rules come into effect when the Conflict Logging setting is enabled. + +You can use the `loggingRules` JSON object to define granular conflict logging rules for specific scopes and collections within a replication. + +* You can assign a custom conflict log collection to a specific scope or collection. + +* If you set a custom conflict log collection for a scope, all collections within that scope will use the specified collection for conflict records. +However, for a collection under the scope, you can: + +** Either assign a different custom conflict log collection. +** Or, specify that the conflict records be put in the default conflict log collection and not in the parent scope's custom conflict log collection. + +* You can specify that conflicts from a scope or a collection not be logged. + +The assigned `loggingRules` override the default setting of putting the conflict records into the default conflict log collection specified for `conflictLogging`. +For examples about using loggingRules, see the xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#enabled-with-special-logging-rules[conflictLogging with loggingRules example]. + +The following section explains the key-value formats of the `loggingRules`. + +|=== +| Key | Value | Description + +| [scope] +| `{}` + +Empty JSON object. +| All conflicts will be logged in the conflict collection defined by bucket and collection in the *conflictLogging* key. + +| [scope] +| `null` +| All conflicts from the specified scope will not be logged. + +| [scope.collection] +| `{}` +| All conflicts from the specified collection will be logged in the conflict collection defined by bucket and collection in the *conflictLogging* key. + +You may use this option if the parent scope of the collection has a custom conflict collection, and you want the conflicts from this collection to be logged in the default conflict collection, and not in the parent scope's custom conflict collection. + +The default behavior, when you define a custom conflict collection for a scope, is that all the conflicts for the collections in that scope are logged into the scope's custom conflict collection. + +| [scope.collection] +| `null` +| All conflicts from the specified collection will not be logged. + +| [scope] +|`{ + “bucket”: [otherBucket], + “collection”: [otherScope].[otherCollection]” + }` +| All conflicts from the chosen scope will be logged in the specific conflict collection defined by otherBucket, otherScope, and otherCollection; instead of the original location mentioned in the *conflictLogging* key. + +| [scope.collection] +|`{ + “bucket”: [otherBucket], + “collection”: [otherScope].[otherCollection]” + }` +| All conflicts from the chosen collection will be logged in the specific conflict collection defined by otherBucket, otherScope, and otherCollection; instead of the original location mentioned in the *conflictLogging* key. +|=== + +[#disable-conflict-logging] +== Disable Conflict Logging + +This section explains disabling XDCR Conflict Logging feature. + +* *HLV*: The prerequisite to using the XDCR Conflict Logging feature is to enable the bucket property `enableCrossClusterVersioning` on all the buckets of the replication topology. +This enables adding or updating the Hybrid Logical Vector (HLV) metadata when the documents are replicated. +As the bucket property `enableCrossClusterVersioning` cannot be disabled after it has been enabled, the update and maintenance of HLV metadata by XDCR cannot be disabled. ++ +For more information, see xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning]. + +* *Disabling Conflict Logging*: Update the replication setting `conflictLogging` value to `{}` or `{ "disabled": true }` to disable `conflictLogging` and remove previous configuration. ++ +To keep the other `conflictLogging` configuration settings and only update `"disabled"` to `true`, then make sure not to remove the other configuration values. +For example, `{"disabled": true, ...}`. ++ +For configuration information, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#configure-conflictlogging-settings[Enabling and Configuring Conflict Logging]. +When it is disabled, any conflict documents previously created by XDCR in the conflict collections remain in place. + +* *Deleting a Replication*: When you delete a replication, conflict record documents in the conflict collection remain until you manually delete them. +After XDCR writes the conflict record documents into the conflict collection, you manage the documents. +As you can change the conflict collection multiple times during the lifecycle of a replication, XDCR will not clean up or delete previously created conflict record documents. \ No newline at end of file diff --git a/modules/learn/pages/clusters-and-availability/xdcr-conflict-resolution.adoc b/modules/learn/pages/clusters-and-availability/xdcr-conflict-resolution.adoc index 96ec6df573..9c5c644916 100644 --- a/modules/learn/pages/clusters-and-availability/xdcr-conflict-resolution.adoc +++ b/modules/learn/pages/clusters-and-availability/xdcr-conflict-resolution.adoc @@ -6,37 +6,21 @@ {description} [#conflicts_and_their_resolution] -== Conflicts and Their Resolution +== Conflict Resolution -A _conflict_ is caused when the source and target copies of an XDCR-replicated document are updated independently of and dissimilarly to one another, each by a local application. -The conflict must be _resolved_, by determining which of the variants should prevail; and then correspondingly saving both documents in identical form. -XDCR provides an automated _conflict resolution_ process. +When a source document is modified, XDCR determines whether this revision of the document must be applied to the target. +This process is called conflict resolution, which is a fully automated process. +XDCR supports the following two alternative conflict resolution policies: -Two, alternative conflict resolution policies are supported: _sequence-number-based_ (which is the default), and _timestamp-based_. -Note that _timestamp-based_ conflict resolution is only available in the Enterprise Edition of Couchbase Server. - -[#the_conflict_resolution_process] -== The Conflict Resolution Process - -When a source document is modified, XDCR determines whether this revision of the document should be applied to the target. -For documents above 256 bytes in size, XDCR fetches metadata from the target cluster before replicating. -The target metadata for the document is compared with the source metadata for the document, in order to choose which document should prevail (the exact subset of metadata used in this comparison depends on the source bucket's _conflict resolution policy_). -If the source document prevails, it is replicated to the target; if the target document prevails, the source document is not replicated. - -Once a replicated document reaches the target, the target cluster also performs a metadata comparison as described, in order to confirm that the document from the source cluster should indeed prevail. If this is confirmed, the document from the source cluster is applied to the target cluster, and the target cluster's previous version of the document is discarded. - -As a performance optimization, XDCR makes no metadata comparison on the source for documents of 256 bytes or less, thus making unnecessary a metadata fetch from the target cluster: instead, the document is replicated immediately to the target, and metadata comparison is performed there. - -If a document is deleted on the source, XDCR makes no metadata comparison on the source before replication. - -Once configured, conflict resolution is a fully automated process, requiring no manual intervention. +* Sequence number-based conflict resolution (This is the default policy). +* Timestamp-based conflict resolution. [#revision-id-based-conflict-resolution] == Conflict Resolution Based on Sequence Number Conflicts can be resolved by referring to documents' _sequence numbers_. Sequence numbers are maintained per document, and are incremented on every document-update. -A document's sequence number is stored as part of its _metadata_: specifically, as the value of the `rev` key (see xref:manage:manage-ui/manage-ui.adoc#console-documents[Documents], for details on how to inspect metadata). +A document's sequence number is stored as a part of its _metadata_: specifically, as the value of the `rev` key (see xref:manage:manage-ui/manage-ui.adoc#console-documents[Documents], for details on how to inspect metadata). The sequence numbers of source and target documents are compared; and the document with the higher sequence number prevails. If both documents have the same sequence number, the conflict is resolved by comparing the following metadata-elements, in the order shown: @@ -44,6 +28,9 @@ If both documents have the same sequence number, the conflict is resolved by com . Expiration (TTL) value . Document flags +When Cross Cluster Versioning is enabled, the Hybrid Logical Vector (HLV) metadata in the source and target documents' xattrs is also used in the conflict resolution processing. +For more information about the `enableCrossClusterVersioning` property and the HLV metadata, see xref:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning]. + [#timestamp-based-conflict-resolution] == Timestamp-Based Conflict Resolution @@ -57,6 +44,8 @@ If both document-versions have the same timestamp-value, the conflict is resolve . Expiration (TTL) value . Document flags +When Cross Cluster Versioning is enabled, the Hybrid Logical Vector (HLV) metadata in the source and target documents' xattrs is also used in the conflict resolution processing. For more information about the `enableCrossClusterVersioning` property and the HLV metadata, see xref:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning]. + [#time-synchronization] === Time Synchronization @@ -84,7 +73,7 @@ Each mutation has its own HLC timestamp. [#ensuring_safe_failover] === Ensuring Safe Failover -When failover (say, from data center A to data center B) is required, timestamp-based conflict resolution requires that applications redirect traffic to data center B only after the greater of the following two time-periods has elapsed: +When failover of an application is required (say, from data center A to data center B), timestamp-based conflict resolution requires that applications redirect traffic to data center B only after the greater of the following two time-periods has elapsed: * The replication latency between data centers A and B. This provides sufficient time for any _in-flight_ mutations to be received by data center B prior to traffic redirection. @@ -98,8 +87,16 @@ When availability is restored to data center A, applications must wait for the s Conflict resolution policy is configured on a per-bucket basis at bucket creation time, it cannot be changed later. For more information, see xref:manage:manage-buckets/create-bucket.adoc[Create a Bucket]. -Choosing a conflict resolution method requires consideration of the logic of the applications that require the data. -This is illustrated by the following examples: + +[IMPORTANT] +==== +* You must select the same conflict resolution policy for all the buckets in the replication topology because you can create a replication between only those buckets that have the same conflict resolution policy. +* When creating a bucket, you must actively choose the conflict resolution policy. +If you do not choose a policy, the Sequence number-based conflict resolution policy is set as default. +* After the bucket is created, you cannot change the conflict resolution policy for that bucket. In general, the Timestamp-based conflict resolution policy is preferred as the logic is easier to understand, feasible with general use cases, and also preferred for working with the latest Server features. +==== + +The following examples illustrate how the two different conflict resolution policies apply: * _Sequence-Number-based_, whereby the document with the higher number of updates wins. A hit-counter, for a website, is stored as a document within Couchbase Server: a value within the document is incremented each time the website is accessed. @@ -114,9 +111,10 @@ Therefore, in this instance, timestamp-based conflict resolution should be used, [#aligning_source_and_target_policies] == Aligning Source and Target Policies -XDCR replications cannot be created between buckets with different conflict resolution policies: source and target buckets must always be configured with the same policy. +XDCR replications cannot be created between buckets with different conflict resolution policies. The source and target buckets must always be configured with the same conflict resolution policy. -When using XDCR with a source cluster running a pre-4.6 version of Couchbase Server, only conflict resolution based on _sequence numbers_ can be used. +When creating a bucket, you must actively choose the conflict resolution policy. +If you do not choose a policy, the Sequence number-based conflict resolution policy is set as default. After the bucket is created, you cannot change the conflict resolution policy for that bucket. In general, the Timestamp-based conflict resolution policy is preferred as the logic is easier to understand. [#monitoring-conflict-resolution] == Monitoring Conflict Resolution on the Target Cluster diff --git a/modules/learn/pages/clusters-and-availability/xdcr-enable-crossclusterversioning.adoc b/modules/learn/pages/clusters-and-availability/xdcr-enable-crossclusterversioning.adoc new file mode 100644 index 0000000000..09fff9f075 --- /dev/null +++ b/modules/learn/pages/clusters-and-availability/xdcr-enable-crossclusterversioning.adoc @@ -0,0 +1,73 @@ += XDCR enableCrossClusterVersioning +:description: pass:q[Enabling Cross Cluster Versioning allows XDCR to add metadata to each replicated document.] + +[abstract] +{description} + + +Enabling Cross Cluster Versioning for all buckets in the replication topology is a pre-requisite for some XDCR features. +The bucket property `enableCrossClusterVersioning` cannot be disabled once it is set to `true`. +Therefore, you must not enable `enableCrossClusterVersioning` casually. + +When you set the bucket property `enableCrossClusterVersioning` (ECCV) to `true`, for each document processed by XDCR, XDCR stores additional metadata for the document in the extended attributes. +This metadata is also called Hybrid Logical Vector (HLV), which is a set of Hybrid Logical Clock (HLC) information. + +[#hlv-data-maintained-in-xattr] +== Hybrid Logical Vector (HLV) data maintained in the extended attributes + +The new metadata, HLV, is stored as a system created extended attribute (xattrs) called `_vv` (`xattrs._vv`). The HLV metadata is also called Version Vectors. + +The HLV metadata takes up `109 + 40N` bytes of space per document, where N is the number of buckets mutating the document in the replication topology. + +NOTE: As long as your replication topology is constant, the size of the HLV metadata will grow to `109 + 40N` bytes and remain constant. +However, the HLV data accumulation occurs when the replication topology changes and the document copy goes through different clusters, for example, when being restored to a new cluster. + +To remove the accumulated or outdated HLV data, the HLV metadata is pruned periodically. +You can control the pruning frequency by setting the bucket property `versionPruningWindowHrs`. + +[#remove-hlv-metadata] +=== Remove HLV metadata + +After enabling, you cannot disable the bucket property `enableCrossClusterVersioning`. +You can, however, backup and restore the data to a bucket where `enableCrossClusterVersioning` is set to `false`, and remove the xattrs information added by XDCR using the option `cbbackupmgr restore --disable-hlv`. + +NOTE: To remove the on-going maintenance of HLV information, you must remove the xattrs information previously added to the documents. +To completely remove the effects of enabling HLV, only restoring the data to a bucket where `enableCrossClusterVersioning` is set to `false` is not enough. + +[#modify-enablecrossclusterversioning] +== Modify the bucket property enableCrossClusterVersioning + +To modify or enable the bucket property `enableCrossClusterVersioning`: + +* Using REST API, see xref:rest-api:rest-bucket-create.adoc#example-enablecrossclusterversioning-edit[Example: Turning on enableCrossClusterVersioning, when Editing]. + +* From the UI, see xref:manage:manage-buckets/edit-bucket.adoc#edit-bucket-for-eccv[Edit a Bucket to Enable Cross Cluster Versioning]. + +NOTE: You cannot enable the bucket property `enableCrossClusterVersioning` while creating the bucket. + +[#version-pruning-window-hrs] +== versionPruningWindowHrs + +`versionPruningWindowHrs` is a bucket property, which controls the pruning frequency of the HLV metadata. +The default value of versionPruningWindowHrs is 720 hours (30 days), which means that any HLV data older than 720 hours is pruned to remove the outdated entries. + +NOTE: `versionPruningWindowHrs` must be set to the same value for all buckets in an XDCR replication topology. + +[#manage-version-pruning-window-hrs] +=== Manage the bucket property versionPruningWindowHrs + +For information about modifying the bucket property `versionPruningWindowHrs` through REST API, see xref:rest-api:rest-bucket-create.adoc#example-versionpruningwindowhrs-edit[Example: Specifying time value for versionPruningWindowHrs, when Editing]. + +[#features-requiring-crossclusterversioning-enabled] +== Features for which Cross Cluster Versioning must be enabled + +* *XDCR Active-Active with Sync Gateway*: The bi-directional, active-active replication with Sync Gateway 4.0* or a later version and XDCR requires enabling Cross Cluster Versioning. +For more information, including important limitations, see xref:learn:clusters-and-availability/xdcr-active-active-sgw.adoc[XDCR Active-Active with Sync Gateway]. ++ +NOTE: *To set up XDCR bi-directional replication with Sync Gateway (SGW), you need to have at least a Server 8.0.0 version and SGW 4.0.0 version. +However, Sync Gateway 4.0.0 is a future release version. + +* *XDCR Conflict Logging*: XDCR Conflict Logging requires enabling Cross Cluster Versioning. +For more information, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc[XDCR Conflict Logging Feature]. + +NOTE: Only the documents mutated after the `enableCrossClusterVersioning` property is enabled on both the source and target clusters, are considered for conflict logging. \ No newline at end of file diff --git a/modules/learn/pages/clusters-and-availability/xdcr-viewing-conflict-logs.adoc b/modules/learn/pages/clusters-and-availability/xdcr-viewing-conflict-logs.adoc new file mode 100644 index 0000000000..1fe70be928 --- /dev/null +++ b/modules/learn/pages/clusters-and-availability/xdcr-viewing-conflict-logs.adoc @@ -0,0 +1,356 @@ += Viewing Conflict Logs +:description: pass:q[View and access conflict logs from the conflict collection.] +:page-toclevels: 3 + +[abstract] +{description} + +When enabling XDCR conflict logging, you must specify the xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-collection[Conflict Log Collection] or conflict collection. +Conflict logs or conflict records are stored as documents within these collections. +As conflict collections are regular collections, you can use RBAC to control access to the conflict logs, which include copies of your application data. +You can also programmatically process and view the conflict records and documents. + +The following sections have examples of how to access, process, or locate information within conflict log documents using {sqlpp} queries, an Eventing function, and a KV Range Scan. +While these are some common methods, other options exist for viewing or processing conflict log records, such as using Couchbase Kafka Connect to stream these documents to an application for further processing. + +After XDCR logs a conflict, it does not modify or remove the conflict records. +You're responsible for managing and deleting them when no longer needed. +Each conflict record has 3 documents. +You can find the following in the conflict collection: + +* A conflict event record in `docId` with the prefix `crd_`. +* A copy of the source document in `docId` with the prefix `src_`. +* A copy of the target document in `docId` with the prefix `tgt_`. + +In the conflict event record, the field `docId` is the ID or key of the conflicting document in the replication. +The documents in the conflict collection that have IDs with prefixes `src_` and `tgt_` are copies of the versions of `docId` that were in conflict. + +For information about the Conflict Record Document, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-record[Conflict Record]. + +[#using-sdk-sqlpp-for-conflict-logs] +== Using {sqlpp} to Query Conflict Logs + +The most common way to view these documents is by using {sqlpp}. + +The following are a few examples. +In the examples, the conflict logs are stored in the collection `conflictlogs.bucket1.app1logs`. + +[#sqlpp-examples-clog] +=== {sqlpp} Examples + +For queries, you can create a custom index or a primary index. + +* Create a primary index as follows. ++ +[source,sqlpp] +---- +CREATE PRIMARY INDEX `#primary` ON `conflictlogs`.`bucket1`.`app1logs` +---- + +* Create a custom index as follows. ++ +You can use the `WHERE` clause with `SUBSTR(META().id,0,3) = “crd”` by making it an indexed field instead of `WHERE META().id LIKE "crd%"`. ++ +For example: ++ +[source,sqlpp] +---- +CREATE INDEX `idx_crd_timestamp_docId` ON `conflictlogs`.`bucket1`.`applogs`(SUBSTR(META().id,0,3), `timestamp`,`docId`) +---- ++ +[NOTE] +==== +* When creating custom indexes, `META().id` is already a part of the index key. +* When querying, if you use underscore in comparison strings, make sure you use the escape character, so that it's used as an exact comparison instead of a wildcard comparison. +For example, `WHERE META().id LIKE “crd\\_%”`. +==== + +*Query example 1*: List all `docId` that had conflicts between two time periods. + +[source,sqlpp] +---- +SELECT META().id, b.docId, b.timestamp +FROM `conflictlogs`.`bucket1`.`app1logs` AS b +WHERE META().id LIKE "crd%" and b.timestamp between "2025-06-04T18:15:16.363Z" AND "2025-06-09T07:26:40.326Z" +---- + +*Query example 2*: Number of conflicts logged between two time periods. + +[source,sqlpp] +---- +SELECT count(*) AS num_conflicts +FROM `conflictlogs`.`bucket1`.`app1logs` AS b +WHERE META().id LIKE "crd%" and b.timestamp between "2025-06-04T18:15:16.363Z" AND "2025-06-09T07:26:40.326Z" +---- + +*Query example 3*: Number of different `docIds` that had conflicts between two time periods. + +[source,sqlpp] +---- +SELECT count (distinct b.docId) as num_different_docIds +FROM `conflictlogs`.`bucket1`.`app1logs` AS b +WHERE META().id LIKE "crd%" and b.timestamp between "2025-06-04T18:15:16.363Z" AND "2025-06-09T07:26:40.326Z" +---- + +*Query example 4*: List all `docIds` and the number of times each `docId` had a logged conflict between 2 time periods. + +The following query may be used if you suspect that the same few documents are being updated multiple times concurrently in two different clusters, generating conflicts. + +[source,sqlpp] +---- +SELECT b.docId AS DOCID, COUNT(1) AS num_times_seen +FROM `conflictlogs`.`bucket1`.`app1logs` AS b +WHERE META(b).id LIKE "crd%" AND b.timestamp BETWEEN "2025-06-04T18:15:16.363Z" AND "2025-06-09T07:26:40.326Z" +GROUP BY b.docId +ORDER BY num_times_seen DESC LIMIT 20 +---- + +The query result is as follows: + +---- +[ + { + "DOCID": "4d9d18e7-5fa2-4eae-b32b-60d3d369da94", + "num_times_seen": 2 + }, + { + "DOCID": "ae03ecde-203c-4a3b-a6d0-36b60f627c8a", + "num_times_seen": 1 + } +] +---- + +*Query example 5*: In the following example, for each `docId`, you can see selected fields from the source and target documents. +Also, the `docId` of the copies of the source and target documents in the conflict collection, if you want to retrieve the entire document. +The only fields that you want to see from the documents that were in conflict are `country` and `note`. + +[source,sqlpp] +---- +SELECT b.docId, b.srcDoc.id AS srcDocId, b.tgtDoc.id AS tgtDocId, +(SELECT b2.country, b2.note +FROM `conflictlogs`.`bucket1`.`app1logs` AS b2 USE KEYS b.srcDoc.id)[0] AS src_doc_fields, +(SELECT b3.country, b3.note +FROM `conflictlogs`.`bucket1`.`app1logs` AS b3 USE KEYS b.tgtDoc.id)[0] AS tgt_doc_fields +FROM `conflictlogs`.`bucket1`.`app1logs` AS b +WHERE META(b).id LIKE "crd%" AND b.timestamp BETWEEN "2025-06-04T18:15:16.363Z" AND "2025-06-09T07:26:40.326Z" +---- + +The query result is as follows: +---- +[ + { + "docId": "ae03ecde-203c-4a3b-a6d0-36b60f627c8a", + "srcDocId": "src_1749454000005461807_e2f416ed317beb4f16a3587656b3d3450e1bec893cabebd3b9bb857d63824da4", + "tgtDocId": "tgt_1749454000005461807_e2f416ed317beb4f16a3587656b3d3450e1bec893cabebd3b9bb857d63824da4", + "src_doc_fields": { + "country": "UK", + “note”: “need review” + }, + "tgt_doc_fields": { + "country": "CA", + “note”: “ok” + } + } +] +---- + +[#using-eventing-for-conflict-logs] +== Using Eventing to Process Conflict Logs + +If you do not like the 3 documents format of the conflict record, then you can create a single document from the 3 documents (`crd_`, `src_`, `tgt_`) for easier processing by using Eventing. +Here, it's assumed that the documents are small. +If the documents are large such that the combined 3 documents are larger than the maximum xref:learn:clusters-and-availability/size-limitations.adoc[size limit]. +Then you can copy only a subset of the fields from the `src_` and `tgt_` documents to the custom single document. + +Another effect of creating your own conflict document as shown in the Eventing example is that the new document that you create will not have the system extended attribute (`{"_xdcr_conflict": true}`). +This system extended attribute is what prevents the conflict documents created by XDCR from being replicated. +As a result, the new combined conflict log document will have the `docId` (or doc key) of your preference, and if needed, it can be replicated. + +[#eventing-example-clog] +=== Eventing example + +This is an Eventing function example to copy the conflict log documents into a single new document. + +Eventing bucket alias: `src` for `conflictlogs._default.bucket1logs` with read only access. + +Eventing bucket alias: `copy` for `conflictlogs._default.bucket1logs_copy` with read/write access. + +[source,javascript] +---- +function OnUpdate(doc, meta, xattrs) { + if (meta.id.startsWith("crd_") === true) + { + log("crd doc created/updated", meta.id); + + //get the conflict source and target document id's + var suffix = (meta.id).substring(4); + var srcId = "src_" + suffix; + var tgtId = "tgt_" + suffix; + + //read the conflicting source and target documents + var src_doc = src[srcId]; + var tgt_doc = src[tgtId]; + + //add the conflicting source and target documents to the crd doc + doc["src_doc"] = src_doc; + doc["tgt_doc"] = tgt_doc; + + //copy the resulting new crd doc to a different collection + copy[meta.id] = doc; + } +} +---- + +Resultant document example: + +NOTE: See xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-record-document-format[Conflict Record Document Format] to understand the document format. + +---- +{ + "timestamp": "2025-05-20T19:25:55.308Z", + "id": "crd_1747769155308581666_c0468a42632daabce649b1fb7e174a530946c04b1dde75218290c4885ad11669", + "docId": "ASHUNK100", + "replId": "1bc375a26b7f822ebe91288c58a6c614/bucket1/bucket1-PN8-oFADADCOe6S6A8YuIw==", + "srcDoc": { + "id": "src_1747769155308581666_c0468a42632daabce649b1fb7e174a530946c04b1dde75218290c4885ad11669", + "nodeId": "10.0.13.10", + "bucketUUID": "6cd3a35ba2d337d84d4fcd30c3a46fb9", + "clusterUUID": "d976db26a5ce512a243db65a3026073f", + "isDeleted": false, + "collection": "_default", + "scope": "_default", + "expiry": 0, + "flags": 33554438, + "cas": 1747769124557291500, + "revSeqno": 4, + "xattrs": { + "_vv": "{\"cvCas\":\"0x00002a1777524118\",\"src\":\"ntR6bcR9VHcjcPpQ5osEiw\",\"ver\":\"0x00002a1777524118\",\"pv\":[\"000018fe76524118@8DdrZBqHaACpmDeYJIv+kg\"]}", + "_sync": "", + "_mou": "" + } + }, + "tgtDoc": { + "id": "tgt_1747769155308581666_c0468a42632daabce649b1fb7e174a530946c04b1dde75218290c4885ad11669", + "nodeId": "10.0.22.102", + "bucketUUID": "fada12469f1027ecb078d2c19a2802fb", + "clusterUUID": "1bc375a26b7f822ebe91288c58a6c614", + "isDeleted": false, + "collection": "_default", + "scope": "_default", + "expiry": 0, + "flags": 33554438, + "cas": 1747769104359030800, + "revSeqno": 4, + "xattrs": { + "_vv": "{\"cvCas\":\"0x000018fe76524118\",\"src\":\"8DdrZBqHaACpmDeYJIv+kg\",\"ver\":\"0x000018fe76524118\"}", + "_sync": "", + "_mou": "" + } + }, +"src_doc": { + "author": "Unknown", + "create_date_utc": "2025-05-20T07:56:26", + "res_from": "WEST", + "res_name": "Harry West", + "res_note": "visitor", + "reserved": true, + "title": "Ashmole History", + "type": "book", + "unix_time": "1747768801", + "update_date_utc": "2025-05-20T19:20:01" + }, + "tgt_doc": { + "author": "Unknown", + "create_date_utc": "2025-05-20T07:56:26", + "res_from": "WEST", + "res_name": "Harry West", + "res_note": "visitor -- HJV NOTE WEST", + "reserved": true, + "title": "Ashmole 782", + "type": "book", + "unix_time": "1747768801", + "update_date_utc": "2025-05-20T19:20:01" + } +} +---- + +[#using-kv-range-scan-clog] +== Using KV Range Scan + +To use only KV, use Couchbase SDK with KV APIs including xref:8.0@server:howtos/kv-operations.adoc#kv-range-scan[KV Range Scan]. + +For example, to get a list of all `crd` prefix documents (CRD documents) using the KV Range Scan xref:8.0@server:howtos/kv-operations.adoc#kv-range-scan-prefix[PrefixScan], use KV `collection.get` to get the conflicting `src` and `tgt` docs. +You can also use KV sub-document API `lookupIn` to retrieve only the selected fields. +After processing the documents, you can remove the conflict log documents using the KV `collection.remove`. + +For more information about KV Range Scan operations and data operations, see xref:8.0@server:howtos/kv-operations.adoc[Data Operations]. For more information about data operations on sub-documents, see xref:8.0@server:howtos/subdocument-operations.adoc[Sub-Document Operations]. + +[#xdcr-kv-range-scan-examples] +=== KV Range Scan examples + +The following example uses Couchbase Python SDK. + +[source,python] +---- +from datetime import timedelta +import json + +from couchbase.auth import PasswordAuthenticator +from couchbase.cluster import Cluster +from couchbase.collection import (GetOptions, RemoveOptions) +from couchbase.collection import DeltaValue, SignedInt64 +from couchbase.exceptions import ( + CouchbaseException, + PathNotFoundException, + SubdocPathMismatchException) +from couchbase.options import ScanOptions +from couchbase.kv_range_scan import PrefixScan +import couchbase.subdocument as SD + +# connection info +connection_string = "couchbase://localhost" +username = "username1" +password = "Some!password" + +# conflict logs collection location +bucket_name = "conflictlogs" +scope_name = "bucket1" +collection_name = "app1logs" + +cluster = Cluster( + connection_string, + authenticator=PasswordAuthenticator(username, password)); + +cluster.wait_until_ready(timedelta(seconds=5)) +cb = cluster.bucket(bucket_name); +logs_coll = cb.scope(scope_name).collection(collection_name) + +# Using KV PrefixScan to get list of crd doc ids +# Get crd document ids only +crd_list = logs_coll.scan(PrefixScan('crd'), ScanOptions(ids_only=True)) + +# For each crd document, get the docId, timestamp, srcDoc.id, tgtDoc.id +# Then, get the src and tgt docs from the conflict log collection +for crd in crd_list: + crdlookup = logs_coll.lookup_in(crd.id, + [ SD.get("docId"), + SD.get("timestamp"), + SD.get("srcDoc.id"), + SD.get("tgtDoc.id") ]) + + doc_id = crdlookup.content_as[str](0) + timestamp = crdlookup.content_as[str](1) + src_doc_id = crdlookup.content_as[str](2) + tgt_doc_id = crdlookup.content_as[str](3) + + # Get the src and tgt documents in conflict + src_doc = logs_coll.get(src_doc_id); + tgt_doc = logs_coll.get(tgt_doc_id); + + # Print the info as json + output = { "docId": doc_id, "log_timestamp": timestamp, + "src_doc": src_doc.content_as[dict], + "tgt_doc": tgt_doc.content_as[dict] } + print (json.dumps(output)) +---- + diff --git a/modules/manage/assets/images/manage-buckets/edit-bucket-with-eccv.png b/modules/manage/assets/images/manage-buckets/edit-bucket-with-eccv.png new file mode 100644 index 0000000000..200b2e3faa Binary files /dev/null and b/modules/manage/assets/images/manage-buckets/edit-bucket-with-eccv.png differ diff --git a/modules/manage/assets/images/manage-xdcr/customise-conflict-logging-at-scope.png b/modules/manage/assets/images/manage-xdcr/customise-conflict-logging-at-scope.png new file mode 100644 index 0000000000..15baa94f13 Binary files /dev/null and b/modules/manage/assets/images/manage-xdcr/customise-conflict-logging-at-scope.png differ diff --git a/modules/manage/assets/images/manage-xdcr/customise-conflict-logging-collection.png b/modules/manage/assets/images/manage-xdcr/customise-conflict-logging-collection.png new file mode 100644 index 0000000000..1cf59abdce Binary files /dev/null and b/modules/manage/assets/images/manage-xdcr/customise-conflict-logging-collection.png differ diff --git a/modules/manage/assets/images/manage-xdcr/xdcr-add-replication-screen.png b/modules/manage/assets/images/manage-xdcr/xdcr-add-replication-screen.png index 82a4f0b688..e38a9212e8 100644 Binary files a/modules/manage/assets/images/manage-xdcr/xdcr-add-replication-screen.png and b/modules/manage/assets/images/manage-xdcr/xdcr-add-replication-screen.png differ diff --git a/modules/manage/assets/images/manage-xdcr/xdcr-outgoing-with-conflict-logging.png b/modules/manage/assets/images/manage-xdcr/xdcr-outgoing-with-conflict-logging.png new file mode 100644 index 0000000000..26c2c546a2 Binary files /dev/null and b/modules/manage/assets/images/manage-xdcr/xdcr-outgoing-with-conflict-logging.png differ diff --git a/modules/manage/pages/manage-buckets/create-bucket.adoc b/modules/manage/pages/manage-buckets/create-bucket.adoc index 7290327995..e40df22cdb 100644 --- a/modules/manage/pages/manage-buckets/create-bucket.adoc +++ b/modules/manage/pages/manage-buckets/create-bucket.adoc @@ -175,6 +175,10 @@ For more information about flushing, see xref:manage-buckets/flush-bucket.adoc[F [#add-data-bucket-dialog-expanded] image::manage-buckets/addBucketWithMagmaOption.png[,400,align=center, alt="An image that displays the Add Data Bucket dialog, with a Couchbase Bucket Type and CouchStore Storage Backend selected. The Advanced bucket settings are expanded and to show the default selections for a Couchbase and Couchstore bucket."] +NOTE: Enable Cross Cluster Versioning can be enabled only in the xref:manage:manage-buckets/edit-bucket.adoc[Edit a Bucket] mode. +Enabling Cross Cluster Versioning is a prerequisite for features like XDCR Conflict Logging and XDCR Active-Active with Sync Gateway 4.0+. +For more information, see xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning]. + [#memcached-bucket-settings] ==== Memcached Bucket Settings diff --git a/modules/manage/pages/manage-buckets/edit-bucket.adoc b/modules/manage/pages/manage-buckets/edit-bucket.adoc index ee736ae74c..5e9419e135 100644 --- a/modules/manage/pages/manage-buckets/edit-bucket.adoc +++ b/modules/manage/pages/manage-buckets/edit-bucket.adoc @@ -44,7 +44,25 @@ To display the user-interface for editing, left-click on the *Edit* button: image::manage-buckets/editBucketButton.png[,260,align=left] This displays the [.ui]*Edit Bucket Settings* dialog, which permits changes to be made to a subset of existing settings. -All the settings contained here are described in detail for the [.ui]*Add Data Bucket* dialog, on the page xref:manage-buckets/create-bucket.adoc[Create a Bucket] +All the settings contained here are described in detail for the [.ui]*Add Data Bucket* dialog, on the page xref:manage-buckets/create-bucket.adoc[Create a Bucket]. + +[#edit-bucket-for-eccv] +=== Edit a Bucket to Enable Cross Cluster Versioning + +[.ui]*Edit Bucket Settings* dialog displays the additional setting, *Enable Cross Cluster Versioning*. Use this setting to enable the xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning] bucket property. + +image::manage-buckets/edit-bucket-with-eccv.png[,399,align=center] + +When you enable the bucket setting [.ui]*Enable Cross Cluster Versioning*, for each document processed by XDCR, XDCR stores additional metadata for the document in the extended attributes. +This metadata is called, xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc#hlv-data-maintained-in-xattr[Hybrid Logical Vector (HLV)] or a version vector. + +NOTE: Once enabled, the Enable Cross Cluster Versioning bucket setting cannot be disabled. + +To use the following features, enable the bucket setting [.ui]*Enable Cross Cluster Versioning* on all buckets that are a part of the XDCR topology: + +* xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc[XDCR Conflict Logging]. + +* xref:learn:clusters-and-availability/xdcr-active-active-sgw.adoc[XDCR Active-Active with Sync Gateway]. == Making Changes diff --git a/modules/manage/pages/manage-xdcr/create-xdcr-replication.adoc b/modules/manage/pages/manage-xdcr/create-xdcr-replication.adoc index 75f93f1dcf..591e5354ba 100644 --- a/modules/manage/pages/manage-xdcr/create-xdcr-replication.adoc +++ b/modules/manage/pages/manage-xdcr/create-xdcr-replication.adoc @@ -90,6 +90,8 @@ For more information , see xref:learn:data/scopes-and-collections.adoc#default-s An account of *Advanced Settings* is provided in xref:manage:manage-xdcr/create-xdcr-replication.adoc#xdcr-advanced-settings-pointer[Advanced Replication Settings with the UI], below. + The practical steps required for establishing filters are explained in xref:manage:manage-xdcr/filter-xdcr-replication.adoc[Filter a Replication]. ++ +Information about xref:manage:manage-xdcr/create-xdcr-replication.adoc#xdcr-ui-settings-for-conflict-logging[Replication Settings for XDCR Conflict Logging] is provided in a later section. . Enter appropriate information into the upper fields of the *Add Replication* screen. Specify `10.144.210.102` as the target cluster, and `travel-sample` as both source and target bucket. @@ -113,6 +115,10 @@ Note that by left-clicking on the row for the replication, additional controls c image::manage-xdcr/xdcr-outgoing-replications-with-replication-opened.png[,800,align=left] +When conflict logging is enabled, Outgoing Replications panel displays the Conflict Logging Rules panel, which shows the conflict collection mapping that you have chosen. Also, the bucket indicates Conflict Logging Enabled. + +image::manage-xdcr/xdcr-outgoing-with-conflict-logging.png[,800,align=left] + Use of the *Pause* control is described in xref:manage:manage-xdcr/pause-xdcr-replication.adoc[Pause a Replication]; use of the *Delete* control in xref:manage:manage-xdcr/delete-xdcr-replication.adoc[Delete a Replication]; and use of the *Edit* control in xref:manage:manage-xdcr/filter-xdcr-replication.adoc#editing-filters[Editing Filters]. [#monitor-current-replications] @@ -137,6 +143,48 @@ image::manage-xdcr/xdcr-advanced-settings.png[,400,align=left] The values displayed in the fields are defaults, which can be modified interactively, and saved: this may help in achieving optimal replication-performance. For details on the significance of each field, see the xref:xdcr-reference:xdcr-reference-intro.adoc[XDCR Reference]. +[#xdcr-ui-settings-for-conflict-logging] +=== Replication Settings for XDCR Conflict Logging + +The *XDCR Add Replication* page displays additional settings, which are specific to logging conflicts during an XDCR replication. + +NOTE: To configure and enable XDCR Conflict Logging, you must enable the bucket property `enableCrossClusterVersioning` on all buckets of the XDCR topology. +This bucket property cannot be disabled once it is enabled. +For information about the bucket property `enableCrossClusterVersioning`, see xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning]. + +The additional settings are as follows: + +* *Specify collections for storing conflict logs and documents*: Use this setting to enable choosing a conflict collection for storing conflict logs and documents. +This setting allows you to specify the bucket, scope, and collection where conflict logs and documents will be stored during XDCR conflict logging process. +Conflict Logging Rules panel displays the mapping of conflict collection that you have chosen. + +* *Enable Conflict Logging*: Use this setting to enable or disable conflict logging for the replication. +When enabled, XDCR starts logging conflicts in the specified bucket, scope, and collection. + +* *Default Conflict Log Collection*: Use this setting to specify the default bucket, scope, and collection for conflict logging. +If a custom collection is not specified, the default collection is used. ++ +NOTE: By default, conflicts are logged in the bucket, scope, and collection of the *Default Conflict Log Collection*. Use *Customise Conflict Logging at scope level* and *Customise Conflict Logging at collection level* to override the default settings. + +* *Customise Conflict Logging at scope level*: Use this setting to specify the scope for a custom conflict collection. ++ +image::manage-xdcr/customise-conflict-logging-at-scope.png[,800,align=left] + +* *Customise Conflict Logging at collection level*: Use this setting to specify a custom collection to store conflict logs. +You can choose a custom conflict collection that overrides the default conflict collection. Choose one of the following collection types: +** *Default collection*: To set the collection to the default collection of the bucket. +** *Parent collection*: To set the collection to that of the scope. +** *Custom collection*: To set the collection to a custom collection that you specify. +** *Do not log*: To disable conflict logging for this collection. ++ +image::manage-xdcr/customise-conflict-logging-collection.png[,800,align=left] + +For more information, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#configure-conflictlogging-settings[Enabling and Configuring Conflict Logging]. + +For information about XDCR Conflict Logging, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc[XDCR Conflict Logging]. + +For information about creating collections, see xref:manage:manage-scopes-and-collections/manage-scopes-and-collections.adoc[Manage Scopes and Collections]. + [#error-notifications] === Error Notifications @@ -214,6 +262,19 @@ If successful, this provides the following response: For more information, see xref:rest-api:rest-xdcr-create-replication.adoc[Creating a Replication]. For information on REST-driven configuration of the xref:manage:manage-xdcr/create-xdcr-replication.adoc#xdcr-advanced-settings-pointer[Advanced Settings] described above, see xref:rest-api:rest-xdcr-adv-settings.adoc[Managing Advanced Settings]. +[#create-an-xdcr-replication-with-mobile-as-active] +== Create an XDCR Replication with mobile=Active + +To create or update an XDCR replication with `mobile=Active`, do the following: + +* Create an XDCR replication with `mobile=Active` or update an existing replication. For information about _creating_ (new) an XDCR replication with `mobile=Active`, see xref:learn:clusters-and-availability/xdcr-active-active-sgw.adoc#xdcr-active-active-sgw-greenfield-deployment[Greenfield deployment], and for information about _updating_ an existing replication with `mobile=Active`, see xref:learn:clusters-and-availability/xdcr-active-active-sgw.adoc#xdcr-active-active-sgw-upgrade[Upgrade an existing setup]. +* Create or update an XDCR replication with `mobile=Active` option using the REST API, starting from Server 7.6.6 version. See xref:rest-api:rest-xdcr-create-replication.adoc[Creating a Replication]. +* Create or update a XDCR replication with `mobile=Active` option from the UI, starting from Server 7.6.6 version. See xref:manage:manage-xdcr/create-xdcr-replication.adoc#create-an-xdcr-replication-with-the-ui[Create an XDCR Replication with the UI]. + +The pre-requisite to use `mobile=Active` is to set the bucket property `enableCrossClusterVersioning`. For more information about the bucket property `enableCrossClusterVersioning`, see xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning]. + +To enable the bucket property `enableCrossClusterVersioning` using REST API, see xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc#modify-enablecrossclusterversioning[Modify the bucket property enableCrossClusterVersioning] or xref:rest-api:rest-bucket-create.adoc#example-enablecrossclusterversioning-edit[Example: Turning on enableCrossClusterVersioning, when Editing]. + [#next-xdcr-steps-after-create-replication] == Next Steps diff --git a/modules/rest-api/pages/rest-bucket-create.adoc b/modules/rest-api/pages/rest-bucket-create.adoc index fbc196b313..8de8467b28 100644 --- a/modules/rest-api/pages/rest-bucket-create.adoc +++ b/modules/rest-api/pages/rest-bucket-create.adoc @@ -78,6 +78,7 @@ curl -X POST -u : -d allowedTimePeriod[toHour]= -d allowedTimePeriod[toMinute]= -d allowedTimePeriod[abortOutside]=[ true | false ] + -d versionPruningWindowHrs= ---- All parameters are described in the following subsections. @@ -1090,6 +1091,52 @@ This parameter is ignored if `autoCompactionDefined` is `false` (which is its de See the examples provided above, in xref:rest-api:rest-bucket-create.adoc#example-databasefragmentationthresholdpercentage-create[Example: Specifying a Data Fragmentation Threshold as a Percentage, when Creating] and xref:rest-api:rest-bucket-create.adoc#example-databasefragmentationthresholdpercentage-edit[Example: Specifying a Data Fragmentation Threshold as a Percentage, when Editing]. + +[#enablecrossclusterversioning] +=== enableCrossClusterVersioning + +Enabling Cross Cluster Versioning is a pre-requisite to a few XDCR features. +The bucket property `enableCrossClusterVersioning` can only be set to true after a bucket has been created. +When enabled, for each document processed by XDCR, XDCR stores additional metadata, called the Hybrid Logical Vector (HLV), in the document extended attributes (xattrs). +For more information, see xref:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning]. + +See the example provided in xref:rest-api:rest-bucket-create.adoc#example-enablecrossclusterversioning-edit[Example: Turning on enableCrossClusterVersioning, when Editing] + +CAUTION: The default value is `false`. Do not change the value of this property unless instructed by a feature configuration. Once enabled, you cannot turn off the `enableCrossClusterVersioning` property. The only way for you to undo setting this value to `true` is to backup your data, create a new bucket, and restore the data, using the option `cbbackupmgr restore --disable-hlv` to remove the HLV info in the xattrs. + +[#example-enablecrossclusterversioning-edit] +==== Example: Turning on enableCrossClusterVersioning, when Editing + +The following example modifies the value of the bucket property `enableCrossClusterVersioning` to `true`. + +---- +curl -v -X POST http://localhost:8091/pools/default/buckets/testBucket \ +-u Administrator:password \ +-d enableCrossClusterVersioning=true +---- + +[#version-pruning-window-hrs-property] +=== versionPruningWindowHrs + +Controls the pruning frequency of the Hybrid Logical Vector (HLV) metadata. +The default value of versionPruningWindowHrs is 720 hours (30 days), which means that any HLV data older than 720 hours is pruned to remove the outdated entries. +For more information, see xref:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[versionPruningWindowHrs] in xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc#version-pruning-window-hrs[XDCR enableCrossClusterVersioning]. + +See the example provided in xref:rest-api:rest-bucket-create.adoc#example-versionpruningwindowhrs-edit[Example: Specifying time value for versionPruningWindowHrs, when Editing] + +NOTE: `versionPruningWindowHrs` must be set to the same value for all buckets in an XDCR replication topology. + +[#example-versionpruningwindowhrs-edit] +==== Example: Specifying time value for versionPruningWindowHrs, when Editing + +The following example sets the time duration value for the bucket property `versionPruningWindowHrs`. + +---- +curl -v -X POST http://localhost:8091/pools/default/buckets/testBucket \ +-u Administrator:password \ +-d versionPruningWindowHrs=120 +---- + == Responses If bucket-creation is successful, HTTP response `202 Accepted` is returned, with empty content. @@ -1099,15 +1146,6 @@ If the bucket cannot created due to a missing or incorrect parameter, a `400` re If the URL is incorrectly specified a `404 (Object Not Found)` error is returned. Failure to authenticate gives `401 Unauthorized`. -[#notes] -== Notes - -Couchbase Server 7.6.2 added a property named `enableCrossClusterVersioning` to support a future feature. -Do not change the value of this property. - -CAUTION: Once enabled, you cannot turn off the `enableCrossClusterVersioning` property. -The only way for you to undo setting this value to `true` is to backup your data, create a new bucket, and reload the data into it. - == See Also A conceptual description of buckets is provided in xref:learn:buckets-memory-and-storage/buckets.adoc[Buckets]. diff --git a/modules/rest-api/pages/rest-xdcr-adv-settings.adoc b/modules/rest-api/pages/rest-xdcr-adv-settings.adoc index 45e1c8fa3b..acd438c63d 100644 --- a/modules/rest-api/pages/rest-xdcr-adv-settings.adoc +++ b/modules/rest-api/pages/rest-xdcr-adv-settings.adoc @@ -94,30 +94,42 @@ If successful, the call returns an object similar to the following: ---- { + "casDriftThresholdSecs": 100, + "conflictLogging": null, "checkpointInterval": 600, "ckptSvcCacheEnabled": true, + "collectionsOSOMode": true, "compressionType": "Auto", + "dcpEnablePurgeRollback": false, "desiredLatency": 50, "docBatchSizeKb": 2048, "failureRestartInterval": 10, + "filterBinary": false, "filterBypassExpiry": false, "filterBypassUncommittedTxn": false, "filterDeletion": false, "filterExpiration": false, + "genericServicesLogLevel": { + < ... diagnostic items cut out due to length ... > + }, "goGC": 100, "goMaxProcs": 4, - "hlvPruningWindowSec": 259200, "jsFunctionTimeoutMs": 20000, "logLevel": "Info", "mergeFunctionMapping": {}, + "mobile": "Off", "networkUsageLimit": 0, "optimisticReplicationThreshold": 256, + "preCheckCasDriftThresholdHours": 8760, "preReplicateVBMasterCheck": true, "priority": "High", "replicateCkptIntervalMin": 20, + "retryOnErrExceptAuthErrMaxWaitSec": 360, "retryOnRemoteAuthErr": true, - "retryOnRemoteAuthErrMaxWaitSec": 3600, + "retryOnRemoteAuthErrMaxWaitSec": 360, + "skipReplSpecAutoGc": false, "sourceNozzlePerNode": 2, + "targetTopologyLogFrequency": 1800, "statsInterval": 1000, "targetNozzlePerNode": 2, "workerBatchSize": 500 @@ -133,6 +145,8 @@ Use the following call, which is explained in detail in xref:rest-api:rest-get-c curl -v -X GET http://localhost:8091/pools/default/tasks -u Administrator:password | jq '.' ---- +*Example*: Retrieve ongoing cluster tasks. + If successful, the call returns an array of ongoing cluster tasks, similar to the following: ---- @@ -195,6 +209,8 @@ If successful, the call returns an array of ongoing cluster tasks, similar to th ] ---- +*Example*: Retrieve second member of the array. + A `settings_URI` is provided for each of the ongoing replications, within the second member of the array. The `settings_URI` can be used to retrieve information on the corresponding replication. @@ -209,6 +225,8 @@ If the call is successful, an object containing the settings for the specified r ---- { + "casDriftThresholdSecs": 100, + "conflictLogging": null, "checkpointInterval": 600, "ckptSvcCacheEnabled": true, "colMappingRules": {}, @@ -217,34 +235,77 @@ If the call is successful, an object containing the settings for the specified r "collectionsMirroringMode": false, "collectionsOSOMode": true, "compressionType": "Auto", + "dcpEnablePurgeRollback": false, "desiredLatency": 50, "docBatchSizeKb": 2048, "failureRestartInterval": 10, + "filterBinary": false, "filterBypassExpiry": false, "filterBypassUncommittedTxn": false, "filterDeletion": false, "filterExpiration": false, "filterExpression": "", - "hlvPruningWindowSec": 259200, "jsFunctionTimeoutMs": 20000, "logLevel": "Info", "mergeFunctionMapping": {}, + "mobile": "Off", "networkUsageLimit": 0, "optimisticReplicationThreshold": 256, "pauseRequested": false, + "preCheckCasDriftThresholdHours": 8760, "preReplicateVBMasterCheck": true, "priority": "High", "replicateCkptIntervalMin": 20, + "retryOnErrExceptAuthErrMaxWaitSec": 360, "retryOnRemoteAuthErr": true, - "retryOnRemoteAuthErrMaxWaitSec": 3600, - "sourceNozzlePerNode": 2, + "retryOnRemoteAuthErrMaxWaitSec": 360, + "skipReplSpecAutoGc": false, "sourceNozzlePerNode": 2, "statsInterval": 1000, "targetNozzlePerNode": 2, + "targetTopologyLogFrequency": 1800, "type": "xmem", "workerBatchSize": 500 } ---- +[#rest-api-conflict-logging-replication-tunables] +*Example*: Retrieve the tunable settings for Conflict Logging in a replication. + +After you create a replication with conflict logging enabled, to view the replication settings, retrieve the replication's `settings_URI`. Here's an example. + +---- +curl -X GET -s -u Administrator:'password' http://localhost:8091/settings/replications/f6f3a51e988f305b16433237d02bec7d%2Fbucket3%2Fbucket3 | jq `.` +---- + +If the call is successful, an object is returned, which contains the replication settings for the Conflict Logging feature: + +NOTE: For more information about the replication settings in the Conflict Logging feature, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#tunable-replication-settings[Conflict Log Settings] and xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#configure-conflictlogging-settings[Enabling and Configuring Conflict Logging]. + +---- +{ + "cLogErrorTimeWindowMs": 120000, + "cLogMaxErrorCount": 10, + "cLogNetworkRetryCount": 5, + "cLogNetworkRetryIntervalMs": 2000, + "cLogPoolGetTimeoutMs": 5000, + "cLogQueueCapacity": 6000, + "cLogReattemptDurationMs": 600000, + "cLogSetTimeoutMs": 5000, + "cLogWorkerCount": 20, + "casDriftThresholdSecs": 3900, + "checkpointInterval": 600, + "ckptSvcCacheEnabled": true, + "colMappingRules": {}, + "collectionsExplicitMapping": false, + "collectionsMigrationMode": false, + "collectionsMirroringMode": false, + "collectionsOSOMode": true, + "compressionType": "Auto", + "conflictLogging": {}, +... +} +---- + === Change a Setting for a Specific Replication The following example modifies the value of `checkpointInterval`, for a specific replication: @@ -259,6 +320,8 @@ If successful, the call returns an object containing all current settings for th ---- { + "casDriftThresholdSecs": 100, + "conflictLogging": null, "checkpointInterval": 700, "ckptSvcCacheEnabled": true, "colMappingRules": {}, @@ -267,34 +330,51 @@ If successful, the call returns an object containing all current settings for th "collectionsMirroringMode": false, "collectionsOSOMode": true, "compressionType": "Auto", + "dcpEnablePurgeRollback": false, "desiredLatency": 50, "docBatchSizeKb": 2048, "failureRestartInterval": 10, + "filterBinary": false, "filterBypassExpiry": false, "filterBypassUncommittedTxn": false, "filterDeletion": false, "filterExpiration": false, "filterExpression": "", - "hlvPruningWindowSec": 259200, "jsFunctionTimeoutMs": 20000, "logLevel": "Info", "mergeFunctionMapping": {}, + "mobile": "Off", "networkUsageLimit": 0, "optimisticReplicationThreshold": 256, "pauseRequested": false, + "preCheckCasDriftThresholdHours": 8760, "preReplicateVBMasterCheck": true, "priority": "High", "replicateCkptIntervalMin": 20, + "retryOnErrExceptAuthErrMaxWaitSec": 360, "retryOnRemoteAuthErr": true, - "retryOnRemoteAuthErrMaxWaitSec": 3600, + "retryOnRemoteAuthErrMaxWaitSec": 360, + "skipReplSpecAutoGc": false, "sourceNozzlePerNode": 2, "statsInterval": 1000, "targetNozzlePerNode": 2, + "targetTopologyLogFrequency": 1800, "type": "xmem", "workerBatchSize": 500 } ---- +[#change-existing-replication-with-mobile-active] +=== Change Settings for an Existing Replication to Set mobile=Active + +The following example modifies the value of `mobile` to `Active` for an existing replication: + +---- +curl -X POST -u Administrator:password http://localhost:8091/ -d mobile=Active | jq '.' +---- + +For information about _XDCR with Sync Gateway mobile clusters in a bi-directional, active-active replication_, see xref:learn:clusters-and-availability/xdcr-active-active-sgw.adoc[XDCR Active-Active with Sync Gateway]. + [#xdcr-advanced-settings-rest] == List of Advanced Settings @@ -345,6 +425,14 @@ For information, see the xref:xdcr-reference:xdcr-advanced-settings.adoc#table-o This setting can be established and retrieved either for an individual replication or globally. +| `conflictLogging` +| JSON Object +| Configuration settings for logging conflicts. + +This configuration setting defines objects/parameters and options used to control how conflicts are logged during an XDCR replication. + +For more information, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#configure-conflictlogging-settings[Enabling and Configuring Conflict Logging]. + | `desiredLatency` | Integer | Specifies the amount of time in milliseconds within which a high-priority replication's currently remaining changes should be reduced in number to zero, by ongoing transmission of the changes from source to target bucket. @@ -455,6 +543,11 @@ This setting can be established and retrieved either for an individual replicati | JSON Object | Each of the key-value pairs in the JSON object should map a collection-specifier (in the form `scope.collection`) to the name of a defined merge-function. +| `mobile` +| Active or Off +| Default: `Off`. +When set to `Active`, enables the setting _XDCR Active-Active with Sync Gateway 4.0+_ on the clusters of both sides of the replication. The default value `Off` indicates that the replication setup supports either _XDCR Active-Passive with Sync Gateway_ or _XDCR Active-Active without Sync Gateway_. For more information, see xref:learn:clusters-and-availability/xdcr-active-active-sgw.adoc[XDCR Active-Active with Sync Gateway]. + | `networkUsageLimit` | Integer | Default: 0. @@ -471,6 +564,8 @@ Documents with sizes less than this threshold (in bytes) will be replicated opti This setting can be established and retrieved either for an individual replication or globally. +XDCR optimistic replication is applicable only when the bucket property `enableCrossClusterVersioning` is disabled. For information about the property `enableCrossClusterVersioning`, see xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc#version-pruning-window-hrs[XDCR enableCrossClusterVersioning]. + | `pauseRequested` | Boolean (true or false) | Default: false. diff --git a/modules/rest-api/pages/rest-xdcr-create-replication.adoc b/modules/rest-api/pages/rest-xdcr-create-replication.adoc index b386c64ab2..6ba8a51605 100644 --- a/modules/rest-api/pages/rest-xdcr-create-replication.adoc +++ b/modules/rest-api/pages/rest-xdcr-create-replication.adoc @@ -28,9 +28,10 @@ curl -v -X POST -u [admin]:[password] -d fromBucket=[bucket-name] -d toCluster=[cluster-name] -d toBucket=[bucket-name] - -d collectionsExplicitMapping [true | false] - -d collectionsMigrationMode [true | false] - -d colMappingRules [JSON-Document] + -d conflictLogging=[JSON-Document] + -d collectionsExplicitMapping=[true | false] + -d collectionsMigrationMode=[true | false] + -d colMappingRules=[JSON-Document] -d replicationType="continuous" -d type="xmem" -d compressionType=["None" | "Auto"] @@ -52,6 +53,7 @@ curl -v -X POST -u [admin]:[password] -d statsInterval=[Integer] -d logLevel=[String] -d networkUsageLimit=[Integer] + -d mobile=[Off | Active] ---- The `type` value must be `xmem`; which is sometimes referred to as *Version 2*, and corresponds to the _Memcached Binary_ protocol, used in XDCR communications. @@ -59,6 +61,13 @@ The `type` value must be `xmem`; which is sometimes referred to as *Version 2*, The `replicationType` value is always `continuous`. This value must be specified. +The `conflictLogging` setting is used to configure and enable or disable logging conflicts for the replication. +For configuration information and examples, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#configure-conflictlogging-settings[Enabling and Configuring Conflict Logging]. + +NOTE: To configure and enable Conflict Logging, you must enable the bucket property `enableCrossClusterVersioning` on all buckets of the XDCR topology. +This bucket property cannot be disabled after it is enabled. +For information about the bucket property `enableCrossClusterVersioning`, see xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning]. + If the optional parameters `collectionsExplicitMapping`, `colMappingRules`, and `collectionsMigrationMode` are all _unspecified_, replication occurs between the specified `fromBucket` and `toBucket` according to whatever _implicit mappings_ can be determined at bucket-level: that is, if a collection in the source bucket has the same _keyspace_ as a collection in the target bucket (i.e., the name of a collection and the name of the scope within which it is located form the identical _scope-name.collection-name_ pattern, on both source and target), data is replicated between those collections; with the optional application of a filter expression. At a minimum, this ensures that data is replicated between the default collections. @@ -122,6 +131,11 @@ The value can be `High`, `Medium`, or `Low`. The default value is `High`. For information, see xref:learn:clusters-and-availability/xdcr-overview.adoc#xdcr-priority[XDCR Priority]. +Use the `mobile=[Off | Active]` flag to enable the setting _XDCR Active-Active with Sync Gateway 4.0+_ by changing the value to `Active` on the clusters of both sides of the replication. The default value is `Off` , which indicates that the setup supports either _XDCR Active-Passive with Sync Gateway_ or _XDCR Active-Active without Sync Gateway_. + +[NOTE] +To enable the setting `mobile=[Off | Active]`, ensure you have enabled the property `enableCrossClusterVersioning` on all the participating buckets, which is a prerequisite. For information about the bucket property `enableCrossClusterVersioning`, see xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning]. + For information on all other flags, see xref:rest-api:rest-xdcr-adv-settings.adoc[Managing Advanced XDCR Settings]. [#responses] diff --git a/modules/xdcr-reference/pages/xdcr-advanced-settings.adoc b/modules/xdcr-reference/pages/xdcr-advanced-settings.adoc index a641283ca3..02a5d39829 100644 --- a/modules/xdcr-reference/pages/xdcr-advanced-settings.adoc +++ b/modules/xdcr-reference/pages/xdcr-advanced-settings.adoc @@ -95,6 +95,8 @@ For documents that are smaller, XDCR replicates the document to the target witho Note that a low setting risks increased latency, due to a higher number or metadata fetches; but may also reduce the number of required replications (due to source and target having identical copies of the document). A high setting lowers latency during replication, since the number of metadata fetches is reduced; but may also raise the replication-rate excessively, overwhelming either network or target cluster. +XDCR optimistic replication is applicable only when the `enableCrossClusterVersioning` property is disabled. For information about the property `enableCrossClusterVersioning`, see xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc#version-pruning-window-hrs[XDCR enableCrossClusterVersioning]. + | XDCR Statistics Collection Interval | Specifies, in seconds, how frequently XDCR Statistics are updated.