Skip to content

DOC-12484 XDCR Conflict Logging feature #3806

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 25 commits into
base: release/8.0
Choose a base branch
from

Conversation

rao-shwe
Copy link
Contributor

@rao-shwe rao-shwe commented May 9, 2025

DOC-12484

Link to the preview doc: https://preview.docs-test.couchbase.com/DOC-12484/server/current/learn/clusters-and-availability/xdcr-conflict-logging-feature.html

Preview pages:

DON'T review the following files: The following are 7.6.6 release docs which were missing in the release/8.0 branch. So I added them again.

  • xdcr-active-active-sgw.adoc
  • xdcr-conflict-resolution.adoc
  • xdcr-enable-crossclusterversioning.adoc
  • The section [Create an XDCR Replication with mobile=Active](Create an XDCR Replication with mobile=Active).
  • Creating and Editing Buckets.

Copy link

@sumukhbhat2701 sumukhbhat2701 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some review feedback which hold true for all pages:

  1. For this feature specifically, we need to use the term "true conflicts" more than just mentioning "conflicts". That means we need to first define what a true conflict is and set the expectation.
  2. There should be a warning that this feature is best effort (and that true conflicts is assumed to be very low). Everything that's in this slide - https://couchbase.slack.com/archives/C0963TSUU0N/p1752763776316649.
  3. The setting is quite complex to understand just from textual description. An example will do a lot of help to someone new reading this.
  4. There should be a mention that on every true conflict detected, XDCR will log 3 documents to the conflict collection - CRD (Conflict record document - contains metadata of detected true conflict), source document in conflict & target document in conflict. It should be mentioned that the CRD will contain the document IDs of source and target documents logged. Maybe an example of source and target document IDs in CRD.
  5. Continuation of (3), I think there should be some examples on how to make use of the detected and logged conflicts. Eg: Use SDK, N1QL, range scan, eventing etc.
  6. There should be a mention that the logged documents will not be replicated by XDCR if conflict collection is a source collection of any XDCR.

@sumukhbhat2701
Copy link

I think I missed one of the pages from reviewing, so if somethings are already done from last comment, please ignore.

Copy link
Contributor Author

@rao-shwe rao-shwe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sumukhbhat2701

I've implemented most of your review inputs and closed the comments.

@rao-shwe
Copy link
Contributor Author

rao-shwe commented Jul 21, 2025

@sumukhbhat2701

Points 1 and 2 are fixed.
Point 3: Already has examples and descriptions. Not okay to repeat the same content in multiple locations. So I've added a link to examples wherever necessary.
Point 4, 5, and 6: Already exists.

Edit: All fixed in draft-2.

Some review feedback which hold true for all pages:

  1. For this feature specifically, we need to use the term "true conflicts" more than just mentioning "conflicts". That means we need to first define what a true conflict is and set the expectation.
  2. There should be a warning that this feature is best effort (and that true conflicts is assumed to be very low). Everything that's in this slide - https://couchbase.slack.com/archives/C0963TSUU0N/p1752763776316649.
  3. The setting is quite complex to understand just from textual description. An example will do a lot of help to someone new reading this.
  4. There should be a mention that on every true conflict detected, XDCR will log 3 documents to the conflict collection - CRD (Conflict record document - contains metadata of detected true conflict), source document in conflict & target document in conflict. It should be mentioned that the CRD will contain the document IDs of source and target documents logged. Maybe an example of source and target document IDs in CRD.
  5. Continuation of (3), I think there should be some examples on how to make use of the detected and logged conflicts. Eg: Use SDK, N1QL, range scan, eventing etc. @hyunjuV I think you had a document prepared for this, was that for public docs?
  6. There should be a mention that the logged documents will not be replicated by XDCR if conflict collection is a source collection of any XDCR.

@rao-shwe rao-shwe marked this pull request as ready for review July 30, 2025 11:12
If you try to use the feature _XDCR Active-Active with Sync Gateway_ when you have more than 10 user xattrs in your document, the XDCR replication **silently skips** replicating that document.
As a result, the data in the replication-skipped document will not be consistent between the target and source clusters.
The only way you will know this skip occured is because the Prometheus stat `subdoc_cmd_docs_skipped` will be incremented and the document will _not_ be consistent between the target and source.
* If you use Eventing service functions that update documents in XDCR-replicated buckets (Eventing source bucket mutations), ensure your functions do not cause continuous replication loops.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This content is approved as a part of the PR #3844.

* Conflict logging is asynchronous and done on a best-effort basis, meaning, not all conflicts may be logged.

* Conflicts are expected to be rare during XDCR replication.
If there is an upsurge of conflicts, so that logging conflicts may cause replication issues, XDCR stops logging conflicts temporarily to prioritize replication.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of "XDCR stops logging conflicts temporarily", I think that we should say "XDCR skips logging conflicts temporarily" to make it clear that there may be some conflicts that are not logged (even though you say that in the first bullet, it would be good to make it clear here as well so that there's no misunderstanding).

If there is an upsurge of conflicts, so that logging conflicts may cause replication issues, XDCR skips logging conflicts temporarily to prioritize replication.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants