Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,7 @@ include::third-party:partial$nav.adoc[]
** xref:manage:manage-xdcr/replicate-using-scopes-and-collections.adoc[Replicate Using Scopes and Collections]
** xref:manage:manage-xdcr/filter-xdcr-replication.adoc[Filter a Replication]
** xref:manage:manage-xdcr/recover-data-with-xdcr.adoc[Recover Data with XDCR]
** xref:manage:manage-xdcr/xdcr-differ.adoc[xdcrDiffer Utility]
* xref:manage:manage-security/security-management-overview.adoc[Manage Security]
** xref:manage:manage-security/manage-security-settings.adoc[Manage Security Settings]
** xref:manage:manage-security/manage-users-and-roles.adoc[Manage Users, Groups, and Roles]
Expand Down
11 changes: 9 additions & 2 deletions modules/introduction/partials/new-features-80.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -117,8 +117,15 @@ Additional information sent by clients at connection time can be found in the lo

[#section-new-feature-800-XDCR]
=== XDCR
https://jira.issues.couchbase.com/browse/MB-61048[MB-61048]::
Once faulty remote cluster credentials are fixed, XDCR will now be able to more quickly restart replications that depend on the repaired references.

https://jira.issues.couchbase.com/browse/MB-62564[MB-62564]::

`xdcrDiffer` is a diagnostic utility included in the Couchbase Server package.
This utility verifies data consistency between XDCR clusters.
It identifies missing or mismatched documents by comparing metadata and content hashes across source and target clusters.
+
For more information, see xref:manage:pages/manage-xdcr/xdcr-differ.adoc[xdcrDiffer Utility].


[#section-new-feature-800-query-service]
=== Query Service
Expand Down
68 changes: 68 additions & 0 deletions modules/manage/pages/manage-xdcr/xdcr-differ.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
= xdcrDiffer Utility
:description: pass:q[The `xdcrDiffer` utility helps you verify data consistency between XDCR clusters by comparing document metadata and values, and reporting missing or mismatched documents.]

[abstract]
{description}

`xdcrDiffer` is a diagnostic utility included in the Server installation package for convenient access.
Previously, you'd build this utility from source using the xdcrDiffer GitHub repository.

== What is xdcrDiffer?

`xdcrDiffer` is a Diff tool that helps you check data consistency between XDCR clusters.
For full usage instructions and Q&A, see the https://github.com/couchbase/xdcrDiffer?tab=readme-ov-file#readme[xdcrDiffer ReadMe].

The output report from `xdcrDiffer` provides information such as:

* A list of document IDs missing from the target cluster.

* A list of documents with mismatched data.

== Utility Files

The utility includes these files:

* `xdcrDiffer` (binary)

* `runDiffer.sh` (shell script)

To run the utility, use the `runDiffer.sh` script.

TIP: Run the xdcrDiffer utility on a cluster node that does not serve Data service, as the utility can consume significant resources.
You can also run it on an arbiter node, which does not run any services.

== How the xdcrDiffer Utility Works

`xdcrDiffer` acts as a DCP consumer and creates DCP streams from the Data nodes.
Then it reads data from both the source and target clusters, and stores results in the directory you specify as `outputDir`.

**Before you begin:**

Estimate the required storage for utility output.
Make sure the specified `outputDir` has enough space for collecting data.

Each mutation from DCP is stored with its metadata.
Each mutation typically uses about 102 bytes and the key size.
Calculate the storage needed as follows:

----
Storage needed = (102 bytes + key size) × number of documents in the source and target buckets being replicated
----

NOTE: Ensure you check the details of usage, limitations, and information about how to read the output reports in https://github.com/couchbase/xdcrDiffer?tab=readme-ov-file#readme[xdcrDiffer ReadMe].

The following is an example of running `xdcrDiffer.sh` with an option to compare both metadata and body during the report generation portion of the utility run.

----
runDiffer.sh --username=Administrator --password=password --hostname=localhost:8091 --sourceBucket=travel-sample --targetBucket=travel-sample --remoteClusterName=cluster2 --compareType=both --outputDir=/has_enough_space/xdcr_differ_outputdir
----

== Comparison Process

* `xdcrDiffer` collects each document's metadata and generates a SHA-512 digest of its value.

* After capturing all data from both clusters, `xdcrDiffer` compares documents by using either document ID or key.

** If a document is missing in either cluster, the utility reports the missing document.

** If all documents are present, the utility compares metadata and the body hash to verify if they match.