[DatastreamToSpanner] Integration test for CDC processing algorithm for Live migration template #3018

darshan-sj · 2025-11-24T08:21:15Z

No description provided.

codecov · 2025-11-24T08:29:54Z

Codecov Report

❌ Patch coverage is 81.96721% with 33 lines in your changes missing coverage. Please review.
✅ Project coverage is 53.88%. Comparing base (793fa0f) to head (df7bcb4).
⚠️ Report is 13 commits behind head on main.

Files with missing lines	Patch %	Lines
...panner/testutils/failureinjectiontesting/User.java	74.46%	12 Missing and 12 partials ⚠️
...failureinjectiontesting/FuzzyCDCLoadGenerator.java	89.88%	5 Missing and 4 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #3018      +/-   ##
============================================
+ Coverage     50.25%   53.88%   +3.62%     
+ Complexity     5020     2137    -2883     
============================================
  Files           967      484     -483     
  Lines         59261    27905   -31356     
  Branches       6458     2933    -3525     
============================================
- Hits          29783    15037   -14746     
+ Misses        27374    11939   -15435     
+ Partials       2104      929    -1175

Components	Coverage Δ
spanner-templates	`71.45% <81.96%> (+1.00%)`	⬆️
spanner-import-export	`∅ <ø> (∅)`
spanner-live-forward-migration	`79.77% <81.96%> (+0.07%)`	⬆️
spanner-live-reverse-replication	`77.19% <81.96%> (+0.11%)`	⬆️
spanner-bulk-migration	`88.19% <81.96%> (-0.14%)`	⬇️

Files with missing lines	Coverage Δ
...failureinjectiontesting/FuzzyCDCLoadGenerator.java	`89.88% <89.88%> (ø)`
...panner/testutils/failureinjectiontesting/User.java	`74.46% <74.46%> (ø)`

... and 507 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…tion template

...com/google/cloud/teleport/v2/templates/failureinjectiontesting/DataStreamToSpannerCDCFT.java

...oogle/cloud/teleport/v2/spanner/testutils/failureinjectiontesting/FuzzyCDCLoadGenerator.java

...ogle/cloud/teleport/v2/spanner/testutils/failureinjectiontesting/CDCCorrectnessTestUtil.java

...com/google/cloud/teleport/v2/templates/failureinjectiontesting/DataStreamToSpannerCDCFT.java

...ogle/cloud/teleport/v2/spanner/testutils/failureinjectiontesting/CDCCorrectnessTestUtil.java

rohitwali · 2025-11-26T08:55:47Z

...oogle/cloud/teleport/v2/spanner/testutils/failureinjectiontesting/FuzzyCDCLoadGenerator.java

+    ExecutorService executor = Executors.newFixedThreadPool(totalRows);
+    try {
+      List<Future<?>> futures = new ArrayList<>();
+
+      for (int i = 0; i < totalRows; i++) {
+        futures.add(
+            executor.submit(
+                () -> {
+                  // Each thread gets its own connection to ensure thread safety
+                  try (Connection conn = DriverManager.getConnection(jdbcUrl, username, password)) {


would it make sense to use a connection pooling library instead?

A connection pool library is useful when the connection utilization is very dynamic. In our case, we get 100 (num of rows) connections at the beginning and they remain in use till the end of load generation. I used the HikariCP and checked, it is actually taking much more time and degrading the performance.

so our template code will perform worse than the test? is that worth poking?

rohitwali · 2025-11-26T08:57:37Z

...oogle/cloud/teleport/v2/spanner/testutils/failureinjectiontesting/FuzzyCDCLoadGenerator.java

+    int id;
+    User user;
+    do {
+      id = getRandom().nextInt(1000000);


what is 1 million here?

generating a random Id between [0, 1000000) .

i guess i followed that part. what I am trying to understand is
1/ why not 1 billion? how did we pick a million?
2/ could this be range be different for different tests?

rohitwali · 2025-11-26T09:03:02Z

...ogle/cloud/teleport/v2/spanner/testutils/failureinjectiontesting/CDCCorrectnessTestUtil.java

+      // If row is present, then update the row with 75% probability and delete the row with 25%
+      // probability
+      if (isRowPresent) {
+        if (getRandom().nextDouble() < 0.75) {


1/ didnt we want to do insert/update/deletes with equal probability?
2/ given its a util, shouldnt this be a configuration that is passed to it?

If we just do Insert or update or delete randomly, Many of the inserts would be no op if the row already exists and many of the deletes would be no op if the row is already deleted, but it would still consume CPU cycles, network bandwidth etc. It doesn't make sense to have such load. insert only when the row doesn't exist, delete or update only when the row exists.

This whole class is written very specifically for the CDC algorithm correctness IT. I don't see it to be used for other test cases and hence chose to not make it generic and config driven. For example, this class doesn't support multiple tables, ordering of events between the tables etc. Making it generic is over engineering and doesn't make sense to me. If there is a use case in future where we want to re-use this class, then we can enhance it at that time.

it makes sense to handle insert in insert and delete on delete, which IIUC we are already handling. but still unclear to me why that would mean equal probably can be used. is there a doc that walks through how the current distribution is picked?

I was assuming this class to provide load generation capabilities for CDC tests. I am ok with just having support for one table, but even for that it makes sense to support different workload distributions (Open/Closed Principle). are there any other aspects of this class that are CDC correctness specific that I may be missing?

...ogle/cloud/teleport/v2/spanner/testutils/failureinjectiontesting/CDCCorrectnessTestUtil.java

rohitwali · 2025-11-28T10:09:43Z

...com/google/cloud/teleport/v2/templates/failureinjectiontesting/DataStreamToSpannerCDCFT.java


    // generate Load
-    CDCCorrectnessTestUtil testUtil = new CDCCorrectnessTestUtil();
+    testUtil = new FuzzyCDCLoadGenerator();


nit: may want to use a better name

rohitwali · 2025-11-28T10:14:12Z

...d/teleport/v2/spanner/testutils/failureinjectiontesting/FuzzyCDCCorrectnessTestUtilTest.java

+
+/** Unit tests for {@link FuzzyCDCLoadGenerator}. */
+@RunWith(JUnit4.class)
+public class FuzzyCDCCorrectnessTestUtilTest {


Should this reflect the class under Test?

rohitwali · 2025-11-28T10:15:27Z

...oogle/cloud/teleport/v2/spanner/testutils/failureinjectiontesting/FuzzyCDCLoadGenerator.java

+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class FuzzyCDCLoadGenerator {


should this be under the main package or test package?

rohitwali · 2025-11-28T10:17:58Z

...oogle/cloud/teleport/v2/spanner/testutils/failureinjectiontesting/FuzzyCDCLoadGenerator.java

+          user.mutateRandomly();
+          user.insert(conn);
+          isRowPresent = true;
+          counter.incrementAndGet();


where is this used?

pull-request-size bot added the size/XL label Nov 24, 2025

Implementing CDC processing algorithm correctness test for Live migra…

b22f585

…tion template

darshan-sj force-pushed the live-cdc-correctness-test branch from 4771b32 to b22f585 Compare November 25, 2025 09:11

darshan-sj added the improvement label Nov 25, 2025

Adding more buffer time

a931229

pull-request-size bot added size/XXL and removed size/XL labels Nov 25, 2025

darshan-sj marked this pull request as ready for review November 25, 2025 09:47

darshan-sj requested a review from a team as a code owner November 25, 2025 09:47

darshan-sj requested review from pratickchokhani and rohitwali November 25, 2025 09:47

bharadwaj-aditya reviewed Nov 26, 2025

View reviewed changes

Addressing comments

7b84956

pull-request-size bot added size/XL and removed size/XXL labels Nov 26, 2025

rohitwali reviewed Nov 26, 2025

View reviewed changes

addressing review comments

df7bcb4

pull-request-size bot added size/XXL and removed size/XL labels Nov 28, 2025

rohitwali reviewed Nov 28, 2025

View reviewed changes

[DatastreamToSpanner] Integration test for CDC processing algorithm for Live migration template #3018

Are you sure you want to change the base?

[DatastreamToSpanner] Integration test for CDC processing algorithm for Live migration template #3018

Uh oh!

Conversation

darshan-sj commented Nov 24, 2025

Uh oh!

codecov bot commented Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov bot commented Nov 24, 2025 •

edited

Loading