refactor(x/sync): remove need for passing proof nodes to completeWorkItem #4064

alarso16 · 2025-07-09T14:13:24Z

Why this should be merged

As a part of the refactor to allow Firewood to use x/sync, we should remove any proof logic from sync manager. This moves finding the next key to request to the proof commit implementation, as well as parsing the proof from bytes.

How this works

The completeWorkItem function does not need to know the largest handled key, but rather the next key that needs retrieved. This is handled through a hook, since even on some non-error returns, it shouldn't be called (see implementation). Instead of passing in the DB to the Sync Manager, it will take this ProofClient

How this was tested

Unit tests.

Need to be documented in RELEASES.md?

No.

Copilot

Pull Request Overview

This PR refactors the completeWorkItem function in the x/sync package to simplify work item completion by removing the need to pass proof nodes and calculating the next key to fetch within the proof response handlers instead.

Moves next key calculation logic from completeWorkItem to handleRangeProofResponse and handleChangeProofResponse
Simplifies completeWorkItem function signature by removing ctx and proofOfLargestKey parameters
Adds test coverage for the findNextKey function with identical keys edge case

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
x/sync/manager.go	Refactors proof response handlers to calculate next key before calling `completeWorkItem` and simplifies the `completeWorkItem` function signature
x/sync/sync_test.go	Adds new test case for `findNextKey` function when start and end keys are identical

x/sync/manager.go

x/sync/sync_test.go

alarso16 · 2025-07-15T18:41:07Z

x/sync/merkledb_client.go

+//
+// Invariant: [lastReceivedKey] < [rangeEnd].
+// If [rangeEnd] is Nothing it's considered > [lastReceivedKey].
+func (c *xSyncClient) findNextKey(


Everything this function and below was completely unmodified.

alarso16 · 2025-07-15T18:41:58Z

x/sync/proof_test.go

+	"github.com/stretchr/testify/require"
+)
+
+func Test_Sync_FindNextKey_InSync(t *testing.T) {


These tests are unmodified besides initialization

These comments significantly reduce review time, ty for doing that!

I would avoid unnecessary moves if possible in this PR, it makes it difficult to review - it's okay (and common) in Go to just stuff things in one file.

alarso16 · 2025-07-15T18:42:31Z

x/sync/db.go

@@ -12,3 +19,18 @@ type DB interface {
 	merkledb.ChangeProofer
 	merkledb.RangeProofer
 }
+
+type ProofClient interface {


Any ideas for a better name?

SyncClient?

What do we define this interface for?

The x/sync manager should be able to work with other database implementations besides MerkleDB (specifically, Firewood). This is the smallest reasonable interface to allow other databases to hook in.

How about DBSyncClient?

rkuris

This looks fine to me, not sure why we are removing BranchFactor16. We support this in firewood as well, although it is not a runtime option, it's compile time. We could check and refuse to open if the feature isn't enabled in a future release. For now, I think we can just implement it without BranchFactor16 working.

rkuris · 2025-07-15T20:51:26Z

x/sync/db.go

@@ -12,3 +19,18 @@ type DB interface {
 	merkledb.ChangeProofer
 	merkledb.RangeProofer
 }
+
+type ProofClient interface {


SyncClient?

x/sync/db.go

x/sync/manager.go

rkuris · 2025-07-15T20:57:19Z

x/sync/proof_test.go

+	"github.com/stretchr/testify/require"
+)
+
+func Test_Sync_FindNextKey_InSync(t *testing.T) {


These comments significantly reduce review time, ty for doing that!

joshua-kim · 2025-07-16T17:22:12Z

x/sync/db.go

@@ -12,3 +19,18 @@ type DB interface {
 	merkledb.ChangeProofer
 	merkledb.RangeProofer
 }
+
+type ProofClient interface {


What do we define this interface for?

joshua-kim · 2025-07-16T17:24:22Z

x/sync/merkledb_client.go

+	BranchFactor merkledb.BranchFactor
+}
+
+type xSyncClient struct {


This abstraction feels very difficult to reason about and there doesn't seem to be a clear abstraction boundary defined - what is this type intended to represent and what is it's responsibility? It's not really a client - it doesn't make any requests, but seems to define handlers for how to interpret responses + you can interact with the underlying database it's coupled to.

It isn't really a client, but I didn't know what to call it. The goal was to remove the dependence on database and proof implementation (specifically merkledb) from the sync manager. The x/sync manager in particular shouldn't depend on any specific database/proof implementation

x/sync/db.go

github-project-automation bot added this to avalanchego Jul 9, 2025

alarso16 added merkledb go Pull requests that update Go code labels Jul 9, 2025

alarso16 force-pushed the alarso16/move-next-key branch 2 times, most recently from 6936bbb to 4a3dae0 Compare July 9, 2025 16:41

alarso16 marked this pull request as ready for review July 9, 2025 16:42

Copilot AI review requested due to automatic review settings July 9, 2025 16:42

alarso16 requested review from joshua-kim and rrazvan1 as code owners July 9, 2025 16:42

Copilot AI reviewed Jul 9, 2025

View reviewed changes

x/sync/manager.go Outdated Show resolved Hide resolved

x/sync/manager.go Outdated Show resolved Hide resolved

joshua-kim reviewed Jul 9, 2025

View reviewed changes

x/sync/sync_test.go Outdated Show resolved Hide resolved

alarso16 requested a review from StephenButtolph as a code owner July 11, 2025 15:08

alarso16 force-pushed the alarso16/move-next-key branch 9 times, most recently from ab1bd24 to c8e2fd9 Compare July 15, 2025 18:52

alarso16 commented Jul 15, 2025

View reviewed changes

rkuris approved these changes Jul 15, 2025

View reviewed changes

joshua-kim reviewed Jul 16, 2025

View reviewed changes

alarso16 force-pushed the alarso16/move-next-key branch 2 times, most recently from 7c2a3af to 7b6273f Compare July 17, 2025 14:35

alarso16 added 3 commits July 17, 2025 10:46

refactor: decouple work enqueuing from proofs

98513a9

refactor: abstract client-side proof logic

7971241

refactor: remove callbacks

389b0bf

alarso16 force-pushed the alarso16/move-next-key branch from 7b6273f to 389b0bf Compare July 17, 2025 14:46

refactor: naming

2fb7fbd

alarso16 force-pushed the alarso16/move-next-key branch from 5cb257f to 2fb7fbd Compare July 17, 2025 15:16

joshua-kim reviewed Jul 17, 2025

View reviewed changes

x/sync/db.go Show resolved Hide resolved

alarso16 force-pushed the alarso16/move-next-key branch 2 times, most recently from b603332 to 333553e Compare July 17, 2025 16:14

test: ensure db errors work

412c512

alarso16 force-pushed the alarso16/move-next-key branch from 333553e to 412c512 Compare July 17, 2025 16:23

alarso16 mentioned this pull request Jul 18, 2025

chore: skeleton of ffi proofs ava-labs/firewood#1092

Draft

refactor(x/sync): remove need for passing proof nodes to completeWorkItem #4064

Are you sure you want to change the base?

refactor(x/sync): remove need for passing proof nodes to completeWorkItem #4064

Conversation

alarso16 commented Jul 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why this should be merged

How this works

How this was tested

Need to be documented in RELEASES.md?

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joshua-kim Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rkuris left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joshua-kim Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joshua-kim Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

alarso16 commented Jul 9, 2025 •

edited

Loading

joshua-kim Jul 16, 2025 •

edited

Loading

joshua-kim Jul 16, 2025 •

edited

Loading

joshua-kim Jul 16, 2025 •

edited

Loading