Remove orphaned blobs on started after incomplete block finalization #2988
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #2988 +/- ##
==========================================
+ Coverage 63.10% 63.26% +0.15%
==========================================
Files 353 353
Lines 17049 17085 +36
==========================================
+ Hits 10759 10809 +50
+ Misses 5447 5424 -23
- Partials 843 852 +9
🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR addresses the issue of orphaned blob sidecars that can remain on disk when block finalization fails after blob sidecars have been persisted. The solution involves detecting and cleaning up these orphaned blobs during node startup.
Key Changes:
- Implements
PruneOrphanedBlobsmethod in blockchain service to detect and remove orphaned blob sidecars atlastBlockHeight + 1 - Adds
DeleteByIndexmethod to the storage layer that removes entries without modifying the lower bound index (unlikePrune) - Integrates orphaned blob cleanup into CometBFT service initialization to run automatically on node startup
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
beacon/blockchain/service.go |
Implements PruneOrphanedBlobs method with orphaned blob detection and cleanup logic |
beacon/blockchain/interfaces.go |
Adds PruneOrphanedBlobs method to BlockchainI interface |
consensus/cometbft/service/service.go |
Calls PruneOrphanedBlobs during service initialization after loading the last block height |
da/store/store.go |
Implements DeleteBlobSidecars method that delegates to IndexDB.DeleteByIndex |
da/store/interfaces.go |
Adds DeleteByIndex method to IndexDB interface |
storage/filedb/range_db.go |
Implements DeleteByIndex method that removes entries at a specific index without updating lowerBoundIndex |
storage/filedb/range_db_test.go |
Adds comprehensive test verifying DeleteByIndex doesn't affect the lower bound index invariant |
testing/simulated/testnode.go |
Exposes Blockchain service in TestNode for test access |
testing/simulated/orphaned_blobs_test.go |
Adds integration test simulating orphaned blob creation and cleanup on node restart |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes potential orphaned blob sidecars on disk from incomplete block finalization.
If
FinalizeBlockfails afterFinalizeSidecarshas persisted blob sidecars to disk, we may leave behind orphaned blobs at slotlastBlockHeight + 1. These orphaned blobs:This PR addresses this so that on node startup (during CometBFT service initialization), check for and remove any orphaned blob sidecars at slot
lastBlockHeight + 1.