Repository navigation
Fix flaky CI: compaction filter race and iterator use-after-free - #13
Merged
Merged
Conversation
filter_key_contains_test flaked on the linux-shared-libs CI job: a _session_ key survived the forced compaction. The rule tests relied on compact_range(undefined, undefined, force), which can trivial-move L0 files to the bottommost level without rewriting them, so the compaction filter never runs and matching keys are not deleted. Background auto-compactions usually filtered the data first, making the outcome timing-dependent. Give compact_range explicit key bounds (and a short settle after flush) in the 8 deletion/transform-asserting tests, matching the pattern already used by the value_empty, always_delete, and column_family tests. A bounded range forces a real merge that runs the filter. The keep-asserting handler tests are unaffected and left as-is.
benoitc
changed the title
transaction:cf_iterators_test segfaulted intermittently on the macOS CI runner. Root cause: a column-family iterator stored only a reference to the DbObject, not to the ColumnFamilyObject. Once the erlang column family handle went out of scope, the BEAM garbage collector could free the ColumnFamilyObject, whose destructor calls DestroyColumnFamilyHandle, while an open iterator still referenced that column family. The next iterator_move then dereferenced freed memory inside DBIter::Seek. The same applies to a transaction iterator and its transaction, whose write batch backs the iterator's delta. Pin the column family (and, for transaction iterators, the transaction) with enif_keep_resource for the iterator's lifetime, releasing them in ~ItrObject after the iterator is deleted. enif_keep_resource is used rather than the ReferencePtr/close-request path, which would block a scheduler in AwaitCloseAndDestructor when the handle is collected by GC. Reproduced under OTP 27 on macOS (forcing GC with the handle dropped crashes immediately); with the fix, 10000 iterations run clean.
benoitc
force-pushed
the
harden-compaction-filter-tests
branch
from
June 14, 2026 13:53
ff87632 to
d74f88c
Compare
benoitc
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two flaky/broken-CI fixes surfaced by the PR #12 merge.
1. Compaction filter tests (linux-shared-libs)
filter_key_contains_testflaked: a_session_key survived a forced compaction. The rule tests usedcompact_range(undefined, undefined, force), which can trivial-move L0 files to the bottommost level without rewriting them, so the compaction filter never runs and matching keys aren't deleted. Deletions actually came from background auto-compactions, making the outcome timing-dependent.Fix: give
compact_rangeexplicit key bounds (plus a short settle after flush) in the 8 deletion/transform-asserting tests, matching the pattern already used byvalue_empty,always_delete, andcolumn_family. A bounded range forces a real merge that runs the filter.2. Use-after-free in column-family and transaction iterators (macOS segfault)
transaction:cf_iterators_testsegfaulted intermittently on macOS. A column-family iterator stored only a reference to theDbObject, not theColumnFamilyObject. Once the erlang CF handle went out of scope, the BEAM GC could free theColumnFamilyObject(whose destructor callsDestroyColumnFamilyHandle) while an open iterator still referenced that CF; the nextiterator_movedereferenced freed memory inDBIter::Seek. The same applies to a transaction iterator and its transaction (whose write batch backs the iterator's delta).Fix: pin the column family (and, for transaction iterators, the transaction) with
enif_keep_resourcefor the iterator's lifetime, released in~ItrObjectafter the iterator is deleted.Reproduced under OTP 27 on macOS (forcing GC with the handle dropped crashes immediately in
DBIter::Seek); with the fix, 10000 iterations run clean. The test is no longer skipped on macOS.