-
Notifications
You must be signed in to change notification settings - Fork 29
LedgerDB: implement predictable snapshotting #1575
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: amesgen/ledgerdb-garbage-collect-states
Are you sure you want to change the base?
LedgerDB: implement predictable snapshotting #1575
Conversation
I integrated this into 10.5 for testing:
|
0e21795
to
4d5bb72
Compare
-- We only write snapshots for ledger states that are /immutable/. Concretely, | ||
-- for every slot @s@ out of | ||
-- | ||
-- > sfaOffset, sfaOffset + sfaInterval, sfaOffset + 2 * sfaInterval, sfaOffset + 3 * sfaInterval |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we provide an example relating this with the intended use on an epoch? Using some real values.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added the same example as in the PR description 👍
} | ||
forM_ snapshotSlots $ \slot -> do | ||
-- Prune the 'DbChangelog' such that the resulting anchor state has slot | ||
-- number @slot@. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
-- number @slot@. | |
-- number @slot@ or younger. |
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Each s
in snapshotSlots
is the slot of a ledger state in DbChangelog
(see the contract of onDiskSnapshotSelector
), so it is true as written. (Otherwise, we wouldn't take a snapshot for the requested slot, so snapshots wouldn't be predictable.)
Superseded by the rework of the snapshot policy for predictable snapshots, with dedicated new tests
It is no longer needed by the predictable snapshotting logic.
a8fa7e2
to
b503dc3
Compare
4d5bb72
to
6f063b3
Compare
Closes #1424
Based on top of #1513, see there for the relation to this change.
Note that we need #1573 before this can be released in a node.
Currently, this PR replaces the previous logic for when to create snapshots (It would be possible to preserve it, but I don't see a big motivation). Concretely,
SnapshotFrequencyArgs
containssfaInterval :: SlotNo
: Create snapshots everysfaInterval
many slots.Default:
2*k = 4320
slots, so 72 min on mainnet as before.sfaOffset :: SlotNo
: Allows to determine the offset of where snapshots are taken, see below.Default:
0
sfaRateLimit :: DiffTime
: A minimum duration between snapshots (used to avoid excessive snapshots while syncing).Default: 10 minutes (previous value was 6 minutes, which seemed a bit low, so I increased it somewhat)
Concretely, the node will try to create snapshots for the last immutable blocks before the slots
but can skip creating some of these depending on the
sfaRateLimit
(which can be disabled by setting it to a non-positive value). Also see the Haddocks.For example, setting
sfaInterval = 10*2160*20
(one mainnet epoch) andsfaOffset = 0
will cause the node to create snapshots for the last block in every Shelley epoch. By settingsfaOffset
to eg5*2160*20
(half an epoch), snapshots are created just before the midway point in each epoch.There is some code that could be shared between V1 and V2 (already even before this PR), but given the upcoming removal of V1, this seems acceptable for now.
TODO: Tests