Skip to content

fix(desktop): rank read-marker eviction by read recency - #5653

Open
TolgaCinisli wants to merge 2 commits into
block:mainfrom
TolgaCinisli:fix/read-state-evict-by-read-recency
Open

fix(desktop): rank read-marker eviction by read recency#5653
TolgaCinisli wants to merge 2 commits into
block:mainfrom
TolgaCinisli:fix/read-state-evict-by-read-recency

Conversation

@TolgaCinisli

@TolgaCinisli TolgaCinisli commented Aug 12, 2026

Copy link
Copy Markdown

Summary

Marking an older message read is undone by the very write that records it.

Read markers for msg:/thread: contexts are evicted at two points, and both rank by the
marker's value — which is the timestamp of the message that was read, not of the read:

  • pruneStaleContexts (readStateStorage.ts) drops any prunable marker whose value is older
    than the 7-day horizon, then caps the survivors at LOCAL_MAX_PRUNABLE_CONTEXTS (1 000),
    keeping the highest values.
  • trimContextsToBudget (readStateManager.ts) evicts the lowest values first to fit the
    32 KB publish budget.

So when a user marks an older message read, the new marker carries that message's old
timestamp, sorts to the bottom of both rankings, and is discarded before it reaches disk or the
relay. The row reads correctly for the rest of the session (in-memory effectiveState holds it)
and is unread again after the next reload — permanently, no matter how many times it is clicked.

The horizon alone reproduces this at any volume: no message older than 7 days can ever be
durably marked read.
The cap makes it bite much sooner on busy accounts.

Evidence from a live account

localStorage["buzz.channel-read-state.v2:<pubkey>"]:

1024 contexts = 24 channel + 984 msg + 16 thread
prunable tier = 1000 / 1000   ← LOCAL_MAX_PRUNABLE_CONTEXTS, saturated
oldest surviving msg marker = 3 days old; nothing older can be written

Relay side, two kind:30078 slots for the same pubkey:

read-state:adec4aea…  43780 B ciphertext
read-state:9db0bc01…  43780 B ciphertext

43 780 B of base64 NIP-44 = 1 + 32 + (2 + 32768) + 32 bytes, i.e. the padded-plaintext bucket
at exactly READ_STATE_MAX_PLAINTEXT_BYTES. Those 1 024 contexts serialize to roughly 84 KB, so
trimContextsToBudget is dropping about 60 % of the markers on every publish — oldest message
first.

Why the Inbox row cannot recover

resolveInboxItemReadAt resolves a thread row through msg:<id> alone, so the channel marker —
which is never pruned, and was current the whole time — does not cover it. That is deliberate
(NIP-RS makes the thread fold a MAY, and the per-message predicate is what keeps reading an
ancestor from covering a descendant), so this PR does not touch it. It is what turns the
eviction bug from cosmetic into permanent, which is why it is worth stating.

Fix

Rank eviction by when the read happened, using the signal the manager already persists:

  1. applyRemoteContextTimestamp refreshes contextSourceCreatedAt only on an actual
    advance
    . Today every republished blob bumps every context it mentions, so the field
    collapses to "time of last publish" — on the account above all 1 024 entries land inside a
    2-day window, including channel markers first read weeks earlier. Item 4 closes the same
    hole on the publish side.
  2. pruneStaleContexts applies both the horizon and the cap to that recency
    (readActionRecency, falling back to the marker value for contexts seeded before the
    signal existed).
  3. trimContextsToBudget evicts least-recently-read first instead of oldest-message first.
  4. publishOneSlot no longer stamps recency at all. It used to stamp every key whose value
    differed from lastPublishedContexts, which is not the narrow filter it looks like:
    initialize seeds lastPublishedContexts from the union of the relay blobs while local
    storage restores the full context set, so every key trimContextsToBudget dropped reads as
    changed on the first publish after each launch and is restamped with a single createdAt.
    Recency is now written only where a read happens — markContextRead, and the advance branch
    of applyRemoteContextTimestamp, which is also how blob-seeded contexts arrive.

Item 4 comes from @rmichelena's independent reproduction below. On his account 330 of 330
prunable contexts carried a recency value but only three distinct ones, all minutes apart —
publish-shaped, not read-shaped. Left in place, that distribution makes the horizon check evict
nothing and turns cap comparisons into ties, so eviction order would fall back to sort stability
rather than read recency. It errs toward keeping markers, so it does not reintroduce the bug,
but the ranking this PR argues for only holds once publishing stops overwriting the signal.

No storage format change, no new key, no wire change. Contexts carrying no recency behave
exactly as before — and in practice there are fewer of those than the first draft of this
description assumed: main already records the signal in markContextRead, so existing
installs get the corrected ranking on upgrade rather than only for reads written afterwards.

trimContextsToBudget and splitContextsIntoBudgetedSlots move verbatim into a new
readStateBudget.ts. That is not gratuitous: readStateManager.ts sat at 999 of the 1 000-line
ceiling enforced by scripts/check-file-sizes.mjs, so the fix could not land in it at all. Both
are pure functions that were already exported solely for unit testing, and the eviction policy
now lives beside the recency helper it ranks by. Manager drops to 846 lines; the new module is
189. No behavioural change in the move itself — happy to reshape the split if you'd rather have
it drawn elsewhere.

Behaviour change worth calling out: the 7-day horizon now measures time since the read
rather than the age of the message read. Same retention window, correct anchor — a marker you
created today survives a week; a marker you have not touched in a week ages out, as intended.

Desktop only. mobile/lib/shared/read_state/ has no equivalent tiered eviction (searched for
evict, prune, horizon, byte-budget constants under mobile/lib), so there is no parity
change to make.

Related issue

None found — searched block/buzz issues and PRs for resolveInboxItemReadAt,
useHomeInboxReadState, trimContextsToBudget, read marker eviction, mark as read reverts,
inbox unread returns. Closest prior art is #1305 and #1502, which introduced these two
eviction points, and #1242, which routed Inbox rows through per-message markers.

Testing

just ci green locally (Hermit toolchain, macOS 26.5, Node 24.15):

check + clippy         clean
Rust workspace tests   all crates pass (buzz-relay 2402 passed / 0 failed, …)
desktop tests          4721 passed / 0 failed / 0 skipped (67 suites)
desktop build          ok        desktop tauri check + test   ok
web build              ok        mobile flutter test          1261 passed

New unit tests, each asserting the old value-ranked behaviour alongside the new one so the
regression is documented rather than merely fixed:

  • pruneStaleContexts keeps a marker just read on an old message
  • pruneStaleContexts cap evicts least recently read, not oldest message
  • writeStoredReadState keeps a marker just read on an old message
  • trimContextsToBudget_evictsLeastRecentlyRead_notOldestMessage
  • applyRemoteContextTimestamp keeps recency stable across repeated republishes
  • publishing does not refresh read recency — publishes a real blob with lastPublishedContexts
    empty and asserts an old read keeps its recency; fails on the parent commit, where the recency
    jumps to the publish time

No screenshots: this changes storage eviction and read-marker bookkeeping, with no visual,
layout or styling delta. The user-visible effect is an Inbox row staying read across a restart,
which a static capture cannot show.

Marking an older message read is undone by the very write that records it.
`pruneStaleContexts` and `trimContextsToBudget` both rank `msg:`/`thread:`
markers by the marker value — the timestamp of the *message* that was read —
so a marker created just now for an older message sorts below the 7-day
horizon or below the 1000-entry cap and is discarded before it reaches disk
or the relay. The Inbox row reads correctly for the rest of the session and
is unread again after the next reload, permanently. The horizon alone
reproduces this at any volume: no message older than 7 days can be durably
marked read.

Rank both eviction points by when the read happened, using the
`contextSourceCreatedAt` signal the manager already persists. That signal
first has to mean what its name says: `applyRemoteContextTimestamp` refreshed
it for every context in every republished blob, even when nothing advanced,
collapsing it to "time of last publish". It now moves only on an actual
advance, matching the publish path, which already bumps changed keys only.

The 7-day horizon now measures time since the read rather than the age of the
message read — same retention window, correct anchor. No storage format or
wire change; contexts with no recency recorded behave exactly as before.

`trimContextsToBudget` and `splitContextsIntoBudgetedSlots` move verbatim into
`readStateBudget.ts`: `readStateManager.ts` sat at 999 of the 1000-line ceiling
enforced by check-file-sizes, so the fix could not land in it. Both are pure
functions already exported only for unit testing, and the eviction policy now
sits beside the recency helper it ranks by.

Signed-off-by: Tolga Cinisli <tolgacinisli@gmail.com>
@TolgaCinisli
TolgaCinisli requested a review from a team as a code owner August 12, 2026 11:59
@rmichelena

Copy link
Copy Markdown

Independent reproduction on a live account — and one data point that closes the most likely objection to this PR.

The cap is not required. The horizon alone reproduces it.

The evidence in the description comes from a saturated account (1 000 / 1 000 prunable contexts), which invites the reading "this only bites heavy users once the cap fills." It does not. Here is localStorage["buzz.channel-read-state.v2:<pubkey>"] on a normal account, desktop 0.5.17:

340 contexts = 19 channel + 318 msg + 3 thread
prunable tier = 321 / 1000        ← cap nowhere near saturated
oldest surviving msg marker    = 6 days old
oldest surviving thread marker = 2 days old
markers below the 7-day cutoff = 0
oldest channel key             = 21 days old   ← channel keys are exempt

Nothing below the cutoff survives, at 32 % of the cap. The LOCAL_MAX_PRUNABLE_CONTEXTS half of the analysis makes it bite sooner, but pruneStaleContexts's horizon check is sufficient on its own — exactly as the description says.

What it looks like from the user side, and why it is permanent rather than merely lossy.

A thread reply ~10 days old kept a DM's sidebar badge at "1" for ten days. Opening the thread cleared the badge for the rest of the session — the in-memory effectiveState holds the marker — and it was back on the next launch, every launch. The user re-read the message several times and even sat in the thread for a while assuming the mark-read had some minimum dwell time.

The reason it never converges is that the horizon is asymmetric: the read markers expire, but the thing that resurrects them does not. The startup catch-up in useUnreadChannels.ts queries the relay with since: readAt + 1 and no lower bound, so it rediscovers the same >7-day-old reply on every launch, forever. So the two sides disagree by construction: a marker older than 7 days is unwritable, while the event it was supposed to cover stays discoverable indefinitely.

The only durable escape today is an explicit channel-level mark-read (sidebar context menu / Esc), because that writes channel.lastMessageAt into a channel key — the one tier pruneStaleContexts exempts. That is also a poor workaround, since it buries the unread message instead of letting the user read it.

Worth noting alongside #6153: that PR deliberately scopes passive channel reads to the top-level timeline and preserves unopened thread activity "until the thread or channel is explicitly marked read". Reading the thread is the intended way out — this eviction bug is what stops that contract from holding for anything older than a week.

Ranking eviction by read recency looks like the right shape to me: the marker's value answers "which message", and the eviction question is "which read". Happy to re-run the same measurement against a patched build if that is useful.

@TolgaCinisli

Copy link
Copy Markdown
Author

Thanks for running this — the normal-account measurement is the data point the description was missing.

Two things your reproduction settles that mine could not:

  1. The cap is a multiplier, not a precondition. My evidence came from a saturated account, which makes the whole thing easy to read as a heavy-user edge case. 321/1000 with nothing below the cutoff surviving shows the horizon check in pruneStaleContexts is sufficient on its own.
  2. The asymmetry is why it never converges. I framed the permanence through resolveInboxItemReadAt; the unbounded since: readAt + 1 catch-up in useUnreadChannels.ts is the cleaner statement of it — the marker expires, the event it covers does not.

On #6153: agreed. Reading the thread is the intended escape, and eviction-by-message-timestamp is what stops that contract from holding past a week.

Yes to the re-run, please — branch fix/read-state-evict-by-read-recency. The check on a patched build: mark a message older than the 7-day cutoff read, restart, and the marker should survive — in the dump, msg: entries whose value is below the cutoff but whose read recency is recent. Contexts seeded before the fix carry no recency and fall back to the old ranking, so it is the newly written reads that show the change.

@rmichelena

Copy link
Copy Markdown

Re-run done. The branch does what you say on this account's real persisted state. One measurement from it contradicts your caveat about seeded contexts — in your favour — and surfaces one thing you may want to decide on.

What I actually ran

Not a Tauri build against the live account: that means installing an unofficial client on the machine that reported the bug, and I didn't want to do that by default. Instead, at head bc225657e:

  1. The desktop read-state suite on the branch — 50/50 pass (node --import ./test-loader.mjs --experimental-strip-types --test "src/features/channels/readState/*.test.mjs").
  2. A differential replay. The account's actual persisted blobs (buzz.channel-read-state.v2, …source-created-at.v1, …publishable.v1 — 349 contexts) loaded into writeStoredReadState, plus the exact read that failed before: the 2026-08-10T20:51:35Z thread reply, now 10 days old, marked read the way markContextRead does it today (marker value = the message's timestamp, contextSourceCreatedAt = now). Same script, run against both trees.
                     main (417eea223)   pr-5653 (bc225657e)
prunableIn                       331    331
prunableOut                      330    331
markerSurvived                 false    true
markerValue                     null    2026-08-10T20:51:35.000Z
recencyKept                    false    true
stillPublishable               false    true

Dropped on main, durable on the branch, on real data at 33 % of the cap. That is the check you described, minus the relaunch — the relaunch only re-reads what this write produced, and all three keys were written consistently.

Incidentally the branch merges cleanly onto current main (git merge-tree, no conflicts), even though its base is 0.5.10 and readStateManager.ts has moved ~100 lines since.

Your caveat about seeded contexts is too pessimistic — but the reason is worth a look

You wrote that contexts seeded before the fix carry no recency and fall back to the old ranking. On this account, all 330 pre-existing prunable contexts already carry a contextSourceCreatedAt — 330/330, zero fallbacks. main already writes that signal in two places: markContextRead stamps max(now, maxFetchedCreatedAt + 1) (readStateManager.ts:334-340), and publishOneSlot stamps the publish createdAt for every key whose value differs from lastPublishedContexts (readStateManager.ts:725-728). So existing installs should get the benefit immediately, not only for reads written after the upgrade.

The catch is the distribution. Across those 330 contexts there are exactly three distinct recency values, all from the same day:

2026-08-20T19:04:01Z   317 contexts
2026-08-20T19:27:52Z    12
2026-08-20T19:26:51Z     1

That is publish-shaped, not read-shaped. It is consistent with the publishOneSlot site re-stamping en masse — plausibly because trimContextsToBudget had already dropped those keys from the relay blob that seeds lastPublishedContexts, so on the next publish they all read as changed. That would be the same ~60 % trim your description measures, feeding back into the recency signal. I did not instrument the running app, so treat the mechanism as inference; the distribution itself is measured.

If that shape is general, then after any restart-and-publish:

  • the 7-day horizon check stops evicting anything — every surviving context's recency is "since the last publish", so recency >= cutoff is ~always true and LOCAL_MAX_PRUNABLE_CONTEXTS becomes the only real bound;
  • at the cap, sort((a, b) => b.recency - a.recency) is mostly ties, so which context is evicted is decided by sort stability rather than by read recency.

Neither reintroduces the bug — both err toward keeping markers, which is the safe direction, and that is exactly why I'd rather flag it than let it pass silently. But the PR's stated intent ("rank by when the read happened") only holds if the signal isn't overwritten by publishing. Narrowing the write to markContextRead plus the result === "advanced" branch of applyRemoteContextTimestamp would make readActionRecency mean what your eviction comment says it means. Whether that belongs here or in a follow-up is your call — the eviction fix stands on its own either way.

Happy to do the full app-level run too (patched build, mark an old message read, quit, relaunch, dump localStorage) if it would move the review along. It just means putting an unofficial build on the machine that reported this, so I held off rather than assume.

Filed by Bumble, an agent working in @rmichelena's Buzz workspace. Both runs on his machine; main side verified byte-identical to upstream/main for desktop/src/features/channels/readState/.

`publishOneSlot` stamped `contextSourceCreatedAt` for every key whose value
differed from `lastPublishedContexts`. That guard is not the narrow filter it
looks like: `initialize` seeds `lastPublishedContexts` from the union of the
*relay* blobs, while local storage restores the full context set, so every key
`trimContextsToBudget` dropped is missing from the relay copy, reads as changed
on the first publish after each launch, and is restamped with a single
`createdAt`.

Measured on a live account (349 contexts, 330 of them prunable and all carrying
a recency value): only three distinct values across the whole tier, all within
half an hour of each other — publish-shaped, not read-shaped. With that
distribution the 7-day horizon check evicts nothing, because every context looks
freshly read, and at the cap the comparator sees ties, so eviction order falls
back to sort stability instead of read recency.

Recency is now written only where a read happens: `markContextRead` and the
advance branch of `applyRemoteContextTimestamp`. No context loses its signal —
locally marked ones are stamped by `markContextRead`, remotely learned ones by
the merge advance, which is also the path blob-seeded contexts arrive through.

Reported-by: Roberto Michelena <rmichelena@users.noreply.github.com>
Signed-off-by: Tolga Cinisli <tolgacinisli@gmail.com>
@TolgaCinisli

Copy link
Copy Markdown
Author

Confirmed, and fixed in b9900dd — thank you for flagging it rather than letting it pass.

The mechanism is not only inference; it is in the code. initialize seeds lastPublishedContexts from the union of the relay blobs (readStateManager.ts:395-409), while local storage restores the full context set. On a trimmed account the relay copy is missing whatever trimContextsToBudget dropped, so on the first publish after every launch those keys satisfy lastPublishedContexts[key] !== contexts[key] and are stamped with one createdAt. That is your 317 / 12 / 1 distribution.

So the publish-path stamp is gone. Recency is now written only where a read happens — markContextRead, and the advance branch of applyRemoteContextTimestamp, which is also how blob-seeded contexts arrive (readStateManager.ts:357-365). Nothing loses its signal.

It belongs in this PR rather than a follow-up: the argument here is that eviction ranks by read recency, and that only holds once publishing stops overwriting the signal. It also made one sentence of my description false — "the publish path already bumps only changed keys" — now corrected, along with the seeded-contexts caveat you disproved.

Regression test publishing does not refresh read recency: publishes a real blob through a stubbed tauri bridge with lastPublishedContexts empty, then asserts an old read keeps its recency. Fails on bc22565, where the recency jumps to the publish time; passes on the fix. just ci green — desktop 4721, buzz-relay 2402, flutter 1261.

On the two consequences you derived: both dissolve once the signal is read-shaped — the horizon evicts again, and cap comparisons stop being ties. I left the comparator without a secondary key deliberately; after the narrowing the only genuine ties are contexts learned from the same remote event, where any order is equally correct.

No need for the app-level run. Putting an unofficial build on the machine that reported the bug is a worse trade than the differential replay you already did, which exercises the same write path against real persisted state.

One thing this does not fix: the unbounded since: readAt + 1 catch-up. A marker that legitimately ages past the horizon still has its event rediscovered on every launch, so the asymmetry you described survives at the boundary. That deserves its own issue against useUnreadChannels — yours to file if you want it, otherwise I will.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants