perf(store): fix the 32× send-path cliff, and add the SLIMRPC binding - #108
Merged
Conversation
… capacity
The default InMemoryTaskStore caps itself at max_capacity (10,000). Once
full it is over that cap on every subsequent write, and every such write
ran a sweep that cloned every TaskId into a Vec, sorted it, and removed —
normally — one task. A blocking send performs several saves, so a single
message/send paid that O(n log n) sweep several times over.
The result was a cliff, not a slope, and it never recovered: handler-side
send latency went from 65 µs to 2.1 ms at the 10,001st task and stayed
there for the life of the process.
Capacity eviction now walks order_index, which is already sorted
oldest-first, and stops as soon as it has enough victims. The search for a
terminal task to prefer is bounded by EVICTION_SCAN_WINDOW, so a sweep is
O(1) in the store size. The O(n) TTL pass no longer rides along with it:
EvictionPasses separates the two triggers, so the amortized sweep stays
amortized instead of running on every write once the store is full.
Measured (4-core, release, examples/send_probe):
sends so far before after
10,000 65.2 µs 62.9 µs
11,000 2.4 ms 67.3 µs
16,000 2.0 ms 64.3 µs
End to end, transport/jsonrpc/send/single_message improves 91%
(2.18 ms -> 191 µs, p < 0.05). Control: get_task on a missing id, one
round trip with no write, is unchanged at -0.25% (p = 0.76).
This disproves the standing hypothesis, recorded in a benchmark comment,
that the cost was cross-thread scheduling on 4-core runners: the same send
on a single-worker runtime does not close the gap. Scheduling is real but
second-order — about 50 µs of the remaining 191 µs — and was entirely
masked by the eviction cost.
Behaviour change: expired tasks are now reclaimed only by the TTL sweep's
own interval (default every 64 writes), not additionally by any
over-capacity write. max_capacity remains a hard cap enforced on every
write, so memory is bounded exactly as before.
Three new tests pin the properties rather than the timings:
over_capacity_does_not_earn_the_ttl_pass,
a_capacity_only_sweep_does_not_run_the_ttl_pass, and
capacity_eviction_stops_looking_for_terminal_tasks_at_the_window.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QEnNK2WY9W3P4BB79qiKF2
Signed-off-by: Tom F. <tomf@tomtomtech.net>
…it needed Carries A2A over the AGNTCY SLIM fabric per a2aproject/experimental-cpb-slimrpc, advertising protocolBinding https://a2a-protocol.org/bindings/experimental-slimrpc/v1 and addressing agents as slim://[node[:port]/]domain/namespace/service. All eleven methods in the spec's inventory: nine unary, plus SendStreamingMessage and SubscribeToTask as unary-request/streaming-response. Payloads are the canonical lf.a2a.v1 protobuf messages — SLIMRPC uses the same service definitions as gRPC — so the wire is byte-compatible with the official Go, Python and Java SDKs. Error identity travels as the spec's "TaskNotFoundError: ..." message prefix, because SLIMRPC has no google.rpc.ErrorInfo equivalent and a status code alone cannot distinguish TaskNotCancelableError from ExtensionSupportRequiredError. SlimRpcServer drives the same RequestHandler every other binding drives, so task state, streaming, push, tenancy and authorisation are not reimplemented and an agent behaves identically however it is reached. Deliberately outside the workspace, with its own Cargo.lock. agntcy-slim-rpc brings 379 transitive dependencies including aws-lc-sys, a native C crypto build; a2a-protocol-types has 12 and a2a-protocol-server 191 at all features. None of that reaches the lockfile, deny.toml allow-list or audit surface of the four published crates, and none of them depends on this one. One change was needed in a published crate, and it is the interesting part. Transport::send_streaming_request must return an EventStream, and every EventStream constructor was pub(crate) — so an out-of-tree transport could implement the unary half of the trait and not the streaming half. The trait is pub, its parameters are pub, its return type is pub, and it was still unimplementable from outside. EventStream::from_event_channel closes that: it takes a channel of decoded StreamResponse values, so a transport hands over domain events without knowing the internal representation is SSE, and an Err on the channel reaches the consumer rather than ending the stream silently. That gap is not visible from the signatures. docs/rust-sdk-assessment.md §4.1.1 previously concluded a SLIMRPC binding needed no change to these crates, verified by reading declarations; the correction is recorded there, along with the dependency measurement that strengthens the separate-crate design. 29 tests. The seven end-to-end ones are not mocked: one in-process SLIM Service hosts an agent app and a caller app and messages cross the real SLIM datapath, covering method registration, a unary round trip, a task fetched back by id, error identity surviving the fabric, a streaming send running to its terminal event, agent-card advertisement, and an unknown method being reported rather than hanging. CI gets a dedicated job: an out-of-workspace crate is skipped by every other job, so without one it would rot while the workspace stayed green. Not implemented: multicast (a separate spec document). Not verified: behaviour across a remote SLIM node — every test here runs in-process. Both are stated in the crate README rather than left to be discovered. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QEnNK2WY9W3P4BB79qiKF2 Signed-off-by: Tom F. <tomf@tomtomtech.net>
…IM node
Two things the previous commit listed as not done.
Multicast implements the separate spec/v1/slimrpc-multicast.md. SlimRpcMulticast
opens a SLIM group channel, invites specific agents by name, and broadcasts.
Only SendMessage and SendStreamingMessage may be broadcast — task management
stays point-to-point, because a task id is meaningful to exactly one agent.
MulticastOutcome carries exactly one outcome per invited agent. That is the
spec's requirement ("Clients must wait for outcomes from every invited agent")
and the reason multicast is not a Transport: send_request returns one value, and
reducing N attributable answers to one would have to drop either the attribution
or the failures.
Two failure kinds stay distinct because they call for different responses. An
agent that errors or stays silent past the timeout is an isolated per-agent
outcome, and the other agents' answers stand. A member that cannot be invited at
all fails the whole call, because the group is misconfigured and waiting will not
fix it — the spec draws the same line: "Only channel creation, agent invitation,
or request delivery failures constitute interaction-level failures."
stream_message gives each agent its own EventStream, demultiplexed from SLIM's
interleaved source-tagged frames.
tests/remote_node.rs runs three separate SLIM services in one process — an
agent, a client, and a node that only routes — connected over loopback TCP. The
agent and client share no Service and no memory; every message crosses a socket
twice and is routed in between.
It found a real bug that in-process testing structurally could not: nothing
announced a client's own name to the node, so while the agent was reachable, no
route existed for its reply and every call failed its session handshake with the
caller's own name reported as unroutable. Channel sets a route outwards only.
The client-side constructor is now async and subscribes the caller's name over
the connection.
Two mistakes worth recording, because both were tests that could not fail:
The multicast join key was wrong — a name arrives on the wire with a fourth
instance component (org/ns/agent/NULL_COMPONENT), so keying on the full
rendering filed every response under a name no member matched. All three agents
answered and all three were reported as timeouts. The unit test missed it by
building both sides of the comparison from the same value; it now asserts
against the literal wire rendering.
The first attempt at the remote-node fix also changed the server's base name
from app.app_name() to the three-component address, with a comment claiming that
was what the remote suite caught. Reverting it showed the tests still pass, so
it was not load-bearing and the comment was a false claim. Reverted, comment
corrected to what is actually verified.
45 tests: 25 unit, 18 end-to-end across three topologies, 2 doc. e2e.rs now
shares tests/common with the two new suites instead of carrying its own copy of
the fixtures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QEnNK2WY9W3P4BB79qiKF2
Signed-off-by: Tom F. <tomf@tomtomtech.net>
…ess, identity The previous commit listed four limitations. Three were real work and are now done; the fourth was a false claim of mine and is corrected. TLS between node and app. remote_node_tls.rs generates a throwaway CA per run rather than committing PEM files, so nothing long-lived is in the repository and no certificate can expire the build. The rejection test is a differential against the same node in the same window — the trusted CA connects, the untrusted one does not — because SLIM retries a failed handshake rather than returning, and a bounded wait on its own would only prove slowness. Multi-hop. remote_node_multihop.rs runs two nodes peered with ConnType::Peer and puts the agent behind the far one. An Edge link carries an attached app's traffic but does not share routing state, so the agent stays invisible without the peer link; the subscription crossing that link is what the test exercises. Unary and streaming both, because frame-by-frame delivery is where a second hop is most likely to go wrong. Out of process. src/bin/slim_node.rs is a standalone node — it routes and runs nothing itself — and out_of_process.rs spawns it with Command. It shares nothing with the test but a TCP port, which is the part of "on another host" that reproduces on one machine: no shared memory, no shared runtime, independent lifetimes. The binary prints "listening on <addr>" once the socket accepts so a supervisor waits for readiness instead of sleeping, and refuses half a TLS configuration rather than silently serving plaintext. It also stands alone as a tool: bringing up a node otherwise means installing the full AGNTCY SLIM distribution. Identity. with_identity now takes SLIM's own AuthProvider and AuthVerifier instead of enumerating mechanisms, so JWT, SPIFFE via SPIRE and static tokens work without this crate growing a method per mechanism and lagging behind SLIM. with_shared_secret becomes a convenience over it, and fallible, since SLIM rejects a secret too short to be a credential. A builder with no identity is a build error: SLIM has no anonymous mode and a default would quietly stand in for one. An end-to-end JWT test proves the general door works, not just the convenience. The fourth item was wrong. I documented that SLIM offers a multicast group inbox via subscribe_group_inbox and this binding did not expose it. That symbol does not exist anywhere in the SLIM crates — it appears exactly twice in the whole source, both in a stale doc comment in upstream's own test file describing a test that is not in the file. I repeated it as fact without checking. Removed. 53 tests across six topologies: in-process, multicast group, one node over TCP, that node with verified TLS, two peered nodes, and a node in its own process. Every file is under the 500-line guideline; client/ and the remote-node suite were split to keep it that way. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QEnNK2WY9W3P4BB79qiKF2 Signed-off-by: Tom F. <tomf@tomtomtech.net>
Both were listed as limitations last time. Neither is now, and one of the reasons I gave was wrong. SPIFFE runs against a real spire-server and spire-agent. The agent attests the test process over the Workload API and issues genuine JWT-SVIDs; a stub would have proven only that the types line up. Three tests: an A2A call carried by SPIFFE identity, the issued SPIFFE ID pinned to the one registered, and a verifier refusing a genuinely-issued SVID minted for a different audience. The last gives the first its meaning — a verifier that accepted everything would pass the positive test. Two SPIFFE properties surfaced only by running it, and both present as a session that never completes rather than as an authentication error: SpireIdentityManager must be built once and cloned for provider and verifier. It generates an MLS signature key at build time and embeds the public half in the SVID audiences, so two separately-built managers carry two different keys and their handshake never finishes. Each app needs its own SPIFFE ID. The unix workload attestor identifies by uid, so every app in one test binary is one workload and would share an identity — and two SLIM apps holding the same SPIFFE ID cannot complete an MLS handshake. The testbed registers an entry per app and selects between them with with_target_spiffe_id, which is how a process hosting several workload identities is meant to work. Mutual TLS covers a node that authenticates its apps rather than only itself. Three cases, meaningful only together: a certificate from the node's client CA connects and A2A works; no certificate is refused; a well-formed ClientAuth certificate from a different CA is refused. The first alone would pass just as happily against a node ignoring client certificates entirely. Every negative is paired with a control that succeeds in the same window, because SLIM retries a failed handshake rather than returning, so "did not connect" alone can mean the fixture broke rather than the control working. The correction: I previously recorded that running SPIRE needed an agent this environment did not have. I never checked. SPIRE downloads and runs here without trouble, which is how these tests exist at all. The SPIFFE suite is #[ignore]d because it needs the SPIRE binaries, so a contributor without them is not blocked; CI installs SPIRE and runs --ignored explicitly. The testbed panics rather than skipping when they are missing — a SPIFFE test that quietly passes without SPIRE reports coverage that does not exist, which is worse than no test. 59 tests across eight topologies. The README now carries a security posture table separating what is verified by a test from what is merely available, because "supported" and "verified" are different claims. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QEnNK2WY9W3P4BB79qiKF2 Signed-off-by: Tom F. <tomf@tomtomtech.net>
…st real SPIRE The two limitations left standing last time. Both are now tested rather than documented. Federation runs two independent SPIRE deployments. An SVID from an unfederated trust domain is refused; after a bundle exchange and entries naming each other, the same SVID is accepted; and a full A2A call runs between an agent attested by one organisation's SPIRE and a caller attested by another's. Both halves are needed — acceptance alone is indistinguishable from a verifier that ignores trust domains, rejection alone from federation being broken. That work surfaced an ordering rule: an entry naming -federatesWith is rejected outright unless that trust domain's bundle is already imported. Bundles before entries. The testbed now splits start_with from register so the ordering is visible at the call site rather than being a comment someone can miss; the first attempt did both in one call and failed with "unable to find federated bundle". Rotation issues 40-second JWT-SVIDs so a renewal happens inside a test instead of half an hour later. Three properties: the manager serves a renewed credential with nobody asking it to, the superseded credential stops verifying, and a live A2A agent keeps answering across the rotation — including a stream opened afterwards — without a restart or a new manager. Each rotation test proves the rotation actually happened before asserting anything about it. A test that waited and then made a successful call would pass just as happily if nothing had rotated, which would be worse than no test: it would report coverage of precisely the thing it missed. 65 tests across ten topologies; 56 run by default, 9 against real SPIRE. The rotation suite is slow by construction — it waits for wall-clock expiry, which is the only way to test what happens after it. Remaining, and stated because it is true rather than because it is comfortable: federation here is manual bundle exchange, not a bundle endpoint; rotation covers JWT-SVIDs, which is what SLIM's app identity uses, not X.509 SVIDs or the node's own TLS certificate under a live connection; and none of it runs under sustained load, so nothing here would catch a slow degradation over hours. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QEnNK2WY9W3P4BB79qiKF2 Signed-off-by: Tom F. <tomf@tomtomtech.net>
Written for a fresh session picking this up: what is settled, what it cost to learn, and what is genuinely next. The settled parts are the send-path cliff (cause found, 91% off, with the scheduling hypothesis disproved) and the SLIMRPC binding (eleven methods plus multicast, 65 tests across ten topologies, three of them against real SPIRE). The section deliberately records four things that cost time to discover and would cost it again: SLIM names carry a fourth instance component; a client must announce its own name to the node or nothing can route a reply back; SpireIdentityManager must be built once and cloned, and each app needs its own SPIFFE ID; and federation is bundles-before-entries. Next three, in order: soak testing — the one axis none of this covers and the class of bug the suite structurally cannot reach, since every test starts from an empty store — then splitting handler/messaging.rs, then the two missing examples at either end of the funnel. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QEnNK2WY9W3P4BB79qiKF2 Signed-off-by: Tom F. <tomf@tomtomtech.net>
… it turned on `cargo-mutants` on the PR diff (shard 6) surfaced two survivors in `capacity_victims`, the sweep this branch introduced. They are different in kind and only one is a test gap: overflow - terminal.len() -> + a real gap still_running.len() < overflow -> <= equivalent The first is the one that matters. Sizing the top-up with `+` makes the sweep evict *more* than the overflow, and no test noticed: every capacity test either fills the quota from terminal tasks alone and returns before the top-up runs, or finds no terminal tasks at all, where the two operators agree. The partial-supply case between them had no test at all. Verified rather than assumed. Reconstructing the previous implementation with the `-` -> `+` mutation applied and the new test added, the new test fails and the other ten still pass — so the mutant really was invisible to the suite, and this test is what closes it. The second survivor is genuinely equivalent: `still_running` is only ever consumed by `.take(shortfall)` with `shortfall <= overflow`, so letting it grow one element longer cannot change the result. Per the standing rule recorded in mutants.yml, an equivalent mutant gets its shape deleted rather than excluded. Rewriting the top-up as a second bounded pass over the same window, sharing the first pass's exit condition, removes both operators at once — and is less work besides: one allocation of exactly the victims instead of two, with no eager cloning of in-flight ids that are usually discarded. Behaviour is unchanged; all five pre-existing capacity tests pass untouched. Also worth recording: the first local mutation run exited 0 having crashed — out of disk after copying the SLIMRPC binding's 16 GB target/ — and produced no report whatsoever. That is precisely the vacuous pass the workflow's "Require a readable mutation report" step exists to catch. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Tom F. <tomf@tomtomtech.net>
… line limit The mutation fix in 8b972de took eviction.rs from 499 lines to 569, across CONTRIBUTING's 500-line rule, and scripts/check_file_lengths.sh caught it. That is the check working: the file was one line under before, so any addition at all would have crossed it. Recording an exemption was the sanctioned alternative and would have been the wrong one. The file already documented the seam it needed — "two passes, on two schedules, because they cost different things" — so the split was waiting to be made rather than invented to satisfy a counter: eviction/mod.rs the schedule. EvictionPasses, should_evict, maybe_evict, evict, and the amortized O(n) TTL pass. eviction/capacity.rs the pass that runs on every write once the store is full. EVICTION_SCAN_WINDOW, evict_over_capacity, capacity_victims, is_terminal. Its bound is the whole subject of the file, which is the point of separating it. eviction/fixtures.rs store_of/config/ids. Shared rather than copied because both suites must agree on what "oldest first" means; two copies that drifted would let one stop matching the other without either failing. The TTL and capacity bodies are now named functions instead of inline branches of evict, leaving evict as the gate it actually is. No behaviour change. The same 11 tests pass, unmodified apart from their module homes. check_file_lengths.sh returns to "79 of 387 tracked source files exceed 500 lines, all recorded" — the ratchet list did not grow — and check_mutation_scope.sh still reports 141 of 141, so all three files remain inside the per-PR mutation gate rather than slipping out of it. Also: the local fmt check that let 8b972de through was unsound. It read `cargo fmt --all --check | head -20 && echo "FMT OK"`, where && binds to head's exit status, so it printed OK regardless — and the four non-rustfmt steps of the Format job were never run locally at all. All five are checked individually here. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Tom F. <tomf@tomtomtech.net>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Two independent pieces of work.
1.
message/sendcost 32× more past 10,000 tasksThe standing hypothesis — cross-thread scheduling on 4-core runners — is disproved: the same send on a single-worker runtime does not close the gap, and cutting an executor event changes it by 2%.
The cause was
InMemoryTaskStore. It caps atmax_capacity(10,000); once full it is over the cap on every subsequent write, and each write cloned everyTaskIdinto aVec, sorted it, and removed one task. A blocking send performs several saves, so one request paid that O(n log n) sweep several times. A cliff, not a slope, and it never recovered.Capacity eviction now walks
order_index(already sorted oldest-first) with a bounded scan window, and the O(n) TTL pass no longer rides along with it —EvictionPassesseparates the two triggers so the amortized sweep stays amortized.The eviction module is split along that same seam, which is why the diff shows three files where there was one:
eviction/mod.rsholds the schedule and the O(n) TTL pass,eviction/capacity.rsholds the pass that runs on every write once the store is full, andeviction/fixtures.rsholds the store builders both suites share.Behaviour change: expired tasks are reclaimed only by the TTL sweep's own interval (default every 64 writes), not additionally by any over-capacity write.
max_capacityremains a hard cap enforced on every write, so memory is bounded exactly as before.2.
bindings/a2a-protocol-slimrpc— the SLIMRPC protocol bindingAll eleven methods from
a2aproject/experimental-cpb-slimrpcplus multicast, advertisingprotocolBinding: https://a2a-protocol.org/bindings/experimental-slimrpc/v1. Payloads are the canonicallf.a2a.v1protobuf messages, so the wire is byte-compatible with the official Go, Python and Java SDKs.SlimRpcServerdrives the sameRequestHandlerevery other binding drives — task state, streaming, push, tenancy and authorisation are not reimplemented.Deliberately outside the workspace, with its own
Cargo.lock.agntcy-slim-rpcbrings 379 transitive dependencies includingaws-lc-sys, a native C crypto build;a2a-protocol-typeshas 12. None of that reaches the lockfile,deny.tomlallow-list or audit surface of the four published crates.One change to a published crate was needed, and it is the finding worth carrying forward.
Transport::send_streaming_requestmust return anEventStream, and everyEventStreamconstructor waspub(crate)— so a third-party binding could implement the unary half of the trait and not the streaming half. The trait ispub, its parameters arepub, its return type ispub, and it was still unimplementable from outside.EventStream::from_event_channelcloses that; purely additive.docs/rust-sdk-assessment.md§4.1.1 previously concluded no change was needed, verified by reading declarations — the correction is recorded there.Also adds
slim-node, a standalone SLIM node binary, so bringing one up for local development does not require installing the full AGNTCY distribution.How it was verified
The binding: 65 tests across ten topologies, 56 by default and 9 against a real SPIRE deployment. None are mocked.
e2e.rsService— eleven methods, error identity, streaming, JWT identitymulticast.rsremote_node.rsremote_node_tls.rsremote_node_mtls.rsremote_node_multihop.rsout_of_process.rsspiffe.rsspiffe_federation.rsspiffe_rotation.rsEvery negative case is paired with a control that succeeds in the same window, because SLIM retries a failed handshake rather than returning, and "did not connect" alone can mean the fixture broke rather than the control working.
CI gets a dedicated job (an out-of-workspace crate is skipped by every other job), which installs SPIRE and runs the
#[ignore]d suites explicitly. The SPIRE testbed panics rather than skipping when the binaries are missing, so it cannot silently report coverage it does not have.Three bugs were found by tests the tier above could not have caught:
remote_node.rs— a client never announced its own name to the node, so nothing could route an agent's reply back and every call failed its session handshake. Invisible in-process.multicast.rs— the response join key was wrong; SLIM names arrive with a fourth instance component, so every response filed under a name no invited member matched. All agents answered; all were reported as timeouts.spiffe.rs— two apps sharing one SPIFFE ID cannot complete an MLS handshake.A fourth was found by this PR's own mutation gate, after review had passed it — see the
cargo mutantschecklist item.Known limits, stated rather than implied
Federation is by manual bundle exchange, not a bundle endpoint. Rotation covers JWT-SVIDs (what SLIM's app identity uses), not X.509 SVIDs or the node's own TLS certificate under a live connection. Everything runs on one machine, so real network loss, latency and NAT are untested. Static-token identity is supported via
with_identitybut has no test. The crate README carries a posture table separating verified by a test from merely available.ROADMAP.mdrecords the handoff: what is settled, the five things that cost time to learn, and the next three items — soak testing first, since it is the one axis none of this covers and the class of bug the suite structurally cannot reach.Checklist
Tom F. <tomf@tomtomtech.net>per PROVENANCE.md §3.2, withCo-Authored-By: Clauderetainedcargo fmt --allpassescargo clippy --workspace --all-targets -- -D warningspassescargo test --workspacepasses — 2,738 passed, 0 failedcargo doc --workspace --no-depspasses without warningscargo mutants— run, and it found a real bug this PR had introduced. The incremental gate flagged two survivors in the newcapacity_victims:overflow - terminal.len()->+, andstill_running.len() < overflow-><=. The first is a genuine test gap — sizing the top-up with+evicts more than the overflow, and no test noticed, because every capacity test either filled the quota from terminal tasks alone (returning before the top-up ran) or found none at all (where the two operators agree). The second is equivalent: the vector is only consumed by.take(shortfall)withshortfall <= overflow. Both addressed in8b972de— a test for the seam, verified to fail against the previous implementation with the mutation applied while the other ten pass, and the equivalent mutant's shape deleted rather than excluded, per the standing rule inmutants.yml. Re-measured after the fix: 27 mutants, 25 caught, 2 unviable, 0 missed. The binding is still outside the workspace, so the gate does not reach it.CHANGELOG.mdupdateddocs/rust-sdk-assessment.md§4.1.1 rather than as an ADR; say the word if you want it promoted.Generated by Claude Code