perf(claude-agent-sdk): bound the per-turn history replay — repeated injected directive blocks compound until context overflow

[ddotz]

Summary

Split out from #494 (Requested fix 3). #494 / PR #511 fixed the deny-induced retry and double-execution paths, but the third cost factor from the same incident remains: the adapter's per-turn full-history replay is unbounded, and large injected directive blocks compound in it until the prompt exceeds the context window.

Incident data (2026-07-29 KST)

One session's first-message prompt, replayed into a fresh CLI session every turn, over 28 minutes:

time (KST)	prompt bytes	<ultrawork-mode> copies	share of prompt
23:28	167,725	21	15%
23:49	575,091	65	—
23:56	875,961	70	73% (637,732 B)

Each copy is a ~24,871-byte block injected once per matching user input by the ultrawork extension — correct, conditional behavior on the extension side. The compounding comes from buildPromptBlocks() (prompt-bridge.ts) re-serializing the entire message history every turn: every past injection is re-sent forever, and each additional matching input adds another full copy. By 23:56 KST the prompt (~220K tokens) exceeded the model context window; auto-compaction did not intervene during the error-churn window.

Why this is a design gap, not a bug

The replay-per-turn design is documented and intentional (providers.md: "senpi executes every tool itself"; session affinity "keeps Anthropic's prompt cache warm") and is cheap while the cache stays warm. Nothing bounds it once the cache goes cold — limit errors, account failover, provider switches — which is exactly the incident condition.

Options considered

resume (rejected). The SDK exposes resume / forkSession / resumeSessionAt, but adopting them inverts the stateless-bridge design: senpi's session is the single source of truth (forking, compaction, steering, mid-session model switches). A resumed CLI session would carry the interrupted deny artifacts the adapter deliberately discards, would miss turns served by other providers mid-session, and interacts badly with account failover.

Dedupe repeated directive blocks in buildPromptBlocks() (proposed). Keep the first <ultrawork-mode>…</ultrawork-mode> occurrence; replace later byte-identical copies with a short marker; leave non-identical variants untouched. Preserves the documented design and prompt-cache behavior.

Proposed implementation (verified locally against the installed dist)

function dedupeRepeatedDirectiveBlocks(blocks) {
    const OPEN = "<ultrawork-mode>";
    const CLOSE = "</ultrawork-mode>";
    const MARKER = OPEN + "[directive unchanged - identical to the first occurrence above]" + CLOSE;
    let first = null;
    for (const block of blocks) {
        if (block.type !== "text" || !block.text.includes(OPEN)) continue;
        const text = block.text;
        let out = "";
        let i = 0;
        let changed = false;
        for (;;) {
            const s = text.indexOf(OPEN, i);
            if (s === -1) { out += text.slice(i); break; }
            const e = text.indexOf(CLOSE, s + OPEN.length);
            if (e === -1) { out += text.slice(i); break; }
            const end = e + CLOSE.length;
            const span = text.slice(s, end);
            if (first === null) { first = span; out += text.slice(i, end); }
            else if (span === first) { out += text.slice(i, s) + MARKER; changed = true; }
            else out += text.slice(i, end);
            i = end;
        }
        if (changed) block.text = out;
    }
    return blocks;
}
// return blocks.length > 0 ? dedupeRepeatedDirectiveBlocks(blocks) : [{ type: "text", text: "" }];

Verification (running locally since 2026-07-30 KST):

• Unit, incident-shaped context (three identical ~25KB injections + one different variant): full copies kept 1, markers 2, payload 76,656 B → 25,788 B; the incident's 70-copy case scales to ~1.7 MB → ~25 KB + 69 markers.
• Non-identical variants and all user text preserved; contexts without the tag byte-identical.
• Cache-prefix stability holds: serializing the first N messages yields an exact prefix of the full serialization (replacement decisions depend only on earlier content), so Anthropic prompt-cache hits are unaffected.
• E2E smoke through the senpi CLI (claude-agent-sdk provider): capture → in-process execution → replay all normal.

Remaining related gap

Auto-compaction (contextTokens > contextWindow - reserveTokens) is the only bound on replay growth, and it failed to intervene during the incident's error-churn window — worth a separate look at whether errored turns feed the context estimator.

Full transcript archaeology: #494 (comments).

[ddotz]

Post-2026.7.30 re-audit: PR #511 compatible with the capture path; the dedupe is still needed — and the update wiped the local mitigation

Re-audited the adapter on senpi 2026.7.30 (which ships PR #511) for residual burn vectors. Three findings for this issue:

1. PR #511's terminal hook stop introduces no new burn path (verified)

The one place a terminal stop could have backfired is the adapter's result handling: stream.ts throws on result.subtype !== "success", and auth-lane.ts sdkFailure() feeds non-success results into the account-failover rotation — so a hook-stopped turn surfacing as a result error would have traded the deny-retry burn for cache-cold failover replays. It doesn't: the PreToolUse hook fires only after the assistant message carrying the tool_use has fully streamed, so the adapter either takes its early-stop path (message_stop + captured tool call → break) or drains to the final result — and the repo's installed-SDK probe (494-claude-agent-sdk-installed-sdk-hook-stop.test.ts) pins that result to subtype: "success" with terminal_reason: "hook_stopped" and providerRequests: 1. Neither the stream.ts throw nor the failover rotation can fire. Capture → in-process execution → replay confirmed intact on 2026.7.30.

Mitigation note for anyone running the combined local hot-fix from #494: its options.js hunk is obsolete on ≥2026.7.30 and actively harmful — a second hooks key would shadow the upstream hook (which is stronger than the hot-fix: it adds top-level continue: false). Apply only the prompt-bridge dedupe hunk.

2. The 2026.7.30 update silently removed the local dedupe — dist-level mitigation cannot survive updates

senpi update replaces dist/ wholesale, so the dedupe hot-fix vanished with the update while this issue's cost factor (73% of the incident's 876 KB prompt) remains unfixed upstream. Reapplied against 2026.7.30 and re-verified: the proposed hunk applies cleanly (the buildPromptBlocks() tail is unchanged), and the OP's verification matrix reproduces identically — 1 full copy kept, later identical copies → markers, non-identical variants and user text preserved, cache-prefix stability holds, tag-free contexts byte-identical. The fragility is itself an argument for landing this upstream rather than leaving it as a dist patch.

3. <ultrawork-mode> is not the only compounding block

Current-session transcripts show the same growth law on smaller injected blocks per replay: <omo-senpi-task> ×3 (~1.5 KB each), <omo-fallback-architect-reminder> (~3.4 KB), <system-reminder> ×2. Negligible today next to the ~25 KB ultrawork block, but the compounding is identical. Suggest implementing the dedupe over a small set of directive tags (or any repeated byte-identical tagged block) rather than hardcoding <ultrawork-mode>.

---

On the OP's remaining-gap note (auto-compaction as the only bound on replay growth): the compaction side has its own open gap — the per-turn cap and circuit breaker are bypassed on blocking compaction routes. That is #527 / PR #529 territory; I've posted the installed-build verification there rather than here.