diff --git a/.github/meta/harness-review-followups.md b/.github/meta/harness-review-followups.md new file mode 100644 index 0000000000..b742a46e80 --- /dev/null +++ b/.github/meta/harness-review-followups.md @@ -0,0 +1,65 @@ +# Harness reliability review — deferred follow-ups + +Deferred MED findings from the pre-PR release review (pre-PR adversarial review). +All 5 HIGH findings plus selected MED/LOW items were fixed on this branch; the +items below were explicitly deferred and are listed verbatim from the review. + +[MED] packages/opencode/src/tool/truncation.ts:66 — the plain-async truncation path hardcodes 2,000 lines/50KiB while the Effect wrapper honors `tool_output` configuration — MCP output through `prompt.ts` therefore ignores user caps despite the shared-core claim — consolidate the wrappers or pass the resolved configuration through both, with parity tests. + +[MED] packages/opencode/src/session/compaction.ts:663 (renderCarryAnchors) — carry-anchor trimming stops when one item remains — one oversized model-generated "Accomplished" item defeats `maxTokens` and can undo compaction — permit dropping or truncating the final item and assert the rendered result satisfies the cap. + +[MED] packages/opencode/src/cli/cmd/idle-done.ts:157 — every command not recognized as read-only is treated as verification — an exit-zero install, cleanup, deployment, or arbitrary unknown command can satisfy the "green verify" precondition and trigger a false completion challenge — require configured or positively classified verification evidence; unknown commands should be ineligible. + +[MED] packages/opencode/src/session/compaction.ts:70 — observation masks retain the first 80 characters of pruned output, while the ledger retains raw command/path/pattern text — credentials, authorization headers, query data, and signed URLs can survive pruning and be recopied into later synthetic prompts — retain only allowlisted metadata or hashes and apply shared secret redaction. + +[MED] packages/opencode/src/session/compaction.ts:572 (renderLedger) — ledger capping repeatedly joins and re-estimates the whole array while removing one line at a time, after collecting the full session history — this is quadratic in unique writes and adds latency at the critical compaction path — bound collection early and trim using accumulated token costs or a single cutoff search. + +The following items from a later review pass were also considered and deliberately +deferred (no behavior change on this branch): + +[LOW] packages/opencode/src/cli/cmd/run/run-mode.ts — the run entrypoint writes its process-scoped mode marker into the environment for the process lifetime, and child processes inherit it; a nested interactive server launched from such a session would arm run-mode mechanisms for genuinely interactive clients — clear or scope the marker in the interactive entrypoints (mirroring the existing child-env cleanup for the sibling non-interactive marker). + +[LOW] packages/opencode/src/session/compaction.ts — module-level per-session pin state and per-session tracker read maps grow without bound in a long-lived server process (the pin map has no production eviction path; per-session read tracking keeps one entry per unique path for the session's lifetime) — bound both with the same LRU pattern now used by the session-state stores. + +[LOW] packages/opencode/src/session/prompt.ts — the post-compaction pinned-task reminder re-derives its source by streaming the FULL session history from the database on every generation once a session has compacted — cache the resolved pin source per session or query only the needed boundary messages. + +[LOW] packages/opencode/src/session/termination.ts — the post-compaction three-option completion nudge (including the completion-token instruction) is injected in ALL modes; interactive users can see an occasional bare completion token line with no interactive function — mode-gate the nudge text or document the cosmetic change. + +The following items came from an external multi-reviewer pass over this branch and +were deliberately deferred (no behavior change on this branch): + +[MED] packages/opencode/src/cli/cmd/run.ts — a prompt retry after a client timeout or connection reset can hit the server AFTER it accepted the original POST, sending the same task again and creating a second user message / duplicate execution — the retry loop has no way to tell "not yet accepted" from "accepted, response lost." Needs a stable idempotency/message key the server honors, or a way to confirm the first attempt was never accepted before retrying. + +[MED] packages/opencode/src/session/compaction.ts (fitHead) — the summarization-request budget reserves a fixed 2,000 tokens for the summary prompt, but the actual assembled prompt (default template + carry anchors + pin-summary addition + first-person reframe, or a plugin-supplied override) can exceed that on an active session — size the reservation from the actual assembled prompt instead of a constant. + +[MED] packages/opencode/src/session/compaction.ts (buildLedger) — on a session's second or later auto-compaction, the ledger is built from the already-filtered/compacted message view, not the full session stream, so verified-write facts from before the first compaction silently drop out of later ledgers — build from the full stream and let selection filter afterward. + +[LOW] packages/opencode/src/session/prompt.ts (uncountedTail) — the proactive overflow estimate sums tool-result tokens on messages AFTER the last-finished assistant message, but a tool call and its result can live on that SAME message when the turn is still mid-flight — those results are excluded from the estimate, so compaction can fire a step later than it should on a heavy tool-output turn. + +[LOW] packages/opencode/src/tool/truncate-core.ts (preview, middle direction) — a degenerate `maxBytes: 1` config (no realistic caller sets this) can still allocate one byte to each of the head/tail halves and exceed the byte budget by a small margin; the equivalent `maxLines: 1` case was already fixed by degrading to tail-only — extend the same degrade to the byte-only case. + +[LOW] packages/opencode/src/session/{termination,nudge,tool-result-cap}.ts, packages/opencode/src/cli/cmd/{run-accounting,idle-done}.ts — these 5 new modules use `export namespace` for organization, which the repo's module-shape convention (`packages/opencode/AGENTS.md`) asks new code to avoid in favor of flat exports + a bottom-of-file self-reexport. ~69 pre-existing files in the package already use the same pattern, so this is consistent with existing debt rather than a regression; fold into a holistic namespace-to-flat-exports cleanup across the package rather than converting these 5 files in isolation. + +[LOW] packages/opencode/test/session/starvation.test.ts — the run-mode "armed" gate test re-implements the gate expression as a local helper instead of importing the real predicate from processor.ts, so a future change to the actual gate (added condition, reordered precedence, renamed exemption) would not be caught by this test — extract the gate into a shared, directly-testable predicate. + +The following items came from a further multi-reviewer pass over this branch and +were deliberately deferred (no behavior change on this branch): + +[MED] packages/opencode/src/session/llm.ts (addHistoricalToolStubs) — the stub-injection skip was narrowed to `toolChoice === "none" && no tools`, which closes the normal-turn regression, but the compaction summarizer still takes that path with a head that references historical tool calls. The skip rests on the claim that omitting both `tools` and `tool_choice` is universally accepted; the issue the stubs exist to fix concerned validation of referenced historical tool calls, which is orthogonal. Verify against the provider that originally needed the stubs before changing anything — reintroducing stubs here would advertise callable tools on a text-only call, so this is a provider-compatibility question, not a code cleanup. + +[MED] packages/opencode/src/session/starvation.ts (repeatSignature / consecutive-signature counter) — the repeat signature folds the failure message in but the counter accumulates for successful calls too, so three identical SUCCESSFUL calls (re-running the same verification command, polling a status probe) reach the threshold and produce a directive asserting that repeating the call cannot change the result, which is untrue for a polling or verification call. Restricting accumulation to calls that carry a failure message narrows the detector's semantics and its overlap with the identical-args doom-loop ladder; make that change with validation data rather than in review, and note the directive text is prompt-visible. + +[LOW] packages/opencode/src/cli/cmd/run-accounting.ts (termination) — `why_model_stopped` falls back to "stop" when no step-finish reason was ever recorded (an aborted run that never completed a step), so the run record attributes an ordinary stop to a session that never reported one. Distinguishing it needs a new value in the published record's enum, which is an output-contract change for downstream consumers — batch it with the next deliberate revision of the run record schema. + +[LOW] packages/opencode/src/session/prompt.ts (post-compaction pin reminder) and packages/opencode/test/cli/idle-done.test.ts — the idle-done unit tests now pin production event ordering (step-finish part before the step's snapshot patch part) in two dedicated cases, but the shared fixtures still emit the patch first; converge the remaining fixtures on production ordering when the file is next touched. + +The following items came from the fourth review pass over this branch and were +deliberately deferred (no behavior change on this branch): + +[MED] packages/opencode/src/session/processor.ts (dispatch tool-result cap) — the per-result hard cap is applied on the successful `tool-result` branch only. A `tool-error` still persists an unbounded error string, and an interrupted running tool keeps partial output in metadata that `message-v2.ts` later replays as a tool result, so a failed call with very large stderr can still overflow the next request. Applying the cap to error text and interrupted partial output changes what gets PERSISTED on the failure path, which is where diagnostics come from — size it against real failure payloads before truncating them. + +[MED] packages/opencode/src/cli/cmd/idle-done.ts (verify classification) — with no configured verify command, ANY non-read-only bash command is treated as a verification, so a mutating command that is not on the mutating-heads list (a build script that also writes generated sources, say) can register as the green verify it is not. The mutation watermark now advances for the in-place, redirection, and known-mutating-head forms, but the underlying "not read-only implies verification" inference still needs replacing with an explicit verify classifier. + +[LOW] packages/opencode/src/cli/cmd/run.ts (--max-turns) — `--max-turns 0` disables the limit entirely because the guard is a truthiness check, and negative or fractional values are accepted without validation. For a governance control an explicit zero should not mean unlimited; fix it together with the CLI validation pass that gives the other numeric options explicit range errors, so the behavior change is documented in one place. + +[LOW] packages/opencode/src/tool/bash.ts (child env) — `ALTIMATE_RUN_RESUMED` joins `ALTIMATE_RUN_MODE` as a marker that nested processes inherit from the bash tool's merged env. Its leak direction is benign (a nested run would use interactive pin selection), but it belongs in the same holistic decision about which run-scoped markers get stripped for child processes. diff --git a/packages/core/src/config/compaction.ts b/packages/core/src/config/compaction.ts index 3c5960c835..5dc3bcb55b 100644 --- a/packages/core/src/config/compaction.ts +++ b/packages/core/src/config/compaction.ts @@ -1,10 +1,14 @@ export * as ConfigCompaction from "./compaction" import { Schema } from "effect" -import { NonNegativeInt } from "../schema" +import { NonNegativeInt, SAFETY_FRACTION_MIN } from "../schema" export class Keep extends Schema.Class("ConfigV2.Compaction.Keep")({ tokens: NonNegativeInt.pipe(Schema.optional), + // altimate_change start — V2 parity for the fork's verbatim-tail turn count + // (V1 compaction.tail_turns; 0 disables the tail entirely). + turns: NonNegativeInt.pipe(Schema.optional), + // altimate_change end }) {} export class Info extends Schema.Class("ConfigV2.Compaction")({ @@ -12,4 +16,32 @@ export class Info extends Schema.Class("ConfigV2.Compaction")({ prune: Schema.Boolean.pipe(Schema.optional), keep: Keep.pipe(Schema.optional), buffer: NonNegativeInt.pipe(Schema.optional), + // altimate_change start — V2 parity for the fork compaction keys (estimator + // safety margin, state ledger/summary carry, task pin). Same names as V1 so + // ConfigMigrateV1 can carry them through without renames. + // upstream_fix: Config.load decodes a document authored directly in V2 (not + // migrated from V1) through THIS schema, so the V1 bounds on these two + // fractions don't apply here — a direct V2 document could carry an + // out-of-range value straight through to + // SessionCompaction.contextSafetyFraction / pinBudget. Bound identically to + // the V1 schema (packages/core/src/v1/config/config.ts). + // SAFETY_FRACTION_MIN: Schema.toArbitrary's fast-check generator requires + // `.check()` bounds to be exact 32-bit floats, and the bound must round DOWN + // so the documented minimum `0.1` still decodes; see the matching V1 comment. + context_safety_fraction: Schema.Number.check( + Schema.isGreaterThanOrEqualTo(SAFETY_FRACTION_MIN), + Schema.isLessThanOrEqualTo(1), + ).pipe(Schema.optional), + state_ledger: Schema.Boolean.pipe(Schema.optional), + ledger_max_tokens: NonNegativeInt.pipe(Schema.optional), + ledger_recent_calls: NonNegativeInt.pipe(Schema.optional), + summary_carry: Schema.Boolean.pipe(Schema.optional), + summary_first_person: Schema.Boolean.pipe(Schema.optional), + pin_task: Schema.Boolean.pipe(Schema.optional), + pin_max_tokens: NonNegativeInt.pipe(Schema.optional), + pin_window_fraction: Schema.Number.check(Schema.isGreaterThanOrEqualTo(0), Schema.isLessThanOrEqualTo(1)).pipe( + Schema.optional, + ), + pin_card_max_tokens: NonNegativeInt.pipe(Schema.optional), + // altimate_change end }) {} diff --git a/packages/core/src/config/experimental.ts b/packages/core/src/config/experimental.ts index 12a02635db..68739d88fb 100644 --- a/packages/core/src/config/experimental.ts +++ b/packages/core/src/config/experimental.ts @@ -3,6 +3,9 @@ export * as ConfigExperimental from "./experimental" import { Schema } from "effect" import { Catalog } from "../catalog" import { Policy as PolicyV2 } from "../policy" +// altimate_change start — V2 parity for the write-starvation breaker +import { PositiveInt } from "../schema" +// altimate_change end // Each core domain exports the policy actions it supports. Adding an action to // this union makes it valid in authored config while keeping Policy generic. @@ -13,6 +16,24 @@ export class Policy extends Schema.Class("ConfigV2.Experimental.Policy") action: PolicyAction, }) {} +// altimate_change start — V2 parity for the write-starvation breaker keys. +// Same names as V1 (config.ts experimental.starvation_breaker) so +// ConfigMigrateV1 can carry them through without renames. +export class StarvationBreaker extends Schema.Class("ConfigV2.Experimental.StarvationBreaker")({ + mode: Schema.Literals(["off", "annotate", "armed"]).pipe(Schema.optional), + max_turns_without_mutation: PositiveInt.pipe(Schema.optional), + repeat_signature_threshold: PositiveInt.pipe(Schema.optional), + doom_loop_threshold: PositiveInt.pipe(Schema.optional), + polling_threshold_multiplier: PositiveInt.pipe(Schema.optional), + polling_pattern: Schema.String.pipe(Schema.optional), + exempt_agents: Schema.String.pipe(Schema.Array, Schema.optional), + generated_path_patterns: Schema.String.pipe(Schema.Array, Schema.optional), +}) {} +// altimate_change end + export class Experimental extends Schema.Class("ConfigV2.Experimental")({ policies: Policy.pipe(Schema.Array, Schema.optional), + // altimate_change start — V2 parity for the write-starvation breaker + starvation_breaker: StarvationBreaker.pipe(Schema.optional), + // altimate_change end }) {} diff --git a/packages/core/src/config/tool-output.ts b/packages/core/src/config/tool-output.ts index 53e4d4d088..af2edaeea7 100644 --- a/packages/core/src/config/tool-output.ts +++ b/packages/core/src/config/tool-output.ts @@ -6,4 +6,7 @@ import { PositiveInt } from "../schema" export class Info extends Schema.Class("ConfigV2.ToolOutput")({ max_lines: PositiveInt.pipe(Schema.optional), max_bytes: PositiveInt.pipe(Schema.optional), + // altimate_change start — V2 parity for the per-tool-result dispatch cap + dispatch_max_tokens: PositiveInt.pipe(Schema.optional), + // altimate_change end }) {} diff --git a/packages/core/src/schema.ts b/packages/core/src/schema.ts index 520491ba25..271b9e3b19 100644 --- a/packages/core/src/schema.ts +++ b/packages/core/src/schema.ts @@ -19,6 +19,18 @@ export const PositiveInt = Schema.Int.check(Schema.isGreaterThan(0)) */ export const NonNegativeInt = Schema.Int.check(Schema.isGreaterThanOrEqualTo(0)) +// altimate_change start — float32-exact lower bound for the compaction +// safety fraction. +// +// `Schema.toArbitrary`'s fast-check generator requires `.check()` bounds to be +// exact 32-bit floats, and 0.1 is not float32-representable. Rounding UP to the +// nearest float32 (`Math.fround(0.1)` ≈ 0.10000000149) makes the schema reject +// the documented minimum `0.1`, so the bound has to round DOWN instead. This is +// the nearest float32 strictly below 0.1 (~1.3e-8 under), which satisfies the +// generator and still accepts the advertised lower endpoint. +export const SAFETY_FRACTION_MIN = Math.fround(0.1 - 1e-8) +// altimate_change end + /** * Relative file path (e.g., `src/components/Button.tsx`). */ diff --git a/packages/core/src/v1/config/config.ts b/packages/core/src/v1/config/config.ts index 41f64f9e10..8dbd2cbbc0 100644 --- a/packages/core/src/v1/config/config.ts +++ b/packages/core/src/v1/config/config.ts @@ -1,7 +1,7 @@ export * as ConfigV1 from "./config" import { Schema } from "effect" -import { NonNegativeInt, PositiveInt, type DeepMutable } from "../../schema" +import { NonNegativeInt, PositiveInt, SAFETY_FRACTION_MIN, type DeepMutable } from "../../schema" import { ConfigExperimental } from "../../config/experimental" import { ConfigReference } from "../../config/reference" import { ConfigAgentV1 } from "./agent" @@ -146,6 +146,12 @@ export const Info = Schema.Struct({ max_bytes: Schema.optional(PositiveInt).annotate({ description: "Maximum bytes of tool output before it is truncated and saved to disk (default: 51200)", }), + // altimate_change start — per-tool-result dispatch cap + dispatch_max_tokens: Schema.optional(PositiveInt).annotate({ + description: + "Hard cap on the estimated token size of a single tool result at dispatch time; oversized results are middle-truncated before entering the conversation (default: min(max_bytes-derived token estimate, 15% of the effective context limit))", + }), + // altimate_change end }), ).annotate({ description: @@ -169,6 +175,69 @@ export const Info = Schema.Struct({ reserved: Schema.optional(NonNegativeInt).annotate({ description: "Token buffer for compaction. Leaves enough window to avoid overflow during compaction.", }), + // altimate_change start — estimator safety margin + // upstream_fix: Schema.toArbitrary's fast-check generator requires + // `.check()` bounds to be exact 32-bit floats (fc.float's `min`/`max` + // constraints); 0.1 is not exactly float32-representable and made the + // property-based V1→V2 migration fuzz test below throw on every run. + // The bound must therefore be a float32 that is BELOW 0.1, not above it: + // Math.fround(0.1) is ~0.10000000149, so a config carrying the documented + // minimum `0.1` failed to decode. SAFETY_FRACTION_MIN is the nearest + // float32 under 0.1 (~1.3e-8 below), which keeps the generator happy and + // still accepts the advertised lower endpoint. + context_safety_fraction: Schema.optional( + Schema.Number.check(Schema.isGreaterThanOrEqualTo(SAFETY_FRACTION_MIN), Schema.isLessThanOrEqualTo(1)), + ).annotate({ + description: + "Fraction of the declared context limit treated as usable when estimated token counts are compared against it for compaction/overflow decisions (default: 0.65 — chars-based estimates can substantially undercount dense SQL/JSON, and compaction must trigger with enough margin that a worst-case underestimate still fits). Env override: ALTIMATE_CONTEXT_SAFETY_FRACTION. Clamped to [0.1, 1].", + }), + // altimate_change end + // altimate_change start — post-compaction state ledger + summary carry + state_ledger: Schema.optional(Schema.Boolean).annotate({ + description: + "Append a harness-computed state ledger (files written with timestamps, recent tool calls with exit codes) to the post-compaction continue message (default: true)", + }), + ledger_max_tokens: Schema.optional(NonNegativeInt).annotate({ + description: + "Token cap for the state ledger and carry anchors, tail-truncated (default: 500 — the ledger must cost less than the duplicate re-reads it prevents; one mid-size file re-read is ~1-3k tokens)", + }), + ledger_recent_calls: Schema.optional(NonNegativeInt).annotate({ + description: + "How many recent tool calls the state ledger lists, newest first (default: 10 — covers several typical edit-verify cycles without dominating the ledger budget)", + }), + summary_carry: Schema.optional(Schema.Boolean).annotate({ + description: + "Carry the previous summary's Accomplished items into the next summarization as anchors; items without a corroborating ledger event are tagged 'claimed, unverified' (default: true)", + }), + summary_first_person: Schema.optional(Schema.Boolean).annotate({ + description: + "Ask the compaction summarizer to write in the first person, as the agent's own working memory (default: true)", + }), + // altimate_change end + // altimate_change start — pin the original task verbatim through compaction + pin_task: Schema.optional(Schema.Boolean).annotate({ + description: + "Pin the original task instruction verbatim through compaction, hoisted as an authoritative reminder alongside the summary (default: true)", + }), + pin_max_tokens: Schema.optional(NonNegativeInt).annotate({ + description: + "Hard token cap for the pinned original task (default: 4096 — effective cap is min(4k, pin_window_fraction of the post-overhead usable window); larger tasks keep verbatim head+tail plus a contract card of extracted literals)", + }), + // altimate_change start — bound like the sibling context_safety_fraction: an + // out-of-range fraction (typo, negative, > 1) must be rejected at the config + // boundary rather than silently mis-sizing the pin. + pin_window_fraction: Schema.optional( + Schema.Number.check(Schema.isGreaterThanOrEqualTo(0), Schema.isLessThanOrEqualTo(1)), + ).annotate({ + description: + "Fraction of the post-overhead usable context window the pinned task may occupy (default: 0.175 — the pin must stay a small minority of the window so working context dominates). Clamped to [0, 1].", + }), + // altimate_change end + pin_card_max_tokens: Schema.optional(NonNegativeInt).annotate({ + description: + "Token cap for the contract card of regex-extracted task literals appended when the pinned task exceeds its cap (default: 500)", + }), + // altimate_change end }), ), // altimate_change start - tracing config (re-applied from main during the v1.17.9 reconciliation) @@ -234,6 +303,53 @@ export const Info = Schema.Struct({ "Auto-discover MCP servers from VS Code, Claude Code, Copilot, and Gemini configs at startup (default: true). Set to false to disable.", }), // altimate_change end + // altimate_change start — write-starvation circuit breaker + loop detection. + // Ships ANNOTATE-ONLY by default: mode "annotate" logs breaker-would-fire events + // and appends informational annotations; "armed" additionally injects outcome-neutral + // directives (run mode only) and enables the doom-loop escalation ladder's hard stop. + // Threshold defaults carry their rationale in session/starvation.ts DEFAULTS + // and are exposed here so they are never constants fitted to any one + // workload. + starvation_breaker: Schema.optional( + Schema.Struct({ + mode: Schema.optional(Schema.Literals(["off", "annotate", "armed"])).annotate({ + description: + "off = disabled; annotate (default) = log would-fire events and append informational annotations only; armed = also inject directives in run mode and enable the doom-loop hard stop.", + }), + max_turns_without_mutation: Schema.optional(PositiveInt).annotate({ + description: + "Consecutive assistant generation steps with zero corroborated file mutation before the write-starvation breaker fires (default: 12). Counted per model step, not per user message — one user turn routinely spans several read-only steps, so tune this against step counts (see session/starvation.ts).", + }), + repeat_signature_threshold: Schema.optional(PositiveInt).annotate({ + description: + "Consecutive identical repeat signatures (tool + normalized args + touched files + failure message) before the loop detector fires (default: 3).", + }), + doom_loop_threshold: Schema.optional(PositiveInt).annotate({ + description: + "Consecutive identical (tool + normalized args) calls before the escalation ladder's first rung (nudge). Rungs: threshold = nudge, 2x = forced status-check, 3x = stop (default: 3).", + }), + polling_threshold_multiplier: Schema.optional(PositiveInt).annotate({ + description: + "Multiplier applied to doom_loop_threshold for recognizable polling commands (default: 5).", + }), + polling_pattern: Schema.optional(Schema.String).annotate({ + description: + "Case-insensitive regex identifying polling-style bash commands whose repeat threshold is raised (default: \\b(sleep|watch|status)\\b).", + }), + exempt_agents: Schema.optional(Schema.mutable(Schema.Array(Schema.String))).annotate({ + description: + "Agent names for which the breaker is skipped entirely — read-only deliverables are their normal outcome (default: plan, review).", + }), + generated_path_patterns: Schema.optional(Schema.mutable(Schema.Array(Schema.String))).annotate({ + description: + "Path patterns exempt from unchanged-read annotation because they regenerate across builds (directory prefixes ending in '/', '*.ext' suffixes, or substrings).", + }), + }), + ).annotate({ + description: + "Write-starvation circuit breaker + loop detection (annotate-only by default; directives are run-mode-only).", + }), + // altimate_change end }), ), }).annotate({ identifier: "Config" }) diff --git a/packages/core/src/v1/config/migrate.ts b/packages/core/src/v1/config/migrate.ts index c474cac51a..fd99e49a72 100644 --- a/packages/core/src/v1/config/migrate.ts +++ b/packages/core/src/v1/config/migrate.ts @@ -57,8 +57,24 @@ export function migrate(info: typeof ConfigV1.Info.Type) { prune: info.compaction.prune, keep: { tokens: info.compaction.preserve_recent_tokens, + // altimate_change start — carry the verbatim-tail turn count (tail_turns: + // 0 disables the tail; dropping it silently restores the default). + turns: info.compaction.tail_turns, + // altimate_change end }, buffer: info.compaction.reserved, + // altimate_change start — carry the fork compaction keys (same names in V2) + context_safety_fraction: info.compaction.context_safety_fraction, + state_ledger: info.compaction.state_ledger, + ledger_max_tokens: info.compaction.ledger_max_tokens, + ledger_recent_calls: info.compaction.ledger_recent_calls, + summary_carry: info.compaction.summary_carry, + summary_first_person: info.compaction.summary_first_person, + pin_task: info.compaction.pin_task, + pin_max_tokens: info.compaction.pin_max_tokens, + pin_window_fraction: info.compaction.pin_window_fraction, + pin_card_max_tokens: info.compaction.pin_card_max_tokens, + // altimate_change end }, skills: info.skills && [...(info.skills.paths ?? []), ...(info.skills.urls ?? [])], commands: info.command, @@ -67,7 +83,15 @@ export function migrate(info: typeof ConfigV1.Info.Type) { plugins: info.plugin?.map((plugin) => typeof plugin === "string" ? plugin : { package: plugin[0], options: plugin[1] }, ), - experimental: info.experimental?.policies && { policies: info.experimental.policies }, + // altimate_change start — carry starvation_breaker alongside policies + experimental: + info.experimental?.policies || info.experimental?.starvation_breaker + ? { + policies: info.experimental?.policies, + starvation_breaker: info.experimental?.starvation_breaker, + } + : undefined, + // altimate_change end providers: providers(info.provider), } } diff --git a/packages/core/test/config/config.test.ts b/packages/core/test/config/config.test.ts index 6275d8fed3..14a0494cd8 100644 --- a/packages/core/test/config/config.test.ts +++ b/packages/core/test/config/config.test.ts @@ -87,6 +87,109 @@ describe("Config", () => { }), ) + // altimate_change start — V2 parity round-trip for the fork reliability keys + // (dispatch cap, compaction safety fraction, task pin, state ledger, + // starvation breaker). A V2 cutover must not silently drop these controls. + it.effect("round-trips the fork reliability keys through ConfigMigrateV1 into valid v2", () => + Effect.sync(() => { + const v1: typeof ConfigV1.Info.Type = { + tool_output: { max_lines: 100, max_bytes: 9_000, dispatch_max_tokens: 5_000 }, + compaction: { + auto: true, + reserved: 12_000, + tail_turns: 0, + preserve_recent_tokens: 3_000, + context_safety_fraction: 0.7, + state_ledger: true, + ledger_max_tokens: 400, + ledger_recent_calls: 8, + summary_carry: false, + summary_first_person: true, + pin_task: true, + pin_max_tokens: 2_048, + pin_window_fraction: 0.15, + pin_card_max_tokens: 300, + }, + experimental: { + starvation_breaker: { + mode: "armed", + max_turns_without_mutation: 10, + repeat_signature_threshold: 4, + doom_loop_threshold: 5, + polling_threshold_multiplier: 6, + polling_pattern: "\\b(sleep)\\b", + exempt_agents: ["plan"], + generated_path_patterns: ["dist/"], + }, + }, + } + const migrated = ConfigMigrateV1.migrate(v1) + const decoded = Schema.decodeUnknownSync(Config.Info)(migrated, { errors: "all" }) + expect(decoded.tool_output?.dispatch_max_tokens).toBe(5_000) + expect(decoded.compaction?.keep?.turns).toBe(0) + expect(decoded.compaction?.keep?.tokens).toBe(3_000) + expect(decoded.compaction?.context_safety_fraction).toBe(0.7) + expect(decoded.compaction?.state_ledger).toBe(true) + expect(decoded.compaction?.ledger_max_tokens).toBe(400) + expect(decoded.compaction?.ledger_recent_calls).toBe(8) + expect(decoded.compaction?.summary_carry).toBe(false) + expect(decoded.compaction?.summary_first_person).toBe(true) + expect(decoded.compaction?.pin_task).toBe(true) + expect(decoded.compaction?.pin_max_tokens).toBe(2_048) + expect(decoded.compaction?.pin_window_fraction).toBe(0.15) + expect(decoded.compaction?.pin_card_max_tokens).toBe(300) + expect(decoded.experimental?.starvation_breaker?.mode).toBe("armed") + expect(decoded.experimental?.starvation_breaker?.max_turns_without_mutation).toBe(10) + expect(decoded.experimental?.starvation_breaker?.repeat_signature_threshold).toBe(4) + expect(decoded.experimental?.starvation_breaker?.doom_loop_threshold).toBe(5) + expect(decoded.experimental?.starvation_breaker?.polling_threshold_multiplier).toBe(6) + expect(decoded.experimental?.starvation_breaker?.polling_pattern).toBe("\\b(sleep)\\b") + expect(decoded.experimental?.starvation_breaker?.exempt_agents).toEqual(["plan"]) + expect(decoded.experimental?.starvation_breaker?.generated_path_patterns).toEqual(["dist/"]) + }), + ) + // altimate_change end + + // altimate_change start — upstream_fix regression: a document authored + // directly in V2 (not migrated from V1) is decoded straight through + // ConfigV2.Compaction.Info, which previously had no bounds on these two + // fractions — only the V1 schema did. Assert the V2 schema rejects + // out-of-range values too. + it.effect("V2 compaction schema rejects out-of-range context_safety_fraction / pin_window_fraction", () => + Effect.sync(() => { + const decodeCompaction = (compaction: Record) => + Schema.decodeUnknownResult(Config.Info)({ compaction }) + + expect(decodeCompaction({ context_safety_fraction: 0.05 })._tag).toBe("Failure") + expect(decodeCompaction({ context_safety_fraction: 1.5 })._tag).toBe("Failure") + expect(decodeCompaction({ context_safety_fraction: 0.65 })._tag).toBe("Success") + + expect(decodeCompaction({ pin_window_fraction: -0.1 })._tag).toBe("Failure") + expect(decodeCompaction({ pin_window_fraction: 1.1 })._tag).toBe("Failure") + expect(decodeCompaction({ pin_window_fraction: 0.175 })._tag).toBe("Success") + }), + ) + + // The float32-exact bound the fast-check generator needs must round DOWN. + // Rounding up (Math.fround(0.1) ≈ 0.10000000149) made the schema reject the + // documented minimum and both endpoints of the advertised [0.1, 1] range. + it.effect("both endpoints of the documented context_safety_fraction range decode", () => + Effect.sync(() => { + const decodeCompaction = (compaction: Record) => + Schema.decodeUnknownResult(Config.Info)({ compaction }) + expect(decodeCompaction({ context_safety_fraction: 0.1 })._tag).toBe("Success") + expect(decodeCompaction({ context_safety_fraction: 1 })._tag).toBe("Success") + expect(decodeCompaction({ context_safety_fraction: 0.099 })._tag).toBe("Failure") + + const decodeV1 = (compaction: Record) => + Schema.decodeUnknownResult(ConfigV1.Info)({ compaction }) + expect(decodeV1({ context_safety_fraction: 0.1 })._tag).toBe("Success") + expect(decodeV1({ context_safety_fraction: 1 })._tag).toBe("Success") + expect(decodeV1({ context_safety_fraction: 0.099 })._tag).toBe("Failure") + }), + ) + // altimate_change end + it.effect("migrates v1 provider setup options into AISDK settings", () => Effect.sync(() => { const migrated = ConfigMigrateV1.migrate({ @@ -620,7 +723,7 @@ describe("Config", () => { expect(documents[0]?.info.compaction).toEqual({ auto: true, prune: undefined, - keep: { tokens: 2000 }, + keep: { tokens: 2000, turns: 3 }, buffer: 10000, }) expect(documents[0]?.info.mcp).toMatchObject({ diff --git a/packages/opencode/src/altimate/prompts/builder.txt b/packages/opencode/src/altimate/prompts/builder.txt index 5fab1f2e02..47ff6e5884 100644 --- a/packages/opencode/src/altimate/prompts/builder.txt +++ b/packages/opencode/src/altimate/prompts/builder.txt @@ -210,3 +210,21 @@ When you detect a correction: - training_save — Save a learned pattern, rule, glossary term, or standard - training_list — List all learned training entries with budget usage - training_remove — Remove outdated training entries + +## Finish Protocol (mandatory before ending any build/fix task) + +Trace analysis of failed sessions shows two dominant, avoidable +failure modes: finishing without the final build, and shipping models/columns +under self-chosen names instead of the task's literal contract. Before you +declare a task complete, ALWAYS: + +1. **Re-read the task's literal requirements** — exact model names, exact + column names, exact file paths. Diff them against what you actually wrote. + Your naming preferences never override the stated contract, even when your + names are "better". +2. **Run the final build and tests** with `altimate-dbt build` (no `--model` + flag) so the compiled manifest reflects every model you created or changed. + Work that exists only as an un-built SQL file does not count as done. +3. **If you are running low on turns or context**, stop exploring and commit: + write the change, build, verify. A completed adequate solution beats an + unfinished perfect one. diff --git a/packages/opencode/src/altimate/telemetry/index.ts b/packages/opencode/src/altimate/telemetry/index.ts index 45dce393fe..0528c0a825 100644 --- a/packages/opencode/src/altimate/telemetry/index.ts +++ b/packages/opencode/src/altimate/telemetry/index.ts @@ -207,6 +207,13 @@ export namespace Telemetry { trigger: "overflow_detection" | "error_recovery" attempt: number } + | { + type: "compaction_head_truncated" + timestamp: number + session_id: string + dropped_messages: number + kept_messages: number + } | { type: "tool_outputs_pruned" timestamp: number @@ -301,6 +308,24 @@ export namespace Telemetry { tool_name: string repeat_count: number } + // Write-starvation breaker + loop detection. In annotate mode every + // event is action "would_fire"/"annotated" ("would_annotate" is the + // interactive-session telemetry-only shadow — output left untouched); + // armed run-mode sessions also emit "registered" (directive handed to the + // nudge arbiter), "injected" (arbiter winner delivered to the model), and + // "stop" (escalation ladder hard stop). + | { + type: "starvation_breaker" + timestamp: number + session_id: string + mode: "annotate" | "armed" + kind: "starvation" | "repeat_signature" | "doom_loop" | "unchanged_read" | "nudge" + action: "would_fire" | "registered" | "injected" | "stop" | "annotated" | "would_annotate" + tool_name?: string + count?: number + escalation?: "nudge" | "status_check" | "stop" + turns_without_mutation?: number + } | { type: "environment_census" timestamp: number diff --git a/packages/opencode/src/cli/cmd/idle-done.ts b/packages/opencode/src/cli/cmd/idle-done.ts new file mode 100644 index 0000000000..5adb339ea4 --- /dev/null +++ b/packages/opencode/src/cli/cmd/idle-done.ts @@ -0,0 +1,356 @@ +// Fork-only helper for the `run` command — idle-done detection, the +// RUN-MODE-ONLY FALLBACK termination path. +// +// Explicit model DONE (SessionTermination) is the primary termination path. +// This module detects the completed-but-not-terminating churn signature — a session +// whose work is done (green verify AFTER the last file mutation) but that keeps +// cycling text-only post-compaction turns instead of ending — and arms a ONE-SHOT +// confirm-DONE challenge. It lives under cli/cmd and is wired only by run.ts, so +// TUI/serve behavior is untouched by construction (the +// interactive loop legitimately idles awaiting user input). +// +// HARD preconditions, all required before the challenge may fire: +// (i) build-after-last-write ordering FROM THE EVENT STREAM: the most recent +// verify-candidate bash command completed green (exit 0) at a stream +// position strictly AFTER the last observed file mutation. Mutations are +// write/edit tool completions AND snapshot `patch` parts — the patch part +// is the harness's ground truth for bash-mediated changes (`sed -i`, +// heredocs) that produce no edit event. "Last build green" alone certifies +// nothing about the current diff. +// (ii) GENERIC verify classification: the project-configured verify command +// (ALTIMATE_RUN_VERIFY_COMMAND) when set; otherwise the most recent +// side-effecting bash command (a conservative read-only-head classifier — +// NO vertical/product tokens). Classifier errs toward +// "read-only" so a trivial `ls`/`git status` can never count as a verify. +// (iii) suppressed while ANY tool call (incl. task-tool subagents) is still +// running or a permission request is pending. +// (iv) compaction-gated: at least `minCompactions` completed compaction cycles — +// idle-done can NEVER fire in a never-compacted session — plus +// `idleTurns` consecutive post-compaction text-only assistant turns. +// (v) one-shot: after the challenge is issued it can never re-arm (recursion +// guard — the challenge cannot breed further challenges). +// +// Threshold rationale (config-exposed, not fitted to any one workload): +// minCompactions=2 — one compaction can be a single oversized tool output; two +// completed cycles with no progress in between is the churn signature. +// idleTurns=3 — kept small because each candidate turn here already passed the +// much stronger green-verify-after-last-write precondition. + +export namespace IdleDone { + export interface Options { + /** Master switch — ALTIMATE_RUN_IDLE_DONE=0 disables the fallback entirely. */ + enabled: boolean + /** Minimum completed compaction cycles before the fallback may arm. */ + minCompactions: number + /** Consecutive post-compaction text-only turns required. */ + idleTurns: number + /** Optional project-configured verify command (prefix match on the bash command). */ + verifyCommand?: string + } + + export function optionsFromEnv(env: Record = process.env): Options { + const bound = (name: string, fallback: number) => { + const raw = env[name]?.trim() + if (!raw) return fallback + const parsed = Number(raw) + return Number.isFinite(parsed) && parsed >= 1 ? Math.floor(parsed) : fallback + } + const enabledRaw = env["ALTIMATE_RUN_IDLE_DONE"]?.trim().toLowerCase() + return { + enabled: enabledRaw !== "0" && enabledRaw !== "false", + minCompactions: bound("ALTIMATE_IDLE_DONE_MIN_COMPACTIONS", 2), + idleTurns: bound("ALTIMATE_IDLE_DONE_IDLE_TURNS", 3), + verifyCommand: env["ALTIMATE_RUN_VERIFY_COMMAND"]?.trim() || undefined, + } + } + + /** + * Arming gate for the run command. The fallback may arm ONLY for a local + * (non-attach) run with run mode active: `--attach` targets a remote, + * possibly shared/interactive server session where aborting the in-flight + * prompt is never acceptable, and an explicit `ALTIMATE_RUN_MODE=0` is the + * documented opt-out for every run-mode-only mechanism (see run/run-mode.ts). + */ + export function armedOptions(options: Options, gate: { attach: boolean; runMode: boolean }): Options { + return { ...options, enabled: options.enabled && !gate.attach && gate.runMode } + } + + // ── Generic bash classifier ─────────────────────────────────────────────── + // Conservative read-only-head allowlist. Direction of safety: a read-only + // command misclassified as side-effecting could count as a green "verify", so + // the allowlist is GREEDY — when in doubt a command is read-only and therefore + // NOT a verify candidate (idle-done then simply never fires). Generic shell + // vocabulary only — no vertical/product tokens. + const READ_ONLY_HEADS = new Set([ + "ls", + "cat", + "head", + "tail", + "less", + "more", + "wc", + "pwd", + "cd", + "echo", + "printf", + "which", + "whereis", + "whoami", + "date", + "env", + "printenv", + "stat", + "file", + "du", + "df", + "tree", + "find", + "grep", + "rg", + "egrep", + "fgrep", + "awk", + "sed", + "cut", + "sort", + "uniq", + "diff", + "cmp", + "md5", + "md5sum", + "shasum", + "sha256sum", + "basename", + "dirname", + "realpath", + "readlink", + "type", + "true", + "false", + "test", + "[", + "sleep", + ]) + const GIT_READ_ONLY_SUBCOMMANDS = new Set([ + "status", + "log", + "diff", + "show", + "branch", + "remote", + "rev-parse", + "ls-files", + "blame", + "describe", + "shortlog", + "config", + ]) + + /** True when every pipeline/statement head in the command is read-only. */ + export function isReadOnlyCommand(command: string): boolean { + const statements = command + .split(/&&|\|\||[;|\n]/) + .map((s) => s.trim()) + .filter((s) => s.length > 0) + if (statements.length === 0) return true + for (const statement of statements) { + // Skip leading VAR=value assignments and common wrappers. + const tokens = statement.split(/\s+/).filter((t) => !/^[A-Za-z_][A-Za-z0-9_]*=/.test(t)) + const head = tokens[0]?.replace(/^\(+/, "") + if (!head) continue + if (head === "git") { + const sub = tokens[1] + if (!sub || !GIT_READ_ONLY_SUBCOMMANDS.has(sub)) return false + continue + } + if (!READ_ONLY_HEADS.has(head)) return false + } + return true + } + + // Heads that always write, and in-place/redirection forms whose head alone + // looks read-only (`sed -i file`, `cat a > b`, `... | tee out`). + // + // Snapshot patch parts normally report bash-mediated writes, but snapshots + // are configurable (`snapshot: false`) and produce no patch part when off. A + // write that goes unrecorded leaves the mutation watermark stale, so an + // EARLIER green verification still satisfies the build-after-last-write + // precondition and idle-done can claim nothing happened after it. Classifying + // by command head alone is what misses these. + const MUTATING_HEADS = new Set([ + "rm", + "mv", + "cp", + "mkdir", + "rmdir", + "touch", + "ln", + "install", + "chmod", + "chown", + "truncate", + "dd", + "tee", + ]) + + /** True when the command writes to the filesystem through a head, flag, or redirection. */ + export function isMutatingCommand(command: string): boolean { + // Output redirection to a file. Excludes fd duplication (`2>&1`, `>&2`). + if (/(?>?\s*(?![&|])/.test(command)) return true + // In-place editors: the head is on the read-only list, the `-i` flag writes. + if (/\b(?:sed|perl|ruby)\b[^|;&]*\s-[A-Za-z]*i\b/.test(command)) return true + for (const statement of command.split(/&&|\|\||[;|\n]/)) { + const tokens = statement + .trim() + .split(/\s+/) + .filter((t) => !/^[A-Za-z_][A-Za-z0-9_]*=/.test(t)) + const head = tokens[0]?.replace(/^\(+/, "") + if (head && MUTATING_HEADS.has(head)) return true + } + return false + } + + // Mutation-classified tool names: the harness's own file-writing tools. Patch + // parts (snapshot diffs) additionally catch bash-mediated mutations. + const MUTATING_TOOLS = new Set(["write", "edit", "multiedit", "patch"]) + + export interface Deps { + /** From RunAccounting — resolves whether a message belongs to compaction machinery. */ + isCompactionStep(messageID: string): boolean + } + + // Minimal structural slice of the SDK part event this module consumes. + export interface PartSlice { + id: string + messageID: string + type: string + tool?: string + state?: { + status?: string + input?: Record + metadata?: Record + } + reason?: string + } + + export function create(options: Options, deps: Deps) { + // Monotonic event-stream position; every observed part advances it, so + // "after" comparisons reflect stream order, not wall clock. + let seq = 0 + let lastMutationSeq = -1 + let lastVerifySeq = -1 + let lastVerifyGreen = false + const runningToolParts = new Set() + const pendingPermissions = new Set() + const compactionsCompleted = new Set() + // Tool/patch activity per assistant message, to classify text-only turns. + const messageHadActivity = new Set() + let consecutiveIdleTurns = 0 + let challengeIssued = false + + function observeBash(part: PartSlice) { + const command = typeof part.state?.input?.["command"] === "string" ? (part.state.input["command"] as string) : "" + const isCandidate = options.verifyCommand + ? command.trimStart().startsWith(options.verifyCommand) + : !isReadOnlyCommand(command) + if (isCandidate) { + const exit = part.state?.metadata?.["exit"] + lastVerifySeq = seq + lastVerifyGreen = exit === 0 + return + } + // Not a verification. If it still wrote, advance the mutation watermark — + // otherwise a stale earlier verify keeps satisfying precondition (i) even + // though the session changed files after it. Checked after the candidate + // test so a configured verify command that redirects its own output + // (`make test > log`) is still counted as the verification it is. + if (isMutatingCommand(command)) lastMutationSeq = seq + } + + return { + /** Feed every message.part.updated event for the session through this. */ + observePart(part: PartSlice) { + seq++ + if (part.type === "patch") { + // Snapshot diff: files changed somewhere in this step (ground truth, + // includes bash-mediated writes). Ordering within the step is unknown, + // so the patch — emitted at step end — conservatively postdates any + // verify that ran inside the same step. + lastMutationSeq = seq + messageHadActivity.add(part.messageID) + return + } + if (part.type === "tool") { + const status = part.state?.status + if (status === "running") { + runningToolParts.add(part.id) + return + } + if (status !== "completed" && status !== "error") return + runningToolParts.delete(part.id) + messageHadActivity.add(part.messageID) + if (status !== "completed") return + if (part.tool && MUTATING_TOOLS.has(part.tool)) lastMutationSeq = seq + if (part.tool === "bash") observeBash(part) + return + } + if (part.type === "step-finish") { + if (deps.isCompactionStep(part.messageID)) { + compactionsCompleted.add(part.messageID) + // A fresh compaction cycle: idle turns are counted per cycle. + consecutiveIdleTurns = 0 + return + } + if (part.reason === "stop" && !messageHadActivity.has(part.messageID)) { + consecutiveIdleTurns++ + } else { + consecutiveIdleTurns = 0 + } + } + }, + onPermissionAsked(requestID: string) { + pendingPermissions.add(requestID) + }, + onPermissionResolved(requestID: string) { + pendingPermissions.delete(requestID) + }, + /** All hard preconditions (i)–(v). Evaluate after each observed step-finish. */ + shouldChallenge(): boolean { + if (!options.enabled) return false + if (challengeIssued) return false // (v) one-shot recursion guard + if (compactionsCompleted.size < options.minCompactions) return false // (iv) + if (consecutiveIdleTurns < options.idleTurns) return false // (iv) + if (runningToolParts.size > 0) return false // (iii) + if (pendingPermissions.size > 0) return false // (iii) + if (!lastVerifyGreen) return false // (i)/(ii) + // altimate_change start — upstream_fix: lastMutationSeq starts at -1, so a + // run that never mutated a file (pure read/explore, or a session that + // only ever verified) satisfied "verify after last write" vacuously — + // there was no completed work for the green verify to actually confirm. + if (lastMutationSeq < 0) return false // (i) at least one mutation must exist + if (lastVerifySeq <= lastMutationSeq) return false // (i) build-after-last-write + // altimate_change end + return true + }, + markChallengeIssued() { + challengeIssued = true + }, + get challengeIssued() { + return challengeIssued + }, + /** Introspection for logs/telemetry when the challenge fires. */ + snapshot() { + return { + compactions: compactionsCompleted.size, + idle_turns: consecutiveIdleTurns, + last_mutation_seq: lastMutationSeq, + last_verify_seq: lastVerifySeq, + last_verify_green: lastVerifyGreen, + running_tools: runningToolParts.size, + pending_permissions: pendingPermissions.size, + } + }, + } + } + export type Info = ReturnType +} diff --git a/packages/opencode/src/cli/cmd/run-accounting.ts b/packages/opencode/src/cli/cmd/run-accounting.ts new file mode 100644 index 0000000000..cf586be28a --- /dev/null +++ b/packages/opencode/src/cli/cmd/run-accounting.ts @@ -0,0 +1,287 @@ +// Fork-only helpers for the `run` command: +// - honest turn accounting: compaction-machinery steps must not consume the +// --max-turns budget. `step-start` parts carry only messageID/sessionID, so +// the owning message's agent is resolved via a lookup populated from +// `message.updated` events (the assistant message row is persisted — and its +// event published — before its first step-start part streams). +// - dual-attribution termination logging: every run records TWO independent +// fields instead of one rc: `why_model_stopped` and `why_harness_stopped`, +// so model-looping, tight budgets, and harness errors stop being conflated +// into a single exit code (SWE-agent #1262 vs OpenHands #9344 needed +// different fixes and were indistinguishable under rc-only accounting). +// - real error serialization: never a bare name, "[object Object]", or a +// literal `{}` — automation needs the actual name/message/status. +// - done_reason emission (explicit_done vs idle_heuristic vs none) and the +// idle-done challenge bookkeeping; DONE detection delegates to the +// SessionTermination completion-token contract. +import { SessionTermination } from "../../session/termination" + +export namespace RunAccounting { + export type WhyModelStopped = "stop" | "tool-call" | "explicit-done" + export type WhyHarnessStopped = "budget-exhausted" | "timeout" | "error" | "idle-done" | "none" + // done_reason distinguishes an unprompted completion assertion + // (explicit_done — the PRIMARY termination path) from one elicited by the + // idle-done confirm challenge (idle_heuristic). "none" = the session ended + // without any completion assertion — bare finishReason "stop" is NEVER + // reported as done. + export type DoneReason = "explicit_done" | "idle_heuristic" | "none" + export type Termination = { + why_model_stopped: WhyModelStopped + why_harness_stopped: WhyHarnessStopped + done_reason: DoneReason + } + + // Recoverable by design: auto-compaction handles context overflow and the session + // continues, so an overflow error event alone must not flip the run's rc or its + // harness-stop attribution. + const RECOVERABLE_ERROR_NAMES = new Set(["ContextOverflowError"]) + + // Timeout classification for why_harness_stopped="timeout" and retry decisions. + const TIMEOUT_PATTERN = /\btimed?\s*out\b|\bETIMEDOUT\b|TimeoutError/i + + // the explicit model DONE assertion is the primary termination path. + // Detection delegates to the SessionTermination completion-token contract — + // the single detector shared with the processor stop-path and the idle-done + // challenge, so instruction and detection can never drift apart. + + export function create() { + const agents = new Map() + let turnCount = 0 + let lastFinishReason: string | undefined + // altimate_change start — upstream_fix: onText and onStepFinish are + // independently overwritten by whichever message last emitted a text/finish + // event. A DONE-bearing message (finish="tool-calls") followed by a + // textless message (finish="stop") left `lastTextExplicitDone` stale from + // the FIRST message paired with `lastFinishReason` from the SECOND — cross- + // message state, not one message's actual outcome. Track whose message each + // came from and only trust the pairing when they agree. + let lastFinishMessageID: string | undefined + let lastTextMessageID: string | undefined + // altimate_change end + let lastTextExplicitDone = false + let budgetExhausted = false + let fatalError: { name: string; timeout: boolean } | undefined + // set when the run-mode idle-done fallback issued its one-shot + // confirm-DONE challenge (see cli/cmd/idle-done.ts). Scoped to the + // challenge GENERATION, not the run lifetime: the turn at issuance is + // recorded so only a DONE in the immediately-following generation is + // attributed to the heuristic — a later unprompted DONE (after the model + // declined the challenge and kept working) is honest explicit_done. + let idleDoneChallengeTurn: number | undefined + let lastExplicitDoneTurn: number | undefined + // the harness delivers the challenge by aborting ONE in-flight prompt; + // each suppression may fire at most once — later aborts/abnormal + // finishes are real failures. + let challengeAbortSuppressed = false + let challengeFinishSuppressed = false + // altimate_change start — upstream_fix: the abort of the interrupted prompt + // can surface as onSessionError(MessageAbortedError), onPromptResult + // (finish="error"/"other"), or both — either channel may fire for that + // SAME abort, so both suppressions above are scoped to it. Once the + // challenge reply itself is sent, a real failure there (e.g. an errorless + // finish="other" on the confirm-DONE reply) must not be silently forgiven + // by whichever suppression the interrupted prompt's abort left unused. + let challengeReplySent = false + // altimate_change end + + function isCompactionStep(messageID: string) { + return agents.get(messageID) === "compaction" + } + + return { + /** Record an assistant message's agent so later part events can be attributed. */ + onAssistantMessage(info: { id: string; agent?: string }) { + agents.set(info.id, info.agent ?? "") + }, + isCompactionStep, + /** + * Count a step-start toward the turn budget unless it belongs to a + * compaction-machinery message. Returns true when the step was counted. + */ + onStepStart(messageID: string): boolean { + if (isCompactionStep(messageID)) return false + turnCount++ + return true + }, + get turnCount() { + return turnCount + }, + onStepFinish(messageID: string, reason: string | undefined) { + if (isCompactionStep(messageID)) return + lastFinishReason = reason + lastFinishMessageID = messageID + }, + onText(messageID: string, text: string) { + if (isCompactionStep(messageID)) return + lastTextExplicitDone = SessionTermination.isExplicitDone(text) + lastTextMessageID = messageID + lastExplicitDoneTurn = lastTextExplicitDone ? turnCount : undefined + }, + /** the idle-done fallback issued its one-shot confirm-DONE challenge. */ + onIdleDoneChallengeIssued() { + idleDoneChallengeTurn = turnCount + }, + // altimate_change start — upstream_fix: see challengeReplySent above. + /** the idle-done confirm-DONE challenge reply has been sent; suppression of the interrupted prompt's own abort no longer applies. */ + onIdleDoneChallengeReplySent() { + challengeReplySent = true + }, + // altimate_change end + onSessionError(name: unknown, message?: string) { + const errorName = typeof name === "string" && name.length > 0 ? name : "UnknownError" + if (RECOVERABLE_ERROR_NAMES.has(errorName)) return + // the idle-done challenge is delivered by aborting the in-flight + // prompt first; that harness-initiated abort surfaces as a + // MessageAbortedError and must not be scored as a fatal run error. + // Exactly ONE such abort exists per challenge — later aborts are real. + if ( + idleDoneChallengeTurn !== undefined && + !challengeAbortSuppressed && + !challengeReplySent && + errorName === "MessageAbortedError" + ) { + challengeAbortSuppressed = true + return + } + fatalError = { + name: errorName, + timeout: TIMEOUT_PATTERN.test(errorName) || TIMEOUT_PATTERN.test(message ?? ""), + } + }, + onBudgetExhausted() { + budgetExhausted = true + }, + /** + * Inspect the prompt call's returned terminal assistant message. Transport + * failures can be swallowed upstream into a clean-looking idle (observed: a + * mid-stream provider error surfaces ONLY as finish="other" with no error + * field and no session.error event), so the terminal message is the last + * honest signal available. finish="error"/"other" are the AI SDK's abnormal + * terminations; "stop"/"length"/"tool-calls"/"content-filter"/"unknown" are + * not treated as fatal. + */ + onPromptResult(info: { finish?: string; error?: { name?: unknown; data?: unknown } } | undefined) { + if (!info) return + if (info.error) { + const data = (info.error.data ?? {}) as Record + this.onSessionError(info.error.name, typeof data.message === "string" ? data.message : undefined) + return + } + if (info.finish === "error" || info.finish === "other") { + // the terminal message of the ONE prompt the idle-done fallback + // aborted (to deliver its challenge) finishes abnormally by design; + // any further abnormal finish is a real failure. + if (idleDoneChallengeTurn !== undefined && !challengeFinishSuppressed && !challengeReplySent) { + challengeFinishSuppressed = true + return + } + fatalError ??= { name: `AbnormalFinish:${info.finish}`, timeout: false } + } + }, + /** True when the run ended by fatal abort — the process must exit nonzero. */ + get fatal() { + return budgetExhausted || fatalError !== undefined + }, + /** Dual-attribution fields + done_reason for the run record/output. */ + termination(): Termination { + // altimate_change start — upstream_fix: only trust the DONE text when it + // came from the SAME message as the finish reason being paired with it — + // see the field comment above. + const explicitDoneOnFinishMessage = + lastTextExplicitDone && lastTextMessageID !== undefined && lastTextMessageID === lastFinishMessageID + // altimate_change end + const model: WhyModelStopped = (() => { + if (lastFinishReason === "stop" && explicitDoneOnFinishMessage) return "explicit-done" + if (lastFinishReason === "tool-calls" || lastFinishReason === "tool-call") return "tool-call" + return "stop" + })() + // A completion assertion requires finishReason "stop" PLUS + // the explicit DONE token — never bare "stop". If the assertion followed + // the idle-done confirm challenge, it is honestly attributed to the + // heuristic, not to unprompted model completion. + const done: DoneReason = (() => { + if (lastFinishReason !== "stop" || !explicitDoneOnFinishMessage) return "none" + // idle_heuristic only when the DONE landed in the challenge's own + // generation (the turn it interrupted, or the reply turn right after). + const challengeScoped = + idleDoneChallengeTurn !== undefined && + lastExplicitDoneTurn !== undefined && + lastExplicitDoneTurn <= idleDoneChallengeTurn + 1 + return challengeScoped ? "idle_heuristic" : "explicit_done" + })() + const harness: WhyHarnessStopped = (() => { + if (budgetExhausted) return "budget-exhausted" + if (fatalError?.timeout) return "timeout" + if (fatalError) return "error" + // the session ended on (or after) the idle-done challenge. + if (done === "idle_heuristic") return "idle-done" + // A session that idles because the model finished is attributed to the + // model, so the harness reason is "none". + return "none" + })() + return { why_model_stopped: model, why_harness_stopped: harness, done_reason: done } + }, + } + } + export type Info = ReturnType + + /** + * Serialize a session error event's payload to a real name/message/status string. + * Never returns a bare "[object Object]" or a literal "{}". + */ + export function serializeSessionError(error: unknown): string { + if (error === undefined || error === null) return "UnknownError" + if (typeof error !== "object") return String(error) + const obj = error as { name?: unknown; message?: unknown; data?: unknown } + const name = typeof obj.name === "string" && obj.name.length > 0 ? obj.name : "UnknownError" + const data = (obj.data && typeof obj.data === "object" ? obj.data : {}) as Record + const status = + typeof data.status === "number" || (typeof data.status === "string" && data.status.length > 0) + ? data.status + : typeof data.statusCode === "number" + ? data.statusCode + : undefined + // altimate_change start — upstream_fix: fall back to the top-level `message` + // (native `Error.message`, e.g. thrown network/transport failures) when the + // nested `data.message` the server-error shape uses is absent — otherwise a + // thrown Error serialized to the bare string "Error" loses its message. + const message = + typeof data.message === "string" && data.message.length > 0 + ? data.message + : typeof obj.message === "string" && obj.message.length > 0 + ? obj.message + : data.message !== undefined + ? JSON.stringify(data.message) + : undefined + // altimate_change end + const head = status !== undefined ? `${name} (status ${status})` : name + return message ? `${head}: ${message}` : head + } + + /** Provider 5xx responses are retryable at the enqueue boundary. */ + export function isRetryableStatus(status: unknown): boolean { + return typeof status === "number" && status >= 500 && status <= 599 + } + + /** setTimeout's signed 32-bit ceiling; a larger delay is clamped by the runtime to ~1ms. */ + export const MAX_TIMER_MS = 2_147_483_647 + + /** + * Exponential backoff clamped to the timer range. Bounding the retry count and + * the base delay separately is NOT enough: at the accepted maximums the + * compounded delay (base * 2**attempt) runs far past MAX_TIMER_MS, and an + * overflowing timeout fires almost immediately — turning the backoff into the + * tight retry loop the bounds exist to prevent. + */ + export function retryDelayMs(baseMs: number, attempt: number): number { + return Math.min(baseMs * 2 ** attempt, MAX_TIMER_MS) + } + + /** Thrown transport failures that warrant an enqueue retry: timeouts and dropped connections. */ + export function isRetryableThrown(error: unknown): boolean { + if (error === undefined || error === null) return false + const err = error as { name?: unknown; message?: unknown; code?: unknown } + const text = [err.name, err.message, err.code].filter((v) => typeof v === "string").join(" ") + return TIMEOUT_PATTERN.test(text) || /ECONNRESET|ECONNREFUSED|fetch failed|network error/i.test(text) + } +} diff --git a/packages/opencode/src/cli/cmd/run.ts b/packages/opencode/src/cli/cmd/run.ts index 50638bcd14..cb84290ec7 100644 --- a/packages/opencode/src/cli/cmd/run.ts +++ b/packages/opencode/src/cli/cmd/run.ts @@ -28,6 +28,20 @@ import { BashTool } from "../../tool/bash" import { TodoWriteTool } from "../../tool/todo" import { Locale } from "../../util/locale" import { Tracer, FileExporter, HttpExporter, type TraceExporter } from "../../altimate/observability/tracing" +// altimate_change start — run accounting helpers (fork-only module) +import { RunAccounting } from "./run-accounting" +// altimate_change end +// altimate_change start — run implies run mode (fork-only module) +import { applyRunModeDefault } from "./run/run-mode" +// altimate_change end +// altimate_change start — run-mode-only idle-done fallback (fork-only modules). +// Detection lives in idle-done.ts; the confirm-DONE challenge text and the DONE +// token contract live in session/termination.ts; delivery goes through the nudge +// arbiter (at most one system-authored directive block per injected turn). +import { IdleDone } from "./idle-done" +import { NudgeArbiter } from "../../session/nudge" +import { SessionTermination } from "../../session/termination" +// altimate_change end // altimate_change start — upstream_fix: type-only import for the tracing-config cast (see tracer setup below) import type { ConfigV1 } from "@opencode-ai/core/v1/config/config" // altimate_change end @@ -400,6 +414,16 @@ export const RunCommand = cmd({ process.env["ALTIMATE_NON_INTERACTIVE"] = "1" } // altimate_change end + // altimate_change start — mark this process as run mode so + // run-mode-only mechanisms (DONE-termination gate, starvation-breaker + // directives, doom-loop escalation ladder) arm in the in-process session. + // Explicit ALTIMATE_RUN_MODE=0 opts out; --attach skips entirely (the agent + // runs on the remote, possibly interactive, server). See run/run-mode.ts. + applyRunModeDefault(process.env, { + attach: Boolean(args.attach), + resumed: Boolean(args.continue || args.session), + }) + // altimate_change end let message = [...args.message, ...(args["--"] || [])] .map((arg) => (arg.includes(" ") ? `"${arg.replace(/"/g, '\\"')}"` : arg)) @@ -595,6 +619,26 @@ You are speaking to a non-technical business executive. Follow these rules stric const events = await sdk.event.subscribe() let error: string | undefined + // altimate_change start — turn accounting + dual-attribution + // termination state for this run (see run-accounting.ts). + const accounting = RunAccounting.create() + // altimate_change end + // altimate_change start — idle-done fallback state (run-mode-only by + // construction — this exists only in the run command). Thresholds are + // config-exposed via env with first-principles provenance (see idle-done.ts). + // Armed ONLY for a local run with run mode active: --attach targets a + // remote, possibly shared/interactive session, and ALTIMATE_RUN_MODE=0 is + // the documented opt-out for every run-mode-only mechanism. + const idleDone = IdleDone.create( + IdleDone.armedOptions(IdleDone.optionsFromEnv(), { + attach: Boolean(args.attach), + runMode: Flag.ALTIMATE_RUN_MODE, + }), + { + isCompactionStep: (messageID) => accounting.isCompactionStep(messageID), + }, + ) + // altimate_change end // Build tracer from config + CLI flags — must never crash the run command const tracer = await (async () => { @@ -628,14 +672,36 @@ You are speaking to a non-technical business executive. Follow these rules stric } })() - async function loop() { + // altimate_change start — the event loop takes its stream as a + // parameter so the idle-done challenge phase can re-run it over a fresh + // subscription after the deliberate mid-run abort (same accounting, same + // max-turns budget — the challenge continuation stays budget-enforced). + // requireBusyFirst: the challenge-phase loop ignores idle events until the + // challenge turn has actually started (a straggler idle from the abort + // would otherwise end the phase before the challenge prompt begins). + async function loop(stream: typeof events.stream, options?: { requireBusyFirst?: boolean }) { + let sawBusy = false + // altimate_change end const toggles = new Map() - // altimate_change start — max-turns budget enforcement - let turnCount = 0 + // altimate_change start — max-turns budget enforcement (count kept in accounting) const maxTurns = args.maxTurns // altimate_change end - for await (const event of events.stream) { + // altimate_change start — parameterized stream + for await (const event of stream) { + // altimate_change end + // altimate_change start — record each assistant message's agent so + // step-start parts (which carry only messageID/sessionID) can be attributed. + // The assistant message row is persisted — and this event published — before + // its first step-start part streams, so the lookup is populated in time. + if ( + event.type === "message.updated" && + event.properties.info.role === "assistant" && + event.properties.info.sessionID === sessionID + ) { + accounting.onAssistantMessage(event.properties.info) + } + // altimate_change end if ( event.type === "message.updated" && event.properties.info.role === "assistant" && @@ -661,6 +727,12 @@ You are speaking to a non-technical business executive. Follow these rules stric const part = event.properties.part if (part.sessionID !== sessionID) continue + // altimate_change start — feed every part event through the + // idle-done observer (event-stream ordering for build-after-last-write, + // text-only-turn counting, outstanding-tool suppression). + idleDone.observePart(part as unknown as IdleDone.PartSlice) + // altimate_change end + if (part.type === "tool" && (part.state.status === "completed" || part.state.status === "error")) { tracer?.logToolCall(part as Parameters[0]) if (emit("tool_use", { part })) continue @@ -689,8 +761,12 @@ You are speaking to a non-technical business executive. Follow these rules stric if (part.type === "step-start") { tracer?.logStepStart(part) // altimate_change start — enforce max-turns budget - turnCount++ - if (maxTurns && turnCount > maxTurns) { + // compaction-machinery steps are excluded from turn accounting — + // the owning message's agent is resolved via the message.updated lookup + // above, so compacting models are not differentially charged turns. + const counted = accounting.onStepStart(part.messageID) + if (counted && maxTurns && accounting.turnCount > maxTurns) { + accounting.onBudgetExhausted() error = `Budget exceeded: reached ${maxTurns} assistant turn${maxTurns !== 1 ? "s" : ""} limit` UI.println(UI.Style.TEXT_DANGER_BOLD + "!", UI.Style.TEXT_NORMAL + ` ${error}. Aborting session.`) await sdk.session.abort({ sessionID }) @@ -702,11 +778,38 @@ You are speaking to a non-technical business executive. Follow these rules stric if (part.type === "step-finish") { tracer?.logStepFinish(part) + // altimate_change start — record the model-side finish reason + accounting.onStepFinish(part.messageID, (part as { reason?: string }).reason) + // altimate_change end + // altimate_change start — idle-done fallback firing point. + // All hard preconditions are checked in idle-done.ts (compaction-gated, + // build-after-last-write green verify, no outstanding tools/permissions, + // one-shot). Firing aborts the churning prompt and hands off to the + // confirm-DONE challenge phase after the event loop drains. Checked + // BEFORE the json-mode emit-continue so headless drivers take this path too. + if (idleDone.shouldChallenge()) { + idleDone.markChallengeIssued() + accounting.onIdleDoneChallengeIssued() + const detail = idleDone.snapshot() + if (!emit("idle_done_challenge", { detail })) { + UI.println( + UI.Style.TEXT_WARNING_BOLD + "!", + UI.Style.TEXT_NORMAL + + ` idle-done: completion signature detected (green verify after last write, ${detail.idle_turns} idle turns, ${detail.compactions} compactions) — issuing one-shot confirm-DONE challenge`, + ) + } + await sdk.session.abort({ sessionID }) + break + } + // altimate_change end if (emit("step_finish", { part })) continue } if (part.type === "text" && part.time?.end) { tracer?.logText(part) + // altimate_change start — explicit-done attribution input + accounting.onText(part.messageID, part.text) + // altimate_change end if (emit("text", { part })) continue const text = part.text.trim() if (!text) continue @@ -738,26 +841,54 @@ You are speaking to a non-technical business executive. Follow these rules stric if (event.type === "session.error") { const props = event.properties if (props.sessionID !== sessionID || !props.error) continue - let err = String(props.error.name) - if ("data" in props.error && props.error.data && "message" in props.error.data) { - err = String(props.error.data.message) - } + // altimate_change start — the idle-done challenge is delivered + // by aborting the in-flight prompt; that harness-initiated abort is not + // a run error — don't display it or fold it into the error record. + if (idleDone.challengeIssued && props.error.name === "MessageAbortedError") continue + // altimate_change end + // altimate_change start — serialize the real error name/message/status + // (never a bare name, "[object Object]", or a literal {}); feed the + // harness-stop attribution (recoverable overflow errors are excluded there). + const err = RunAccounting.serializeSessionError(props.error) + accounting.onSessionError( + props.error.name, + "data" in props.error && props.error.data && "message" in props.error.data + ? String(props.error.data.message) + : undefined, + ) + // altimate_change end error = error ? error + EOL + err : err if (emit("error", { error: props.error })) continue UI.error(err) } + // altimate_change start — track busy for the challenge-phase guard + if ( + event.type === "session.status" && + event.properties.sessionID === sessionID && + event.properties.status.type === "busy" + ) { + sawBusy = true + } + // altimate_change end if ( event.type === "session.status" && event.properties.sessionID === sessionID && event.properties.status.type === "idle" ) { + // altimate_change start — ignore stale pre-challenge idles + if (options?.requireBusyFirst && !sawBusy) continue + // altimate_change end break } if (event.type === "permission.asked") { const permission = event.properties if (permission.sessionID !== sessionID) continue + // altimate_change start — idle-done is suppressed while a + // permission request is outstanding (hard precondition iii). + idleDone.onPermissionAsked(permission.id) + // altimate_change end // altimate_change start - yolo mode: auto-approve but respect explicit deny rules. // --dangerously-skip-permissions (backport of upstream PR #21266) is treated as // an alias — same auto-approve behavior, plus our deny-rule safety net which @@ -807,6 +938,9 @@ You are speaking to a non-technical business executive. Follow these rules stric }) } // altimate_change end + // altimate_change start — every branch above replied; clear the pending flag + idleDone.onPermissionResolved(permission.id) + // altimate_change end } } } @@ -867,31 +1001,68 @@ You are speaking to a non-technical business executive. Follow these rules stric tracer?.flushSync("Process interrupted") process.exit(143) } + // altimate_change start — honest rc on fatal abort. beforeExit firing + // before the run finishes means the event loop drained before the run + // completed — the prompt/event stream was abandoned (observed: a + // mid-stream provider failure tears everything down and the process used + // to die here with rc 0). The flag (not just listener removal) makes the + // outcome sticky in the right direction: a spurious firing during an + // event-loop gap on a run that later completes must not poison the rc — + // the success path sets runFinished and restores exitCode explicitly. + let runFinished = false const onBeforeExit = () => { tracer?.flushSync("Process exited") + if (!runFinished) process.exitCode = 1 } + // altimate_change end process.on("SIGINT", onSigint) process.on("SIGTERM", onSigterm) process.on("beforeExit", onBeforeExit) // Start event listener before sending the prompt so no events are missed - const loopPromise = loop().catch((e) => { + // altimate_change start — pass the stream explicitly (see loop signature) + const loopPromise = loop(events.stream).catch((e) => { + // altimate_change end console.error(e) process.exit(1) }) - if (args.command) { - await sdk.session.command({ - sessionID, - agent, - model: args.model, - command: args.command, - arguments: message, - variant: args.variant, - }) - } else { + // altimate_change start — bounded retry-with-backoff on provider 5xx/timeout + // at the enqueue boundary. Bounds are config-exposed via env (provenance: + // bounded retries with every retry logged so they can + // never mask a persistent provider failure; defaults mirror the in-stream + // SessionRetry posture — bounded and visible). On exhaustion the error is thrown + // so the process exits nonzero instead of hanging on an idle event that will + // never arrive. + // altimate_change start — upstream_fix: cap the upper bound too, not just + // reject non-finite/negative — an unbounded ALTIMATE_RUN_RETRY_MAX permits + // runaway retries, and an unbounded base delay compounds through + // `retryBaseMs * 2 ** attempt` past setTimeout's ~24.8-day int32 ceiling + // (Node clamps an oversized delay to fire immediately, turning "backoff" + // into a tight retry loop). + const envBound = (name: string, fallback: number, max: number) => { + const raw = process.env[name]?.trim() + if (!raw) return fallback + const parsed = Number(raw) + return Number.isFinite(parsed) && parsed >= 0 ? Math.min(parsed, max) : fallback + } + const retryMax = envBound("ALTIMATE_RUN_RETRY_MAX", 3, 20) + const retryBaseMs = envBound("ALTIMATE_RUN_RETRY_BASE_MS", 1000, 60_000) + // The per-value bounds alone do NOT keep the compounded delay inside the + // timer range — see RunAccounting.retryDelayMs, which clamps it. + // altimate_change end + const send = () => { + if (args.command) + return sdk.session.command({ + sessionID, + agent, + model: args.model, + command: args.command, + arguments: message, + variant: args.variant, + }) const model = args.model ? Provider.parseModel(args.model) : undefined - await sdk.session.prompt({ + return sdk.session.prompt({ sessionID, agent, model, @@ -900,15 +1071,163 @@ You are speaking to a non-technical business executive. Follow these rules stric ...(audienceSystem ? { system: audienceSystem } : {}), }) } + type SendResult = { + error?: unknown + response?: Response + data?: { info?: { finish?: string; error?: { name?: unknown; data?: unknown } } } + } + let sendResult: SendResult | undefined + for (let sendAttempt = 0; ; sendAttempt++) { + let reason: string + try { + const res = (await send()) as SendResult + const status = res?.response?.status + if (!res?.error || !RunAccounting.isRetryableStatus(status)) { + sendResult = res + break + } + reason = `provider returned status ${status}` + } catch (e) { + if (!RunAccounting.isRetryableThrown(e)) throw e + reason = e instanceof Error ? e.message : String(e) + } + if (sendAttempt >= retryMax) throw new Error(`prompt failed after ${retryMax} retries: ${reason}`) + const delay = RunAccounting.retryDelayMs(retryBaseMs, sendAttempt) + if (!emit("retry", { attempt: sendAttempt + 1, max: retryMax, reason, delayMs: delay })) { + UI.println( + UI.Style.TEXT_WARNING_BOLD + "!", + UI.Style.TEXT_NORMAL + ` retrying prompt (${sendAttempt + 1}/${retryMax}) in ${delay}ms — ${reason}`, + ) + } + await new Promise((resolve) => setTimeout(resolve, delay)) + } + // the prompt response carries the TERMINAL assistant message — + // inspect it for swallowed abnormal endings (see RunAccounting.onPromptResult). + accounting.onPromptResult(sendResult?.data?.info) + // altimate_change end // Wait for the event loop to drain (breaks when session reaches idle) await loopPromise + // altimate_change start — one-shot confirm-DONE challenge phase. + // Reached only when the idle-done detector fired (all hard preconditions + // held) and aborted the churning prompt. The challenge is a normal prompt: + // the model either confirms DONE (session ends, done_reason=idle_heuristic) + // or states what remains and continues working — budget enforcement, + // accounting, and display all flow through the same loop() over a fresh + // event subscription. Recursion guard: the detector is one-shot, so the + // challenge can never breed further challenges. The directive is + // delivered via the nudge arbiter so this injected turn carries exactly + // ONE system-authored directive. + if (idleDone.challengeIssued && !accounting.fatal) { + // altimate_change start — upstream_fix: mark the challenge reply as + // sent BEFORE anything in this phase can raise a session-error/prompt- + // result event, so a genuine failure of the reply itself is never + // absorbed by the interrupted prompt's own abort suppression. + accounting.onIdleDoneChallengeReplySent() + // altimate_change end + // Dedicated abort for the challenge subscription so a failed challenge + // send can cancel the event-stream loop deterministically (the SSE + // generator exits cleanly on abort; the loop's for-await then drains). + const challengeAbort = new AbortController() + const challengeEvents = await sdk.event.subscribe(undefined, { signal: challengeAbort.signal }) + NudgeArbiter.register(sessionID, { + source: "termination_challenge", + kind: "confirm_done", + text: SessionTermination.CONFIRM_DONE_CHALLENGE, + }) + const challengeDirective = NudgeArbiter.take(sessionID) + let challengeSendFailed!: () => void + const challengeFailure = new Promise((resolveFailure) => { + challengeSendFailed = resolveFailure + }) + const challengePromise = (async () => { + // The abort releases the session lock asynchronously — retry briefly + // while the server still reports the session busy. Bounded so a + // persistent failure surfaces instead of hanging the run. + for (let challengeAttempt = 0; ; challengeAttempt++) { + const res = (await sdk.session + .prompt({ + sessionID, + agent, + model: args.model ? Provider.parseModel(args.model) : undefined, + variant: args.variant, + // altimate_change start — upstream_fix: forward the same audience + // directive as the original turns; otherwise a continuing + // challenge (model says what remains and keeps working) can drop + // back to technical output under --audience executive. + ...(audienceSystem ? { system: audienceSystem } : {}), + // altimate_change end + parts: [ + { type: "text", text: challengeDirective?.text ?? SessionTermination.CONFIRM_DONE_CHALLENGE }, + ], + }) + .catch((e) => ({ error: e }) as SendResult)) as SendResult + if (!res?.error) return res + if (challengeAttempt >= 8) { + emit("idle_done_challenge_failed", { error: RunAccounting.serializeSessionError(res.error) }) + throw new Error(`idle-done challenge prompt failed: ${RunAccounting.serializeSessionError(res.error)}`) + } + await new Promise((resolve) => setTimeout(resolve, 250 * (challengeAttempt + 1))) + } + })() + challengePromise.catch(() => challengeSendFailed()) + await Promise.race([ + loop(challengeEvents.stream, { requireBusyFirst: true }).catch((e) => { + console.error(e) + process.exit(1) + }), + challengeFailure, + ]) + // A failed challenge send must never be swallowed: the completion + // confirmation did not happen, so the run cannot report success (rc 0) + // — record it as a fatal harness error (why_harness_stopped=error) and + // cancel the still-pending event subscription so nothing keeps + // listening on a session whose confirmation path is dead. + const challengeResult = await challengePromise.catch((e) => { + accounting.onSessionError( + "IdleDoneChallengeFailed", + e instanceof Error ? e.message : String(e), + ) + return undefined + }) + // altimate_change start — upstream_fix: abort was only reached on the + // rejection path — the success path (and a `loop()` rejection racing + // ahead of it) left this event subscription open indefinitely. + // AbortController.abort() is idempotent, so calling it unconditionally + // here is safe even after the failure-path abort above. + challengeAbort.abort() + // altimate_change end + accounting.onPromptResult(challengeResult?.data?.info) + } + // altimate_change end + // Remove crash handlers — trace will be finalized cleanly + // altimate_change start — the run loop drained normally: mark the run + // finished and clear any exit code a premature beforeExit firing set. + // accounting.fatal below remains the single authority for a nonzero rc. + runFinished = true + process.exitCode = 0 + // altimate_change end process.removeListener("SIGINT", onSigint) process.removeListener("SIGTERM", onSigterm) process.removeListener("beforeExit", onBeforeExit) + // altimate_change start — dual-attribution termination + // record with done_reason. why_model_stopped and why_harness_stopped are + // independent fields so model-looping, tight budgets, and harness errors + // are distinguishable in the run output (rc alone conflates them); + // done_reason distinguishes explicit_done vs idle_heuristic vs none. + const termination = accounting.termination() + if (!emit("termination", { ...termination }) && process.stdout.isTTY) { + UI.println( + UI.Style.TEXT_DIM + + `why_model_stopped=${termination.why_model_stopped} why_harness_stopped=${termination.why_harness_stopped} done_reason=${termination.done_reason}` + + UI.Style.TEXT_NORMAL, + ) + } + // altimate_change end + // Finalize trace and save to disk if (tracer) { Tracer.setActive(null) @@ -928,6 +1247,12 @@ You are speaking to a non-technical business executive. Follow these rules stric await Bun.write(outputPath, content) process.stderr.write(`\n✓ Output saved to: ${outputPath}\n`) } + + // altimate_change start — honest rc — exit nonzero on fatal abort + // (budget exhaustion or an unrecovered session error). Uses process.exitCode + // (not process.exit) so pending stdout/trace writes still flush. + if (accounting.fatal) process.exitCode = 1 + // altimate_change end } if (args.attach) { diff --git a/packages/opencode/src/cli/cmd/run/run-mode.ts b/packages/opencode/src/cli/cmd/run/run-mode.ts new file mode 100644 index 0000000000..63e6aae9e0 --- /dev/null +++ b/packages/opencode/src/cli/cmd/run/run-mode.ts @@ -0,0 +1,32 @@ +// `altimate-code run` implies run mode. External drivers (CI, headless +// harnesses) invoke `run` without exporting ALTIMATE_RUN_MODE, which used to +// leave run-mode-only mechanisms (DONE-termination gate, starvation-breaker +// directives, doom-loop escalation ladder) disarmed. The run command applies +// this default at handler startup; interactive TUI/serve entrypoints never +// call it, so their behavior is unchanged. +// +// Opt-out: any explicit non-blank value is preserved — exporting +// ALTIMATE_RUN_MODE=0 (or "false") before launching `run` disables run mode. +// A blank/whitespace value is treated as unset, mirroring the +// ALTIMATE_NON_INTERACTIVE convention, so a stray `export ALTIMATE_RUN_MODE=` +// cannot silently disable termination. +export function applyRunModeDefault( + env: Record, + opts: { attach?: boolean; resumed?: boolean } = {}, +) { + // --attach: the agent runs on the remote (possibly interactive) server, so + // the local env var would be a no-op locally and must not leak run-mode + // semantics into other tools that consult it. + if (opts.attach) return + // A resumed run (`--continue`, `--session`, or `--fork`) starts from a + // session that already holds earlier invocations' messages. Run mode + // otherwise pins the session's FIRST user message as the authoritative task, + // which for a resumed session is a previous — possibly completed or + // conflicting — request rather than the one this invocation supplied. The + // marker lets the pin selector fall back to the latest substantive + // instruction for exactly these sessions. Set before the run-mode early + // return so an explicitly exported ALTIMATE_RUN_MODE=1 gets it too. + if (opts.resumed) env["ALTIMATE_RUN_RESUMED"] = "1" + if (env["ALTIMATE_RUN_MODE"]?.trim()) return + env["ALTIMATE_RUN_MODE"] = "1" +} diff --git a/packages/opencode/src/flag/flag.ts b/packages/opencode/src/flag/flag.ts index a5603215db..f20e1d0444 100644 --- a/packages/opencode/src/flag/flag.ts +++ b/packages/opencode/src/flag/flag.ts @@ -63,6 +63,30 @@ export namespace Flag { // altimate_change start - opt-out for AI Teammate training system export const ALTIMATE_DISABLE_TRAINING = altTruthy("ALTIMATE_DISABLE_TRAINING", "OPENCODE_DISABLE_TRAINING") // altimate_change end + // altimate_change start — run-mode marker. Set by `cli/cmd/run.ts` for + // in-process (non-attach) runs so run-mode-only mechanisms (starvation breaker + // directives, doom-loop escalation ladder) can arm. Never set by the TUI or + // `serve`, so interactive behavior is untouched by construction. Declared here, + // defined via dynamic getter below (run.ts sets the env var at handler time, + // after module load). + export declare const ALTIMATE_RUN_MODE: boolean + + // Strict trimmed boolean parser for the run-mode env var: " 1 " arms, "0" / + // "false" / blank disarm, and any other value warns once and disarms — a typo + // must never silently flip run-mode mechanisms without a trace. + const runModeWarned = new Set() + export function parseRunModeValue(raw: string | undefined): boolean { + const value = raw?.trim().toLowerCase() + if (!value) return false + if (value === "1" || value === "true") return true + if (value === "0" || value === "false") return false + if (!runModeWarned.has(value)) { + runModeWarned.add(value) + console.warn(`invalid ALTIMATE_RUN_MODE value ${JSON.stringify(raw)} ignored — use 1/0/true/false`) + } + return false + } + // altimate_change end export const OPENCODE_DISABLE_TERMINAL_TITLE = truthy("OPENCODE_DISABLE_TERMINAL_TITLE") export const OPENCODE_PERMISSION = process.env["OPENCODE_PERMISSION"] export const OPENCODE_DISABLE_DEFAULT_PLUGINS = truthy("OPENCODE_DISABLE_DEFAULT_PLUGINS") @@ -192,6 +216,16 @@ Object.defineProperty(Flag, "ALTIMATE_CLI_YOLO", { }) // altimate_change end +// altimate_change start — run-mode flag (dynamic getter; run.ts sets the env var at handler time) +Object.defineProperty(Flag, "ALTIMATE_RUN_MODE", { + get() { + return Flag.parseRunModeValue(process.env["ALTIMATE_RUN_MODE"]) + }, + enumerable: true, + configurable: false, +}) +// altimate_change end + // altimate_change start - ALTIMATE_CLI_CLIENT with OPENCODE_CLIENT fallback Object.defineProperty(Flag, "ALTIMATE_CLI_CLIENT", { get() { diff --git a/packages/opencode/src/session/compaction.ts b/packages/opencode/src/session/compaction.ts index 8dbc383bdf..3a34eeb4a1 100644 --- a/packages/opencode/src/session/compaction.ts +++ b/packages/opencode/src/session/compaction.ts @@ -16,6 +16,14 @@ import { Config } from "@/config/config" import { ProviderTransform } from "@/provider/transform" import { Telemetry } from "@/telemetry" // altimate_change — telemetry for compaction events import { ModelID, ProviderID } from "@/provider/schema" +// altimate_change start — summarizer-integrity error +import { NamedError } from "@opencode-ai/util/error" +import type { LLM } from "./llm" +// altimate_change start — completion-aware continue nudge via the nudge arbiter +import { NudgeArbiter } from "./nudge" +import { SessionTermination } from "./termination" +// altimate_change end +// altimate_change end // altimate_change start — Effect Context.Service facade for the upstream runtime import { Context, Effect, Layer } from "effect" import { LayerNode } from "@opencode-ai/core/effect/layer-node" @@ -78,7 +86,68 @@ export namespace SessionCompaction { // altimate_change start — improved isOverflow formula with safety guard and unified headroom // See PR #35 — fixes upstream bugs with limit.input models and small-context models - export async function isOverflow(input: { tokens: MessageV2.Assistant["tokens"]; model: Provider.Model }) { + // + // Estimator safety margin: chars-based Token.estimate values substantially + // undercount real tokenization of dense SQL/JSON (a request can exceed the + // provider limit while the estimate still looks safe). The safety fraction + // (context_safety_fraction, default 0.65) corrects for that — but ONLY where + // estimates are involved: estimate-derived budgets (fitHead, pin sizing) are + // computed against base * fraction, and the estimated component a caller + // passes to isOverflow is inflated by 1/fraction. PROVIDER-REPORTED usage is + // exact and is always compared against the raw limit minus headroom — + // scaling exact counts by the fraction forfeited ~35% of every window for + // sessions whose counts contain no estimate at all. + const DEFAULT_CONTEXT_SAFETY_FRACTION = 0.65 + // Trigger floor for small-context models where the safety fraction would push + // the threshold to ~0 tokens — firing on a near-empty session would livelock + // compaction. Clamped to the raw threshold so the margin can only ever make + // the trigger MORE conservative than the pre-margin formula. + const MIN_OVERFLOW_THRESHOLD = 4_000 + + export function contextSafetyFraction(cfg?: { compaction?: { context_safety_fraction?: number } }) { + // globalThis.process: SessionCompaction.process shadows the Node global here. + // Number() over the full trimmed value — parseFloat would accept numeric + // prefixes ("0.65junk") and silently override configuration. + const raw = globalThis.process.env["ALTIMATE_CONTEXT_SAFETY_FRACTION"]?.trim() + let env = Number.NaN + if (raw) { + env = Number(raw) + if (!Number.isFinite(env)) log.warn("invalid ALTIMATE_CONTEXT_SAFETY_FRACTION ignored", { value: raw }) + } + const value = Number.isFinite(env) ? env : (cfg?.compaction?.context_safety_fraction ?? DEFAULT_CONTEXT_SAFETY_FRACTION) + if (!Number.isFinite(value)) return DEFAULT_CONTEXT_SAFETY_FRACTION + return Math.min(1, Math.max(0.1, value)) + } + + /** Portion of a declared token limit treated as usable for estimate-vs-limit decisions. */ + export function effectiveContextLimit(base: number, fraction: number) { + return Math.floor(base * fraction) + } + + /** + * THE compaction-trigger threshold — the single formula shared by isOverflow + * (when to compact) and pinBudget (how much pinned content may survive + * compaction). Any consumer computing its own boundary from `base - headroom` + * risks admitting more retained content than the trigger allows, which + * re-fires compaction immediately (livelock). + */ + export function overflowThreshold(input: { base: number; headroom: number; fraction: number }) { + const effectiveBase = effectiveContextLimit(input.base, input.fraction) + return Math.min(input.base - input.headroom, Math.max(effectiveBase - input.headroom, MIN_OVERFLOW_THRESHOLD)) + } + + export async function isOverflow(input: { + tokens: MessageV2.Assistant["tokens"] + model: Provider.Model + /** + * Chars-based-estimated tokens included in the decision (e.g. the uncounted + * tail appended since the last provider-reported usage reading). The safety + * fraction applies ONLY to this component — it is inflated by 1/fraction to + * cover worst-case estimator undercount. `tokens` itself is provider-reported + * (exact) and is compared against the raw limit minus headroom. + */ + estimatedTokens?: number + }) { const config = await Config.get() if (config.compaction?.auto === false) return false const context = input.model.limit.context @@ -93,7 +162,10 @@ export namespace SessionCompaction { const headroom = Math.max(reserved, maxOutput) const base = input.model.limit.input ?? context if (base <= headroom) return false - return count >= base - headroom + const estimated = input.estimatedTokens ?? 0 + const adjusted = estimated > 0 ? count + Math.ceil(estimated / contextSafetyFraction(config)) : count + const threshold = overflowThreshold({ base, headroom, fraction: 1 }) + return adjusted >= threshold } // altimate_change end @@ -143,7 +215,14 @@ export namespace SessionCompaction { }) } - function preserveRecentBudget(input: { cfg: ConfigInfo; model: Provider.Model }) { + // Retained post-compaction content (verbatim tail + state ledger) may never + // by itself approach the overflow trigger: on small-window models a tail + // sized from the raw limit could exceed the threshold alone, so every + // compaction immediately re-triggered (per-turn summarization churn). Cap + // tail + ledger at this fraction of the trigger threshold. + const MAX_RETAINED_THRESHOLD_FRACTION = 0.5 + + export function preserveRecentBudget(input: { cfg: ConfigInfo; model: Provider.Model }) { const context = input.model.limit.context if (context === 0) return 0 @@ -152,10 +231,18 @@ export namespace SessionCompaction { const usable = input.model.limit.input ? Math.max(0, input.model.limit.input - reserved) : Math.max(0, context - maxOutput) - return ( + const candidate = input.cfg.compaction?.preserve_recent_tokens ?? Math.min(MAX_PRESERVE_RECENT_TOKENS, Math.max(MIN_PRESERVE_RECENT_TOKENS, Math.floor(usable * 0.25))) - ) + // Clamp (applies to explicit config too — the no-churn invariant is + // unconditional): tail budget + ledger budget <= fraction of the trigger. + const triggerHeadroom = Math.max(input.cfg.compaction?.reserved ?? COMPACTION_BUFFER, maxOutput) + const base = input.model.limit.input ?? context + if (base <= triggerHeadroom) return candidate // compaction disabled entirely; no trigger to protect + const threshold = overflowThreshold({ base, headroom: triggerHeadroom, fraction: 1 }) + const ledgerMax = input.cfg.compaction?.ledger_max_tokens ?? LEDGER_MAX_TOKENS + const retainCap = Math.max(0, Math.floor(threshold * MAX_RETAINED_THRESHOLD_FRACTION) - ledgerMax) + return Math.min(candidate, retainCap) } function turns(messages: MessageV2.WithParts[]) { @@ -203,6 +290,43 @@ export namespace SessionCompaction { return undefined } + // altimate_change start — head-truncation fallback for un-compactable sessions + // A session can overflow so far past the window (huge tool result landing in + // one turn) that the summarization request itself no longer fits, which used + // to terminate the session with "too large to compact". Summarizing a + // truncated head is lossy; killing the session loses everything. + export async function fitHead(input: { head: MessageV2.WithParts[]; model: Provider.Model; fraction?: number }) { + const context = input.model.limit.context + if (context === 0) return { head: input.head, dropped: 0 } + const maxOutput = ProviderTransform.maxOutputTokens(input.model) + const base = input.model.limit.input ?? context + // The summarization-request budget derives from the SAME safety-fraction + // helper as the overflow trigger — Token.estimate undercounts dense + // code/tool output, and a fallback sized against the raw limit can itself + // overflow under that estimator error. 2k covers the summary prompt. + const fraction = input.fraction ?? contextSafetyFraction() + const budget = Math.max(0, effectiveContextLimit(base, fraction) - maxOutput - 2_000) + if (budget <= 0) return { head: input.head, dropped: 0 } + let head = input.head + let dropped = 0 + while (head.length > 1 && (await estimate({ messages: head, model: input.model })) > budget) { + const step = Math.max(1, Math.floor(head.length / 8)) + // Round the cut forward to the next turn boundary: a head that starts + // mid-turn (assistant/tool messages with no leading user turn) is + // rejected by providers with a 400, defeating the fallback entirely. + let cut = step + while (cut < head.length && head[cut]!.info.role !== "user") cut++ + // No user boundary to cut at: fail closed with the current (still + // user-leading) head rather than slice mid-turn — an assistant/tool-leading + // head is rejected by providers with a 400, defeating the fallback. + if (cut >= head.length) break + head = head.slice(cut) + dropped += cut + } + return { head, dropped } + } + // altimate_change end + async function select(input: { messages: MessageV2.WithParts[]; cfg: ConfigInfo; model: Provider.Model }) { const limit = input.cfg.compaction?.tail_turns ?? DEFAULT_TAIL_TURNS if (limit <= 0) return { head: input.messages, tail_start_id: undefined } @@ -334,10 +458,374 @@ export namespace SessionCompaction { } } + // altimate_change start — post-compaction state ledger (5a), + // append-only summary carry (5b), first-person summary reframe (5c). + // + // 5a: a deterministic, corroborated-facts-only ledger appended to the synthetic + // post-compaction continue message. Facts come from harness tool events ONLY: + // write/edit/apply_patch completion events (path + event timestamp) and the last N + // tool calls with exit codes where recorded. Command-agnostic by design — no + // "build/test" classifier, no vertical (dbt/warehouse) token matching. + // Bash-mediated file changes produce no edit event, so they are flagged as possible + // but unverified rather than guessed at. The re-read directive is advisory and + // mtime-anchored, never an absolute prohibition (the model's read-before-edit habit + // is load-bearing, and external IDE edits can change disk mid-session). + // + // Thresholds are config-exposed (compaction.ledger_max_tokens / ledger_recent_calls). + // Rationale: the 500-token cap (tail-truncate) keeps the ledger cheaper than the + // duplicate re-reads it prevents (a single mid-size file re-read is ~1–3k tokens). + // 10 recent calls covers several typical edit→verify cycles without dominating + // the budget. Neither is fitted to any one workload. + export const LEDGER_MAX_TOKENS = 500 + export const LEDGER_RECENT_CALLS = 10 + + // Harness-corroborated write events: tools whose completion PROVES a file write. + // Deliberately excludes bash — shell writes are unverifiable from tool events. + const LEDGER_WRITE_TOOLS = new Set(["write", "edit"]) + const LEDGER_DETAIL_MAX = 100 + + export type LedgerWrite = { path: string; mtime: number; tool: string } + export type LedgerCall = { + tool: string + detail: string + exit?: number | null + errored: boolean + } + export type Ledger = { writes: LedgerWrite[]; calls: LedgerCall[]; sawBash: boolean } + + function callDetail(input: Record | null | undefined): string { + if (!input || typeof input !== "object") return "" + // Generic primary-argument pick — identical treatment for every tool. + const candidate = input.command ?? input.filePath ?? input.path ?? input.pattern ?? "" + const str = typeof candidate === "string" ? candidate.replace(/\s+/g, " ").trim() : "" + return str.length > LEDGER_DETAIL_MAX ? str.slice(0, LEDGER_DETAIL_MAX) + "…" : str + } + + /** Deterministic: output depends only on the message list passed in. */ + export function buildLedger(messages: MessageV2.WithParts[]): Ledger { + const writes = new Map() + const calls: LedgerCall[] = [] + let sawBash = false + for (const msg of messages) { + for (const part of msg.parts) { + if (part.type !== "tool") continue + const state = part.state + if (state.status !== "completed" && state.status !== "error") continue + const errored = state.status === "error" + const metadata: Record = state.metadata ?? {} + const exit = typeof metadata.exit === "number" || metadata.exit === null ? metadata.exit : undefined + calls.push({ tool: part.tool, detail: callDetail(state.input), exit, errored }) + if (part.tool === "bash") sawBash = true + if (errored) continue + if (LEDGER_WRITE_TOOLS.has(part.tool)) { + const filePath = typeof state.input?.filePath === "string" ? state.input.filePath : undefined + // mtime = tool-event completion time, NOT an fs.stat — corroborated facts only. + if (filePath) writes.set(filePath, { path: filePath, mtime: state.time.end, tool: part.tool }) + } + if (part.tool === "apply_patch") { + const files = Array.isArray(metadata.files) ? metadata.files : [] + for (const f of files) { + // A delete wrote nothing — recording it would advertise a file that + // no longer exists as freshly written. + if (f?.type === "delete") continue + // On a move, `filePath` is the SOURCE and `movePath` is where the + // content actually landed; the ledger must name the destination or + // it sends the continuing agent back to the path that was removed. + const target = typeof f?.movePath === "string" ? f.movePath : f?.filePath + if (typeof target === "string") + writes.set(target, { path: target, mtime: state.time.end, tool: "apply_patch" }) + } + } + } + } + return { + writes: [...writes.values()].sort((a, b) => b.mtime - a.mtime || a.path.localeCompare(b.path)), + calls, + sawBash, + } + } + + /** + * Render the ledger for the continue message. ≤ maxTokens, tail-truncated: + * content is ordered by importance (verified writes → unverified-shell note → + * advisory → recent calls newest-first) so truncation drops the oldest calls first. + */ + export function renderLedger(ledger: Ledger, opts?: { maxTokens?: number; recentCalls?: number }): string { + const maxTokens = opts?.maxTokens ?? LEDGER_MAX_TOKENS + const recentCalls = opts?.recentCalls ?? LEDGER_RECENT_CALLS + if (!ledger.writes.length && !ledger.calls.length) return "" + const lines: string[] = ["[Session state ledger — harness-recorded facts, generated automatically at compaction]"] + if (ledger.writes.length) { + lines.push("Files you wrote this session (verified write/edit tool events):") + for (const w of ledger.writes) { + lines.push(`- ${w.path} — last written by you at ${new Date(w.mtime).toISOString()} via ${w.tool}`) + } + } + if (ledger.sawBash) { + lines.push( + "Shell commands also ran this session; any file changes they made are possible but unverified (shell writes produce no edit event).", + ) + } + lines.push( + "Advisory: these files were last written by you at the times shown — prefer this ledger over re-reading them; re-read a file only if a tool errored, you suspect external changes (e.g. IDE edits), or you are about to edit it.", + ) + if (ledger.calls.length) { + const recent = ledger.calls.slice(-recentCalls).reverse() + lines.push(`Recent tool calls, newest first (last ${recent.length} of ${ledger.calls.length}):`) + for (const c of recent) { + const status = c.errored ? "errored" : c.exit === undefined ? "ok" : c.exit === null ? "exit ?" : `exit ${c.exit}` + lines.push(`- ${c.tool} (${status})${c.detail ? ` — ${c.detail}` : ""}`) + } + } + while (lines.length > 1 && Token.estimate(lines.join("\n")) > maxTokens) lines.pop() + // A header with every fact truncated away carries no information but is + // still charged against a budget the caller assumed was spent on facts — + // and `ledger_max_tokens: 0` (which the schema accepts) would otherwise + // inject unbudgeted text. Emit nothing when not even the header fits. + if (Token.estimate(lines.join("\n")) > maxTokens) return "" + return lines.join("\n") + } + + // ── 5b: append-only summary carry ───────────────────────────────────────── + // Previous round's Accomplished items are threaded into the next summarization + // as anchors. An item carries as FACT ([verified]) only when a corroborating + // ledger event exists (a write/edit event, or a zero-exit command naming the + // artifact); otherwise it carries tagged "claimed, unverified". A naive carry + // would REMEMBER invented deliverables (summaries can fabricate them) and + // propagate them to every later summary and subagent. + + export type CarryStatus = "verified" | "claimed, unverified" + export type CarryItem = { text: string; status: CarryStatus } + + const CARRY_TAG_RE = /^\[(verified|claimed, unverified)\]\s*(.*)$/i + + /** Extract bullet items under the "## Accomplished" heading of a summary. */ + export function extractAccomplished(summary: string): { text: string; priorStatus?: CarryStatus }[] { + const out: { text: string; priorStatus?: CarryStatus }[] = [] + let inSection = false + for (const raw of summary.split("\n")) { + const line = raw.trim() + if (/^#{1,6}\s/.test(line)) { + inSection = /^#{1,6}\s*accomplished\b/i.test(line) + continue + } + if (!inSection) continue + const m = line.match(/^[-*]\s+(.*\S)\s*$/) + if (!m) continue + let text = m[1]! + let priorStatus: CarryStatus | undefined + const tag = text.match(CARRY_TAG_RE) + if (tag) { + priorStatus = tag[1]!.toLowerCase() as CarryStatus + text = tag[2]! + } + if (text) out.push({ text, priorStatus }) + } + return out + } + + /** Path-like tokens (contain a dot or slash) — the artifact names a claim can be checked against. */ + function artifactTokens(text: string): string[] { + return (text.match(/[A-Za-z0-9_@-]*[./][A-Za-z0-9_./-]+/g) ?? []).filter((t) => t.length >= 3 && /[A-Za-z]/.test(t)) + } + + function itemCorroborated(text: string, ledger: Ledger): boolean { + for (const token of artifactTokens(text)) { + // Basename fallback only for a bare filename. A directory-qualified token + // (`src/index.ts`) must match its own path — otherwise any unrelated + // `test/index.ts` write corroborates it, and because carry status is + // append-only that wrong [verified] fact survives every later compaction. + const base = token.includes("/") ? "" : token + for (const w of ledger.writes) { + if (w.path === token || w.path.endsWith("/" + token)) return true + if (base && w.path.split("/").pop() === base) return true + } + // A zero-exit command naming the artifact also corroborates (command-agnostic — + // no build/test classifier; the exit code plus artifact mention is the evidence). + for (const c of ledger.calls) { + if (!c.errored && c.exit === 0 && c.detail.includes(token)) return true + } + } + return false + } + + /** + * Append-only status resolution: once [verified], always [verified] — the + * corroborating event may have been compacted out of the retained window, so a + * prior verified tag is preserved. Unverified claims may be promoted when + * evidence appears, never silently demoted or dropped. + */ + export function corroborateCarry(items: { text: string; priorStatus?: CarryStatus }[], ledger: Ledger): CarryItem[] { + return items.map((item) => ({ + text: item.text, + status: + item.priorStatus === "verified" || itemCorroborated(item.text, ledger) + ? "verified" + : ("claimed, unverified" as const), + })) + } + + export function renderCarryAnchors(items: CarryItem[], maxTokens: number = LEDGER_MAX_TOKENS): string { + if (!items.length) return "" + const header = [ + "## Previous-summary anchors (append-only carry)", + "Earlier compaction rounds recorded these Accomplished items. Carry EVERY item below into the new summary's Accomplished section with its tag verbatim, then append newly accomplished work after them:", + ] + const footer = [ + "Items tagged [claimed, unverified] had no corroborating tool event (no write/edit event or successful command naming that artifact); keep the tag so later agents do not treat them as established fact. Never promote or remove a tag yourself.", + ] + let body = items.map((i) => `- [${i.status}] ${i.text}`) + // Append-only carry grows monotonically; when over budget drop the OLDEST + // items (front of the list) — the freshest anchors are the ones the next + // round needs to not lose. + while (body.length > 1 && Token.estimate([...header, ...body, ...footer].join("\n")) > maxTokens) { + body = body.slice(1) + } + return [...header, ...body, ...footer].join("\n") + } + + /** Most recent committed summary text, if any (assistant, summary, finished, no error). */ + export function latestSummaryText(messages: MessageV2.WithParts[]): string | undefined { + for (let i = messages.length - 1; i >= 0; i--) { + const msg = messages[i]! + if (msg.info.role !== "assistant" || !msg.info.summary || !msg.info.finish || msg.info.error) continue + const text = msg.parts + .filter((p): p is MessageV2.TextPart => p.type === "text") + .map((p) => p.text) + .join("\n") + return text.trim() ? text : undefined + } + return undefined + } + + // ── 5c: first-person summary reframe — layered as an ADDITION to whatever + // summary prompt is active (default or plugin-provided), never a replacement. + export const FIRST_PERSON_REFRAME = + "Additionally: write the summary in the first person, as your own working memory — you are summarizing YOUR OWN work in progress, and the agent reading it next is you, continuing the same task. Say \"I edited…\", \"I verified…\", \"I still need to…\" rather than describing the work as another agent's or the user's." + // altimate_change end + // altimate_change start — compaction attempt tracking for loop protection const compactionAttempts = new Map() // altimate_change end + // altimate_change start — pin the original task + // verbatim through compaction (budget math + livelock guard). + // + // Threshold rationale (config-exposed defaults, not fitted to any one workload): + // - PIN_MAX_TOKENS 4096: task statements rarely exceed ~4k tokens; larger + // ones keep verbatim head+tail plus a contract card. + // - PIN_WINDOW_FRACTION 0.175: the pin must stay a small minority of the + // post-overhead usable window so working context dominates. + // - PIN_WORKING_SLACK 2000: hard invariant + // `pin + reserved + ≥2k working slack < compaction threshold`. A fixed 4k + // pin on a small window would otherwise produce a compaction livelock + // (fires, cannot reduce below threshold, re-fires). Shrink the pin, never + // violate the invariant. + // - PIN_CARD_MAX_TOKENS 500: contract-card budget. + export const PIN_MAX_TOKENS = 4_096 + export const PIN_WINDOW_FRACTION = 0.175 + export const PIN_WORKING_SLACK = 2_000 + export const PIN_CARD_MAX_TOKENS = 500 + + export function pinEnabled(cfg: ConfigInfo) { + return cfg.compaction?.pin_task !== false + } + + export function pinCardBudget(cfg: ConfigInfo) { + return cfg.compaction?.pin_card_max_tokens ?? PIN_CARD_MAX_TOKENS + } + + // Dynamic cap: min(pin_max_tokens, pin_window_fraction × post-overhead usable + // window), clamped by the livelock invariant and any per-session livelock + // halving. Returns 0 when no pin fits — the pin is then skipped entirely. + export function pinBudget(input: { cfg: ConfigInfo; model: Provider.Model; sessionID?: string }): number { + if (!pinEnabled(input.cfg)) return 0 + const context = input.model.limit.context + if (context === 0) return 0 + const maxOutput = ProviderTransform.maxOutputTokens(input.model) + const reserved = input.cfg.compaction?.reserved ?? COMPACTION_BUFFER + const headroom = Math.max(reserved, maxOutput) + const base = input.model.limit.input ?? context + // The pin capacity derives from the shared overflowThreshold helper, at the + // safety-fraction-scaled (estimate-domain) boundary: pin sizes are + // Token.estimate values, so they are budgeted conservatively. This is always + // <= the raw trigger isOverflow() compares provider-reported usage against, + // so an admitted pin (plus reserved buffer and working slack) can never by + // itself re-fire compaction immediately after a compaction. + if (base <= headroom) return 0 + const threshold = overflowThreshold({ base, headroom, fraction: contextSafetyFraction(input.cfg) }) + if (threshold <= 0) return 0 + const maxTokens = input.cfg.compaction?.pin_max_tokens ?? PIN_MAX_TOKENS + const fraction = input.cfg.compaction?.pin_window_fraction ?? PIN_WINDOW_FRACTION + // Hard invariant: pin + ≥2k working slack < compaction threshold. The + // threshold already excludes the reserved/headroom buffer (overflowThreshold + // subtracts it from base), so subtracting `reserved` again here + // double-counted it and silently zeroed the pin on smaller windows. + const invariantCap = threshold - PIN_WORKING_SLACK + const cap = Math.min(maxTokens, Math.floor(threshold * fraction), invariantCap) + if (cap <= 0) return 0 + return Math.max(0, Math.floor(cap * pinScale(input.sessionID))) + } + + // Livelock guard: two CONSECUTIVE auto-compactions that failed to get the + // session below threshold halve the pin for the rest of the session (and + // halve again on each further pair). "Failed to reduce below threshold" is + // detected structurally: a new auto-compaction fires while at most one + // finished non-summary assistant turn exists after the previous completed + // summary — i.e. the session re-overflowed immediately. + const pinState = new Map() + + export function pinScale(sessionID?: string): number { + if (!sessionID) return 1 + return pinState.get(sessionID)?.scale ?? 1 + } + + /** Test hook: clear livelock state for one session, or all sessions. */ + export function resetPinState(sessionID?: string) { + if (sessionID) pinState.delete(sessionID) + else pinState.clear() + } + + /** Called by the auto-overflow paths in prompt.ts BEFORE creating a new compaction. */ + export function notePinCompaction(sessionID: string, msgs: MessageV2.WithParts[]) { + const state = pinState.get(sessionID) ?? { failures: 0, scale: 1 } + let lastSummary = -1 + for (let i = msgs.length - 1; i >= 0; i--) { + const info = msgs[i].info + if (info.role === "assistant" && info.summary && info.finish && !info.error) { + lastSummary = i + break + } + } + let immediate = false + if (lastSummary >= 0) { + let finished = 0 + for (let i = lastSummary + 1; i < msgs.length; i++) { + const info = msgs[i].info + if (info.role === "assistant" && info.finish && !info.summary) finished++ + } + immediate = finished <= 1 + } + state.failures = immediate ? state.failures + 1 : 0 + if (state.failures >= 2) { + state.scale /= 2 + state.failures = 0 + log.warn("task pin halved — consecutive compactions failed to reduce below threshold", { + sessionID, + scale: state.scale, + }) + } + pinState.set(sessionID, state) + } + + // Summary-template line, layered as an ADDITION to the active summary prompt + // (never a replacement — a plugin-supplied custom prompt REPLACES the + // platform's preservation prompt, which is exactly the failure mode the plan + // warns about; this constant is only ever appended). + export const PIN_SUMMARY_ADDITION = + "Do NOT restate the original task requirements in the summary — the original task text is pinned separately and stays visible alongside this summary. If anything in this summary conflicts with the pinned original task, the pinned task is authoritative." + // altimate_change end + export async function process(input: { parentID: MessageID messages: MessageV2.WithParts[] @@ -364,8 +852,14 @@ export namespace SessionCompaction { attempt, }) if (attempt > 3) { + // Returning undefined here made the prompt loop's `continue` re-enter + // process() immediately (the pending compaction marker stays unresolved), + // hot-spinning with a telemetry event per iteration. Return "stop" so the + // caller breaks, and clear the counter so a later prompt gets a fresh + // bounded set of attempts instead of tripping the breaker instantly. log.warn("compaction circuit breaker", { sessionID: input.sessionID, attempt }) - return + compactionAttempts.delete(input.sessionID) + return "stop" } // altimate_change end const parent = input.messages.findLast((m) => m.info.id === input.parentID) @@ -404,8 +898,26 @@ export namespace SessionCompaction { await Provider.getModel(ProviderID.make(agent.model.providerID), ModelID.make(agent.model.modelID)) : // altimate_change end await Provider.getModel(userMessage.model.providerID, userMessage.model.modelID) + // altimate_change start — upstream_fix: the compaction agent may override its + // own model (`agent.model` above), but pinBudget must be computed against the + // SESSION's model — the pin is re-injected into the session's next turn, not + // the compaction agent's. Reuse `model` when they're the same (the common, + // no-override case) instead of resolving twice. + const sessionModel = agent.model + ? await Provider.getModel(userMessage.model.providerID, userMessage.model.modelID) + : model + // altimate_change end // altimate_change start — upstream_fix: restore tail-preserving compaction selection const cfg = await Config.get() + // altimate_change start — state ledger + summary carry wiring + const ledgerEnabled = cfg.compaction?.state_ledger !== false + const carryEnabled = cfg.compaction?.summary_carry !== false + const firstPersonEnabled = cfg.compaction?.summary_first_person !== false + const ledgerMaxTokens = cfg.compaction?.ledger_max_tokens ?? LEDGER_MAX_TOKENS + const ledgerRecentCalls = cfg.compaction?.ledger_recent_calls ?? LEDGER_RECENT_CALLS + const ledger: Ledger = + ledgerEnabled || carryEnabled ? buildLedger(input.messages) : { writes: [], calls: [], sawBash: false } + // altimate_change end const history = compactionPart && messages.at(-1)?.info.id === input.parentID ? messages.slice(0, -1) : messages const prior = completedCompactions(history) const hidden = new Set(prior.flatMap((item) => [item.userIndex, item.assistantIndex])) @@ -491,17 +1003,69 @@ When constructing the summary, try to stick to this template: [Construct a structured list of relevant files that have been read, edited, or created that pertain to the task at hand. If all the files in a directory are relevant, include the path to the directory.] ---` - const promptText = compacting.prompt ?? [defaultPrompt, ...compacting.context].join("\n\n") - const result = await processor.process({ + // altimate_change start — summary carry + first-person reframe: layered ADDITIONS to whichever + // summary prompt is active (default or plugin-provided) — never a replacement. + let promptText = compacting.prompt ?? [defaultPrompt, ...compacting.context].join("\n\n") + if (carryEnabled) { + const previousSummary = latestSummaryText(input.messages) + if (previousSummary) { + const anchors = renderCarryAnchors( + corroborateCarry(extractAccomplished(previousSummary), ledger), + ledgerMaxTokens, + ) + if (anchors) promptText += "\n\n" + anchors + } + } + if (firstPersonEnabled) promptText += "\n\n" + FIRST_PERSON_REFRAME + // altimate_change end + // altimate_change start — when task pinning is + // active AND will actually fit a nonzero budget for the session's model, + // tell the summarizer not to burn summary tokens restating the task (the + // original task is pinned separately and re-injected after compaction). + // pinEnabled(cfg) alone doesn't guarantee a pin: pinBudget can return 0 on a + // small-window session, which would otherwise tell the summarizer to omit + // the task while no pin exists to compensate. Layered as an ADDITION to + // whichever summary prompt is active — never a replacement. + if (pinEnabled(cfg) && pinBudget({ cfg, model: sessionModel, sessionID: input.sessionID }) > 0) + promptText += "\n\n" + PIN_SUMMARY_ADDITION + // altimate_change end + // altimate_change start — summarizer integrity: + // hoist the summarizer input so a failed attempt can be retried with identical + // input, and pass an explicit toolChoice "none". Previously toolChoice was + // undefined, which the AI SDK defaults to "auto" — models could spend the + // summary step on a tool call and commit a summary with no text. + const summarizerInput: LLM.StreamInput = { user: userMessage, agent, abort: input.abort, sessionID: input.sessionID, tools: {}, system: [], + toolChoice: "none" as const, messages: [ - // altimate_change start — upstream_fix: summarize only the selected head when preserving recent tail - ...(await MessageV2.toModelMessages(selected.head, model, { stripMedia: true })), + // altimate_change start — upstream_fix: summarize only the selected head when preserving recent tail; + // trim the head from the front when even the summarization request cannot fit the window + ...(await MessageV2.toModelMessages( + await (async () => { + const fitted = await fitHead({ head: selected.head, model, fraction: contextSafetyFraction(cfg) }) + if (fitted.dropped > 0) { + log.warn("compaction head truncated to fit window", { + dropped: fitted.dropped, + kept: fitted.head.length, + }) + Telemetry.track({ + type: "compaction_head_truncated", + timestamp: Date.now(), + session_id: input.sessionID, + dropped_messages: fitted.dropped, + kept_messages: fitted.head.length, + }) + } + return fitted.head + })(), + model, + { stripMedia: true }, + )), // altimate_change end { role: "user", @@ -514,7 +1078,29 @@ When constructing the summary, try to stick to this template: }, ], model, - }) + } + // A "continue" result was previously committed regardless of whether the + // summary step produced any text — an empty summary erases history (the + // post-compaction amnesia signature). Guard the commit: retry ONCE with + // identical input, then mark the summary message as errored and stop. + const summaryHasText = () => + MessageV2.get({ sessionID: input.sessionID, messageID: msg.id }).parts.some( + (part) => part.type === "text" && part.text.trim().length > 0, + ) + let result = await processor.process(summarizerInput) + if (result === "continue" && !summaryHasText()) { + log.warn("compaction summary empty, retrying once", { sessionID: input.sessionID }) + result = await processor.process(summarizerInput) + if (result === "continue" && !summaryHasText()) { + processor.message.error = new NamedError.Unknown({ + message: "Compaction summarizer produced no summary text after retry", + }).toObject() + processor.message.finish = "error" + await Session.updateMessage(processor.message) + result = "stop" + } + } + // altimate_change end if (result === "compact") { processor.message.error = new MessageV2.ContextOverflowError({ @@ -524,6 +1110,7 @@ When constructing the summary, try to stick to this template: }).toObject() processor.message.finish = "error" await Session.updateMessage(processor.message) + compactionAttempts.delete(input.sessionID) // altimate_change — cleanup on too-large-to-compact stop return "stop" } @@ -565,6 +1152,16 @@ When constructing the summary, try to stick to this template: }) } } else { + // altimate_change start — the continue message + // carries the original format/tools/system/variant, exactly as the replay + // branch above copies them from the original user message. Dropping them made + // the first auto-compaction silently reset the session's tool allowlist, + // custom system prompt, output format, and variant. The compaction marker + // (this branch's userMessage) never carries these fields, so source them from + // the most recent real (non-compaction) user message; no-op when never set. + const original = messages.findLast( + (m) => m.info.role === "user" && !m.parts.some((p) => p.type === "compaction"), + )?.info as MessageV2.User | undefined const continueMsg = await Session.updateMessage({ id: MessageID.ascending(), role: "user", @@ -572,12 +1169,42 @@ When constructing the summary, try to stick to this template: time: { created: Date.now() }, agent: userMessage.agent, model: userMessage.model, + format: original?.format ?? userMessage.format, + tools: original?.tools ?? userMessage.tools, + system: original?.system ?? userMessage.system, + variant: original?.variant ?? userMessage.variant, }) + // altimate_change end + // altimate_change start — deterministic corroborated-facts-only + // state ledger appended to the synthetic continue message (all-modes, compaction-gated). + const ledgerText = ledgerEnabled + ? renderLedger(ledger, { maxTokens: ledgerMaxTokens, recentCalls: ledgerRecentCalls }) + : "" + // altimate_change end + // altimate_change start — completion-aware termination path: + // (b) the continue message carries the three-option completion-aware nudge + // (continue / ask for clarification / assert DONE), giving a finished + // session a termination path. Delivered via the NudgeArbiter (Global + // rule 5): this injection point registers its candidate and takes the + // single winner, so pending lower-precedence directives (starvation + // breaker, budget reminder) are consumed here and the injected turn + // never carries two system-authored directive blocks. The termination + // nudge has top precedence, so it always wins at this site. + // (d) the overflow notice is mechanism-accurate — the old text falsely + // blamed "large media attachments" (see SessionTermination.OVERFLOW_NOTICE). + NudgeArbiter.register(input.sessionID, { + source: "termination_challenge", + kind: "completion_nudge", + text: SessionTermination.COMPLETION_NUDGE, + }) + const directive = NudgeArbiter.take(input.sessionID) const text = - (input.overflow - ? "The previous request exceeded the provider's size limit due to large media attachments. The conversation was compacted and media files were removed from context. If the user was asking about attached images or files, explain that the attachments were too large to process and suggest they try again with smaller or fewer files.\n\n" - : "") + - "Continue if you have next steps, or stop and ask for clarification if you are unsure how to proceed." + (input.overflow ? SessionTermination.OVERFLOW_NOTICE + "\n\n" : "") + + (directive?.text ?? SessionTermination.COMPLETION_NUDGE) + + // altimate_change end + // altimate_change start — state ledger + (ledgerText ? "\n\n" + ledgerText : "") + // altimate_change end await Session.updatePart({ id: PartID.ascending(), messageID: continueMsg.id, diff --git a/packages/opencode/src/session/llm.ts b/packages/opencode/src/session/llm.ts index 893f4dda4d..a4c826d5f5 100644 --- a/packages/opencode/src/session/llm.ts +++ b/packages/opencode/src/session/llm.ts @@ -173,19 +173,7 @@ export namespace LLM { // tools absent from the current set. Add stub definitions for any missing tools. // Fixes: https://github.com/AltimateAI/altimate-code/issues/678 const referencedTools = toolNamesFromMessages(input.messages) - for (const name of referencedTools) { - if (!Object.hasOwn(tools, name)) { - tools[name] = tool({ - description: `[Historical] Tool no longer available in this session`, - inputSchema: jsonSchema({ type: "object", properties: {} }), - execute: async () => ({ - output: "This tool is no longer available. Please use an alternative approach.", - title: "", - metadata: {}, - }), - }) - } - } + addHistoricalToolStubs(tools, referencedTools, input.toolChoice) // altimate_change end // altimate_change start — tool retrieval @@ -340,6 +328,45 @@ export namespace LLM { } return names } + + // Mutates `tools`, adding a stub definition for every referenced historical tool + // name that has no real definition (see toolNamesFromMessages above / issue #678). + // + // Skip stub injection only for the explicit toolChoice "none" no-tool-call + // contract (e.g. the compaction summarizer, which passes tools: {} and + // toolChoice "none"). With an empty tool set AND toolChoice "none" the AI SDK + // omits both `tools` and `tool_choice` from the request, which every provider + // accepts — this is the compat fallback for providers whose OpenAI-compat + // layer rejects toolChoice "none". Injecting stubs there would instead + // advertise callable tools on a call that must produce text only. + // + // altimate_change start — upstream_fix: gating on `tools` being empty alone + // (rather than toolChoice) also matched a NORMAL turn whose tool allowlist or + // agent permissions stripped every tool but whose history still references + // tool calls — skipping stubs there could reintroduce the Anthropic + // "tool_use with no matching definition" 400 this function exists to fix. + export function addHistoricalToolStubs( + tools: Record, + referenced: Iterable, + toolChoice?: "auto" | "required" | "none", + ) { + if (toolChoice === "none" && Object.keys(tools).length === 0) return tools + // altimate_change end + for (const name of referenced) { + if (!Object.hasOwn(tools, name)) { + tools[name] = tool({ + description: `[Historical] Tool no longer available in this session`, + inputSchema: jsonSchema({ type: "object", properties: {} }), + execute: async () => ({ + output: "This tool is no longer available. Please use an alternative approach.", + title: "", + metadata: {}, + }), + }) + } + } + return tools + } // altimate_change end // altimate_change start — Effect Context.Service facade so the new upstream consumers diff --git a/packages/opencode/src/session/message-v2.ts b/packages/opencode/src/session/message-v2.ts index d89899520e..0440742e76 100644 --- a/packages/opencode/src/session/message-v2.ts +++ b/packages/opencode/src/session/message-v2.ts @@ -31,6 +31,30 @@ export namespace MessageV2 { return mime.startsWith("image/") || mime === "application/pdf" } + // altimate_change start — deterministic tool-call id sanitation. Some + // OpenAI-compatible servers emit non-string (numeric/object) tool-call ids; + // providers reject any request whose tool_use/tool_result pair carries a + // malformed or mismatched id. Valid non-empty strings pass through untouched. + // Anything else is regenerated deterministically (FNV-1a over the JSON form), + // so the SAME raw value always maps to the SAME id — the property that keeps + // the call half and the result half of a pair consistent whether coerced at + // ingestion (processor.ts) or defensively at replay (toModelMessagesEffect). + // `salt` (optional; e.g. the processor's assistant message id) is folded into + // the hash so regenerated ids for empty/identical malformed raw values do not + // collide across processors/steps. Persisted sanitized ids are valid strings + // and pass through unchanged on replay, so salting never breaks a stored pair. + export function sanitizeToolCallID(id: unknown, salt?: string): string { + if (typeof id === "string" && id.length > 0) return id + const raw = (salt ?? "") + "\u0000" + (typeof id === "string" ? id : (JSON.stringify(id) ?? String(id))) + let hash = 0x811c9dc5 + for (let i = 0; i < raw.length; i++) { + hash ^= raw.charCodeAt(i) + hash = Math.imul(hash, 0x01000193) + } + return "call_" + (hash >>> 0).toString(16).padStart(8, "0") + } + // altimate_change end + // altimate_change start — shared synthetic-attachment prompt text. Used both when // injecting tool-result media as a user message (below) and by the GitHub Copilot // plugin's imgMsg() heuristic so the two stay in sync. @@ -781,6 +805,14 @@ export namespace MessageV2 { }) if (part.type === "tool") { toolNames.add(part.tool) + // altimate_change start — defensive replay-side id coercion. Parts + // persisted after the ingestion fix already carry sanitized string ids; + // transcripts written before it may hold malformed (non-string) callIDs. + // Computing the sanitized id ONCE per tool part and using it for every + // rendered half guarantees the tool-call and its paired tool-result emit + // identical toolCallId values, so provider pairing validation cannot 400. + const replayCallID = sanitizeToolCallID(part.callID) + // altimate_change end if (part.state.status === "completed") { // altimate_change start — toolOutputMaxChars truncates long tool output for compaction const rawOutputText = part.state.time.compacted @@ -816,7 +848,9 @@ export namespace MessageV2 { assistantMessage.parts.push({ type: ("tool-" + part.tool) as `tool-${string}`, state: "output-available", - toolCallId: part.callID, + // altimate_change start — replay-side id coercion + toolCallId: replayCallID, + // altimate_change end input: part.state.input, output, ...(differentModel ? {} : { callProviderMetadata: part.metadata }), @@ -829,7 +863,7 @@ export namespace MessageV2 { assistantMessage.parts.push({ type: ("tool-" + part.tool) as `tool-${string}`, state: "output-available", - toolCallId: part.callID, + toolCallId: replayCallID, input: part.state.input, output, ...(differentModel ? {} : { callProviderMetadata: part.metadata }), @@ -838,7 +872,7 @@ export namespace MessageV2 { assistantMessage.parts.push({ type: ("tool-" + part.tool) as `tool-${string}`, state: "output-error", - toolCallId: part.callID, + toolCallId: replayCallID, input: part.state.input, errorText: part.state.error, ...(differentModel ? {} : { callProviderMetadata: part.metadata }), @@ -852,7 +886,9 @@ export namespace MessageV2 { assistantMessage.parts.push({ type: ("tool-" + part.tool) as `tool-${string}`, state: "output-error", - toolCallId: part.callID, + // altimate_change start — replay-side id coercion + toolCallId: replayCallID, + // altimate_change end input: part.state.input, errorText: "[Tool execution was interrupted]", ...(differentModel ? {} : { callProviderMetadata: part.metadata }), diff --git a/packages/opencode/src/session/nudge.ts b/packages/opencode/src/session/nudge.ts new file mode 100644 index 0000000000..15f3102753 --- /dev/null +++ b/packages/opencode/src/session/nudge.ts @@ -0,0 +1,83 @@ +// Fork-only module — nudge arbiter. +// +// At most ONE system-authored directive block may be injected per turn. +// Precedence (highest first): +// termination_challenge (item 1) > starvation_breaker (item 4) > budget_reminder (item 9) +// +// Items register candidate directives during a step; the delivery site (the +// session processor, at the start of the next generation) calls `take()` and +// injects only the single highest-precedence winner. All other pending +// directives for that turn are DROPPED, not deferred — detectors re-register +// on the next step if their condition still holds, so deferral would only +// create stale directives. This ships with item 4 (the first of items 1/4/9 +// to land); items 1 and 9 register through the same registry when they ship. +export namespace NudgeArbiter { + export type Source = "termination_challenge" | "starvation_breaker" | "budget_reminder" + + // Precedence order — index 0 wins. + export const PRECEDENCE: readonly Source[] = ["termination_challenge", "starvation_breaker", "budget_reminder"] + + export interface Directive { + source: Source + // A stable machine-readable tag for telemetry (e.g. "starvation", "repeat_signature"). + kind: string + text: string + } + + // Session-scoped pending directives. Bounded so long-lived server processes + // cannot accumulate state for dead sessions. + const MAX_SESSIONS = 128 + const pendingBySession = new Map() + + function bucket(sessionID: string): Directive[] { + let b = pendingBySession.get(sessionID) + if (!b) { + b = [] + if (pendingBySession.size >= MAX_SESSIONS) { + // Evict the LEAST-RECENTLY-USED session (front of the Map after the + // refresh-on-access below), never the oldest-created — a long-running + // active session must not lose a pending directive to churn from + // short-lived ones. + const oldest = pendingBySession.keys().next().value + if (oldest !== undefined) pendingBySession.delete(oldest) + } + } else { + // Refresh recency: re-insert so Map iteration order tracks last access. + pendingBySession.delete(sessionID) + } + pendingBySession.set(sessionID, b) + return b + } + + /** Register a candidate directive for the session's next injected turn. + * Multiple registrations from the same source+kind replace, not stack. */ + export function register(sessionID: string, directive: Directive): void { + const b = bucket(sessionID) + const existing = b.findIndex((d) => d.source === directive.source && d.kind === directive.kind) + if (existing >= 0) b[existing] = directive + else b.push(directive) + } + + /** Pending directives (test/telemetry visibility only). */ + export function pending(sessionID: string): readonly Directive[] { + return pendingBySession.get(sessionID) ?? [] + } + + /** Return the single highest-precedence directive and clear ALL pending + * directives for the session — at most one directive block per turn. */ + export function take(sessionID: string): Directive | undefined { + const b = pendingBySession.get(sessionID) + if (!b || b.length === 0) return undefined + let winner: Directive | undefined + for (const source of PRECEDENCE) { + winner = b.find((d) => d.source === source) + if (winner) break + } + pendingBySession.delete(sessionID) + return winner + } + + export function clear(sessionID: string): void { + pendingBySession.delete(sessionID) + } +} diff --git a/packages/opencode/src/session/processor.ts b/packages/opencode/src/session/processor.ts index acc3235c61..ed3d172a30 100644 --- a/packages/opencode/src/session/processor.ts +++ b/packages/opencode/src/session/processor.ts @@ -19,6 +19,17 @@ import type { SessionID, MessageID } from "./schema" // altimate_change start — import Telemetry for per-generation token tracking import { Telemetry } from "@/altimate/telemetry" // altimate_change end +// altimate_change start — write-starvation breaker + loop detection (fork-only +// modules) and the run-mode flag that gates armed behavior. +import { SessionStarvation } from "./starvation" +import { NudgeArbiter } from "./nudge" +// completion-token contract for the explicit-DONE stop path +import { SessionTermination } from "./termination" +import { Flag } from "@/flag/flag" +// altimate_change end +// altimate_change start — per-tool-result dispatch cap (fork-only module) +import { ToolResultCap } from "./tool-result-cap" +// altimate_change end // altimate_change start — Effect Context.Service facade so the upstream Effect runtime // (app-runtime AppLayer + httpapi server LayerNode list) can compose SessionProcessor as // a Service. The fork keeps the imperative `create()` namespace function below; this is a @@ -39,15 +50,48 @@ export namespace SessionProcessor { export type Info = Awaited> export type Result = Awaited> + // altimate_change start — per-processor tool-call id coercer. Malformed + // (non-string) ids from OpenAI-compatible servers are regenerated deterministically + // via MessageV2.sanitizeToolCallID; the raw→sanitized alias map (keyed on the JSON + // form) makes the propagation to paired tool-result/tool-error events atomic — even + // when the provider flips the value's type mid-pair (numeric call id, string result + // id), both halves resolve to the SAME sanitized id. A regenerated call id with an + // un-regenerated result id would 400 every subsequent provider request. + // Exported as a factory so the ingestion half is unit-testable against the replay + // half in message-v2.ts (they must produce identical output for a pair). + // The alias table is a Map, never a plain object: adversarial ids like + // "__proto__"/"constructor"/"toString" hit inherited Object.prototype members + // on a plain-object index and return non-strings as the "sanitized id", + // erroring the stream loop. `salt` (per processor/step) keeps regenerated + // ids for empty/duplicate malformed raw values from colliding across steps. + export function createToolCallIDCoercer(salt?: string) { + const aliases = new Map() + return (raw: unknown): string => { + const key = typeof raw === "string" ? raw : (JSON.stringify(raw) ?? String(raw)) + const existing = aliases.get(key) + if (existing !== undefined) return existing + const sanitized = MessageV2.sanitizeToolCallID(raw, salt) + aliases.set(key, sanitized) + return sanitized + } + } + // altimate_change end + export function create(input: { assistantMessage: MessageV2.Assistant sessionID: SessionID model: Provider.Model abort: AbortSignal }) { - const toolcalls: Record = {} - // altimate_change start — per-tool call counter for varied-input loop detection - const toolCallCounts: Record = {} + // altimate_change start — Map (not plain object) so adversarial ids can + // never resolve to inherited Object.prototype members. + const toolcalls = new Map() + // coerce malformed tool-call ids at ingestion; sanitized ids are used as + // BOTH the persisted callID and the pairing key. Salted per processor so + // regenerated ids for empty/duplicate raw values cannot collide across steps. + const coerceToolCallID = createToolCallIDCoercer(input.assistantMessage.id) + // per-tool call counter for varied-input loop detection + const toolCallCounts = new Map() // altimate_change end let snapshot: string | undefined let blocked = false @@ -70,12 +114,97 @@ export namespace SessionProcessor { return input.assistantMessage }, partFromToolCall(toolCallID: string) { - return toolcalls[toolCallID] + // altimate_change start — tool-execution lookups use the same coercion + return toolcalls.get(coerceToolCallID(toolCallID)) + // altimate_change end }, async process(streamInput: LLM.StreamInput) { log.info("process") needsCompaction = false - const shouldBreak = (await Config.get()).experimental?.continue_loop_on_deny !== true + // altimate_change start — resolve breaker config + arm state once per step. + // ANNOTATE-ONLY by default (mode "annotate"): directives and the hard stop + // require mode "armed" AND run mode. Skipped entirely for plan/review-class + // agents (read-only deliverables are their normal outcome). Interactive + // TUI/serve sessions never set ALTIMATE_RUN_MODE, so they can at most + // receive informational annotations — never directives or stops. + const processConfig = await Config.get() + const shouldBreak = processConfig.experimental?.continue_loop_on_deny !== true + const sbConfig = SessionStarvation.resolveConfig( + processConfig.experimental?.starvation_breaker as SessionStarvation.ConfigShape | undefined, + ) + const runMode = Flag.ALTIMATE_RUN_MODE + // The compaction summarizer runs through this same processor under the + // session's OWN id, so it would otherwise share the working agent's + // per-session tracker: its single mutation-free step increments + // `turnsWithoutMutation`, inflating the real agent's counter (spurious + // would-fire telemetry in annotate mode, a premature directive in armed + // mode). It can only produce a summary, so it is exempt from starvation + // accounting entirely — the same reason it is excluded from directive + // delivery below. + const sbSummarizer = input.assistantMessage.summary === true + const sbExempt = sbConfig.exemptAgents.includes(input.assistantMessage.agent) || sbSummarizer + const starvation = + sbConfig.mode === "off" || sbExempt ? undefined : SessionStarvation.forSession(input.sessionID, sbConfig) + const sbArmed = sbConfig.mode === "armed" && runMode && !sbExempt + const sbMode = sbConfig.mode === "armed" ? ("armed" as const) : ("annotate" as const) + let starvationStop = false + // altimate_change start — per-tool-result dispatch cap, resolved once + // per step. Hard bound on the token estimate any single tool result may + // contribute to the conversation — closes the observed bypass where one + // giant query dump jumped a ~4K-token session past a 65K window in one step. + const toolResultCapTokens = ToolResultCap.resolve({ + config: processConfig, + model: input.model, + safetyFraction: SessionCompaction.contextSafetyFraction(processConfig), + }) + // altimate_change end + // Nudge arbiter delivery: at most ONE system-authored + // directive block per injected turn, highest precedence wins. Run-mode-only. + // altimate_change start — upstream_fix: never let the compaction summarizer + // consume a pending nudge/starvation/doom-loop directive — it can only + // produce a summary and cannot act on it, so the real working turn right + // after compaction would silently never see the breaker/loop nudge. + let effectiveStreamInput = streamInput + if (runMode && !input.assistantMessage.summary) { + const directive = NudgeArbiter.take(input.sessionID) + // altimate_change end + if (directive) { + // Attribute the injection to the DIRECTIVE that won arbitration, + // not a hardcoded "nudge" — otherwise every injected doom-loop + // status-check, starvation, and repeat-signature directive is + // indistinguishable in telemetry. + const telemetryKind = (() => { + if (directive.kind.startsWith("doom_loop")) return "doom_loop" as const + if (directive.kind === "repeat_signature") return "repeat_signature" as const + if (directive.kind === "starvation") return "starvation" as const + return "nudge" as const + })() + Telemetry.track({ + type: "starvation_breaker", + timestamp: Date.now(), + session_id: input.sessionID, + mode: sbMode, + kind: telemetryKind, + action: "injected", + }) + log.info("nudge arbiter directive injected", { + sessionID: input.sessionID, + source: directive.source, + kind: directive.kind, + }) + effectiveStreamInput = { + ...streamInput, + messages: [ + ...streamInput.messages, + { + role: "user" as const, + content: `\n${directive.text}\n`, + }, + ], + } + } + } + // altimate_change end while (true) { try { let currentText: MessageV2.TextPart | undefined @@ -85,7 +214,9 @@ export namespace SessionProcessor { // before the LLM stream can execute provider-side tools. snapshot = await Snapshot.track() } - const stream = await LLM.stream(streamInput) + // altimate_change start — stream with the (possibly directive-augmented) input + const stream = await LLM.stream(effectiveStreamInput) + // altimate_change end for await (const value of stream.fullStream) { input.abort.throwIfAborted() @@ -145,22 +276,26 @@ export namespace SessionProcessor { } break - case "tool-input-start": + // altimate_change start — sanitize the incoming id before it becomes the persisted callID and pairing key; braced so the consts do not leak into sibling clauses + case "tool-input-start": { + const inputStartCallID = coerceToolCallID(value.id) const part = await Session.updatePart({ - id: toolcalls[value.id]?.id ?? PartID.ascending(), + id: toolcalls.get(inputStartCallID)?.id ?? PartID.ascending(), messageID: input.assistantMessage.id, sessionID: input.assistantMessage.sessionID, type: "tool", tool: value.toolName, - callID: value.id, + callID: inputStartCallID, state: { status: "pending", input: {}, raw: "", }, }) - toolcalls[value.id] = part as MessageV2.ToolPart + toolcalls.set(inputStartCallID, part as MessageV2.ToolPart) break + } + // altimate_change end case "tool-input-delta": break @@ -169,7 +304,10 @@ export namespace SessionProcessor { break case "tool-call": { - const match = toolcalls[value.toolCallId] + // altimate_change start — resolve the pair via the coerced id + const toolCallCallID = coerceToolCallID(value.toolCallId) + const match = toolcalls.get(toolCallCallID) + // altimate_change end if (match) { const part = await Session.updatePart({ ...match, @@ -190,80 +328,212 @@ export namespace SessionProcessor { : value.providerMetadata, // altimate_change end }) - toolcalls[value.toolCallId] = part as MessageV2.ToolPart + // altimate_change start — key by the coerced id + toolcalls.set(toolCallCallID, part as MessageV2.ToolPart) + // altimate_change end // altimate_change start — session has now tool-called; suppresses plan refusal warning sessionToolCallsMade++ // altimate_change end - const parts = await MessageV2.parts(input.assistantMessage.id) - const lastThree = parts.slice(-DOOM_LOOP_THRESHOLD) + // altimate_change start — doom-loop guard re-keyed + escalation ladder. + // Interactive sessions keep the existing (toolName + identical args) + // permission ask EXACTLY as before. Run mode with the ladder ARMED + // replaces the permission channel with the ladder below (nudge → + // forced status-check → stop; never straight to stop). Run mode with + // the ladder NOT armed (default annotate mode) KEEPS the legacy ask: + // yolo auto-approves it (no-op there), but a non-yolo headless run + // auto-rejects it — the hard brake that previously converted an + // identical-args loop into a stop, which must not regress to + // telemetry-only during the annotate validation period. + if (!runMode || !sbArmed) { + const parts = await MessageV2.parts(input.assistantMessage.id) + const lastThree = parts.slice(-DOOM_LOOP_THRESHOLD) - if ( - lastThree.length === DOOM_LOOP_THRESHOLD && - lastThree.every( - (p) => - p.type === "tool" && - p.tool === value.toolName && - p.state.status !== "pending" && - JSON.stringify(p.state.input) === JSON.stringify(value.input), - ) - ) { - const agent = await Agent.get(input.assistantMessage.agent) - await PermissionNext.ask({ - permission: "doom_loop", - patterns: [value.toolName], - sessionID: input.assistantMessage.sessionID, - metadata: { - tool: value.toolName, - input: value.input, - }, - always: [value.toolName], - ruleset: agent.permission, - }) + if ( + lastThree.length === DOOM_LOOP_THRESHOLD && + lastThree.every( + (p) => + p.type === "tool" && + p.tool === value.toolName && + p.state.status !== "pending" && + JSON.stringify(p.state.input) === JSON.stringify(value.input), + ) + ) { + const agent = await Agent.get(input.assistantMessage.agent) + await PermissionNext.ask({ + permission: "doom_loop", + patterns: [value.toolName], + sessionID: input.assistantMessage.sessionID, + metadata: { + tool: value.toolName, + input: value.input, + }, + always: [value.toolName], + ruleset: agent.permission, + }) + } } + // altimate_change end - // altimate_change start — per-tool repeat counter (catches varied-input loops like todowrite 2,080x) - // Counter is scoped to the processor lifetime (create() call), so it accumulates - // across multiple process() invocations within a session. This is intentional: - // cross-turn accumulation catches slow-burn loops that stay under the threshold - // per-turn but add up over the session. - toolCallCounts[value.toolName] = (toolCallCounts[value.toolName] ?? 0) + 1 - if (toolCallCounts[value.toolName] >= TOOL_REPEAT_THRESHOLD) { + // altimate_change start — per-tool repeat counter, DEMOTED to telemetry only. + // The per-NAME counter (30 calls of any kind per tool) was crossed by + // legitimate multi-step work — attaching any hard consequence to it would + // kill ~half of legitimate work. It remains as telemetry; consequences + // hang off the (toolName + normalized args) ladder below instead. + toolCallCounts.set(value.toolName, (toolCallCounts.get(value.toolName) ?? 0) + 1) + if ((toolCallCounts.get(value.toolName) ?? 0) >= TOOL_REPEAT_THRESHOLD) { Telemetry.track({ type: "doom_loop_detected", timestamp: Date.now(), session_id: input.sessionID, tool_name: value.toolName, - repeat_count: toolCallCounts[value.toolName], + repeat_count: toolCallCounts.get(value.toolName) ?? 0, }) - const agent = await Agent.get(input.assistantMessage.agent) - await PermissionNext.ask({ - permission: "doom_loop", - patterns: [value.toolName], - sessionID: input.assistantMessage.sessionID, - metadata: { + toolCallCounts.set(value.toolName, 0) + } + // altimate_change end + + // altimate_change start — (toolName + normalized args) escalation ladder. + // Polling patterns (sleep/watch/status probes) get a raised threshold + // inside the tracker. Annotate mode only logs would-fire events; armed + // run mode registers outcome-neutral directives via the nudge arbiter + // and hard-stops only at the ladder's final rung. + if (starvation) { + const call = starvation.onToolCall({ tool: value.toolName, input: value.input }) + if (call.doomLoop) { + const wouldStop = call.doomLoop.escalation === "stop" + Telemetry.track({ + type: "starvation_breaker", + timestamp: Date.now(), + session_id: input.sessionID, + mode: sbMode, + kind: "doom_loop", + action: sbArmed ? (wouldStop ? "stop" : "registered") : "would_fire", + tool_name: value.toolName, + count: call.doomLoop.count, + escalation: call.doomLoop.escalation, + }) + log.warn("doom-loop ladder rung crossed", { + sessionID: input.sessionID, tool: value.toolName, - input: value.input, - repeat_count: toolCallCounts[value.toolName], - }, - always: [value.toolName], - ruleset: agent.permission, - }) - toolCallCounts[value.toolName] = 0 + count: call.doomLoop.count, + escalation: call.doomLoop.escalation, + armed: sbArmed, + }) + if (sbArmed) { + if (wouldStop) { + starvationStop = true + await Session.updatePart({ + id: PartID.ascending(), + messageID: input.assistantMessage.id, + sessionID: input.assistantMessage.sessionID, + type: "text", + synthetic: true, + text: + `altimate-code: stopping — the same \`${value.toolName}\` call with identical ` + + `arguments was repeated ${call.doomLoop.count} times despite a nudge and a ` + + `forced status-check (doom-loop escalation ladder, run mode).`, + time: { start: Date.now(), end: Date.now() }, + }) + } else { + NudgeArbiter.register(input.sessionID, { + source: "starvation_breaker", + kind: call.doomLoop.escalation === "nudge" ? "doom_loop_nudge" : "doom_loop_status_check", + text: call.doomLoop.directive, + }) + } + } + } } // altimate_change end } break } case "tool-result": { - const match = toolcalls[value.toolCallId] + // altimate_change start — resolve the pair via the coerced id + const toolResultCallID = coerceToolCallID(value.toolCallId) + const match = toolcalls.get(toolResultCallID) + // altimate_change end if (match && match.state.status === "running") { + // altimate_change start — unchanged-read annotation (content hash + // at read time; annotate, NEVER suppress — generated paths exempt) and + // repeat-signature loop detection on successful results. The annotation + // is appended to the persisted output in run mode only (interactive + // sessions get a telemetry-only shadow); the loop directive is + // arbiter-registered only when armed (run mode). + let toolResultOutput = value.output.output + if (starvation) { + const resultInput = value.input ?? match.state.input + const touched = (resultInput as any)?.filePath + const outcome = starvation.onToolResult({ + tool: match.tool, + input: resultInput, + output: typeof toolResultOutput === "string" ? toolResultOutput : undefined, + touchedFiles: typeof touched === "string" ? [touched] : undefined, + }) + if (outcome.readAnnotation && typeof toolResultOutput === "string") { + // Persisted-output mutation is run-mode-only: interactive + // (TUI/serve) sessions keep tool output byte-identical and + // get a telemetry-only shadow event instead. + toolResultOutput = SessionStarvation.applyReadAnnotation( + toolResultOutput, + outcome.readAnnotation, + runMode, + ) + Telemetry.track({ + type: "starvation_breaker", + timestamp: Date.now(), + session_id: input.sessionID, + mode: sbMode, + kind: "unchanged_read", + action: runMode ? "annotated" : "would_annotate", + tool_name: match.tool, + }) + } + if (outcome.repeatLoop) { + Telemetry.track({ + type: "starvation_breaker", + timestamp: Date.now(), + session_id: input.sessionID, + mode: sbMode, + kind: "repeat_signature", + action: sbArmed ? "registered" : "would_fire", + tool_name: match.tool, + count: outcome.repeatLoop.count, + }) + if (sbArmed) { + NudgeArbiter.register(input.sessionID, { + source: "starvation_breaker", + kind: "repeat_signature", + text: outcome.repeatLoop.directive, + }) + } + } + } + // altimate_change end + // altimate_change start — hard per-result dispatch cap. Every + // completed tool result is bounded here regardless of which tool + // path produced it — the tool-level truncation service can be + // bypassed, and one uncapped result overflows the whole window. + if (typeof toolResultOutput === "string") { + const capped = ToolResultCap.apply(toolResultOutput, toolResultCapTokens) + if (capped.truncated) { + toolResultOutput = capped.content + log.info("tool result capped at dispatch", { + tool: match.tool, + capTokens: toolResultCapTokens, + }) + } + } + // altimate_change end await Session.updatePart({ ...match, state: { status: "completed", input: value.input ?? match.state.input, - output: value.output.output, + // altimate_change start — annotated output (append-only) + output: toolResultOutput, + // altimate_change end metadata: value.output.metadata, title: value.output.title, time: { @@ -274,14 +544,53 @@ export namespace SessionProcessor { }, }) - delete toolcalls[value.toolCallId] + // altimate_change start — delete by the coerced id + toolcalls.delete(toolResultCallID) + // altimate_change end } break } case "tool-error": { - const match = toolcalls[value.toolCallId] + // altimate_change start — resolve the pair via the coerced id + const toolErrorCallID = coerceToolCallID(value.toolCallId) + const match = toolcalls.get(toolErrorCallID) + // altimate_change end if (match && match.state.status === "running") { + // altimate_change start — repeat-signature loop detection on + // failures — hash(tool + normalized args + touched files + failure + // message). Catches edit-verify-fail-revert-reedit loops that mutate + // files every turn but make no progress. + if (starvation) { + const errorInput = value.input ?? match.state.input + const touched = (errorInput as any)?.filePath + const outcome = starvation.onToolResult({ + tool: match.tool, + input: errorInput, + failureMessage: (value.error as any)?.toString?.() ?? String(value.error), + touchedFiles: typeof touched === "string" ? [touched] : undefined, + }) + if (outcome.repeatLoop) { + Telemetry.track({ + type: "starvation_breaker", + timestamp: Date.now(), + session_id: input.sessionID, + mode: sbMode, + kind: "repeat_signature", + action: sbArmed ? "registered" : "would_fire", + tool_name: match.tool, + count: outcome.repeatLoop.count, + }) + if (sbArmed) { + NudgeArbiter.register(input.sessionID, { + source: "starvation_breaker", + kind: "repeat_signature", + text: outcome.repeatLoop.directive, + }) + } + } + } + // altimate_change end await Session.updatePart({ ...match, state: { @@ -301,7 +610,9 @@ export namespace SessionProcessor { ) { blocked = shouldBreak } - delete toolcalls[value.toolCallId] + // altimate_change start — delete by the coerced id + toolcalls.delete(toolErrorCallID) + // altimate_change end } break } @@ -451,6 +762,11 @@ export namespace SessionProcessor { cost: usage.cost, }) await Session.updateMessage(input.assistantMessage) + // altimate_change start — capture the snapshot diff as the generic, + // command-agnostic mutation ground truth (also catches bash-mediated + // writes like `sed -i`/heredocs, which emit no edit event). + let stepPatchFiles: string[] = [] + // altimate_change end if (snapshot) { const patch = await Snapshot.patch(snapshot) if (patch.files.length) { @@ -463,8 +779,42 @@ export namespace SessionProcessor { files: patch.files, }) } + // altimate_change start + stepPatchFiles = [...patch.files] + // altimate_change end snapshot = undefined } + // altimate_change start — per-step write-starvation evaluation. + // Annotate mode only logs a would-fire event; armed run mode registers + // the outcome-neutral directive (with its DONE alternative) via the + // nudge arbiter for delivery on the next generation. + if (starvation) { + const stepOutcome = starvation.onStepFinish({ mutatedFiles: stepPatchFiles }) + if (stepOutcome.starvation) { + Telemetry.track({ + type: "starvation_breaker", + timestamp: Date.now(), + session_id: input.sessionID, + mode: sbMode, + kind: "starvation", + action: sbArmed ? "registered" : "would_fire", + turns_without_mutation: stepOutcome.turnsWithoutMutation, + }) + log.warn("write-starvation breaker", { + sessionID: input.sessionID, + turnsWithoutMutation: stepOutcome.turnsWithoutMutation, + armed: sbArmed, + }) + if (sbArmed) { + NudgeArbiter.register(input.sessionID, { + source: "starvation_breaker", + kind: "starvation", + text: stepOutcome.starvation.directive, + }) + } + } + } + // altimate_change end SessionSummary.summarize({ sessionID: input.sessionID, messageID: input.assistantMessage.parentID, @@ -658,9 +1008,49 @@ export namespace SessionProcessor { } input.assistantMessage.time.completed = Date.now() await Session.updateMessage(input.assistantMessage) - if (needsCompaction) return "compact" + // altimate_change start — explicit model DONE is the PRIMARY + // termination path. A turn that finished with "stop", has no error, and + // asserts completion (trailing DONE token per the SessionTermination + // contract) terminates the session EVEN IF overflow was detected. + // Returning "compact" here is the termination-impossibility triangle: + // the finished session gets summarized and the post-compaction continue + // message breeds further turns. Deferring compaction is safe in every + // mode — prompt.ts's pre-dispatch overflow check compacts before the + // next request. Never bare finishReason "stop" (that ends nearly every + // ordinary text turn), and never for the compaction summarizer itself. + if ( + needsCompaction && + !input.assistantMessage.summary && + SessionTermination.explicitDoneStop({ + finish: input.assistantMessage.finish, + hasError: input.assistantMessage.error !== undefined, + parts: p, + }) + ) { + log.info("explicit DONE with pending compaction — terminating instead of compacting", { + sessionID: input.sessionID, + messageID: input.assistantMessage.id, + }) + return "stop" + } + // altimate_change end + // altimate_change start — terminal outcomes take precedence over + // "compact": a blocked/errored/doom-loop-stopped turn must actually + // stop. Returning "compact" first made those stops no-ops under + // overflow — the session summarized and kept running. Deferring the + // compaction is safe: prompt.ts's pre-dispatch overflow check compacts + // before any next request. (Explicit DONE above still overrides + // compaction the same way.) if (blocked) return "stop" if (input.assistantMessage.error) return "stop" + // Doom-loop escalation ladder final rung. Reachable only when mode is + // "armed" AND the process is in run mode (never TUI/serve) AND the + // same (toolName + normalized args) call repeated through nudge and + // forced status-check without changing. + if (starvationStop) return "stop" + // Upstream's compact check, relocated below the terminal outcomes. + if (needsCompaction) return "compact" + // altimate_change end return "continue" } }, diff --git a/packages/opencode/src/session/prompt.ts b/packages/opencode/src/session/prompt.ts index 268babfc66..179d200334 100644 --- a/packages/opencode/src/session/prompt.ts +++ b/packages/opencode/src/session/prompt.ts @@ -1,4 +1,5 @@ import path from "path" +import { Token } from "@/util/token" import os from "os" import fs from "fs/promises" import z from "zod" @@ -812,16 +813,51 @@ export namespace SessionPrompt { auto: task.auto, overflow: task.overflow, }) - if (result === "stop") break + // altimate_change start — treat any non-"continue" result as stop: an + // undefined/unknown result must never fall through to `continue`, which + // re-enters compaction on the same unresolved marker and busy-loops. + if (result !== "continue") break + // altimate_change end continue } // context overflow, needs compaction + // altimate_change start — proactive overflow check: the recorded usage is + // from the LAST assistant turn; tool results appended since then are not + // counted, and one oversized output can jump the session past the window + // between checks (a common failure mode for long headless runs). Estimate the + // uncounted tail and include it. + const uncountedTail = (() => { + if (!lastFinished) return 0 + const index = msgs.findIndex((m) => m.info.id === lastFinished.id) + if (index < 0) return 0 + let tokens = 0 + for (const m of msgs.slice(index + 1)) { + for (const part of m.parts) { + if (part.type === "text") tokens += Token.estimate(part.text ?? "") + if (part.type === "tool" && part.state?.status === "completed") + tokens += Token.estimate(part.state.output ?? "") + } + } + return tokens + })() if ( lastFinished && lastFinished.summary !== true && - (await SessionCompaction.isOverflow({ tokens: lastFinished.tokens, model })) + (await SessionCompaction.isOverflow({ + tokens: lastFinished.tokens, + // Estimated component passed separately: the safety fraction applies + // only to it, never to the provider-reported usage above. + estimatedTokens: uncountedTail, + model, + })) ) { + // altimate_change end + // altimate_change start — task-pin livelock guard: record this + // auto-compaction so consecutive threshold-reduction failures halve the + // task pin instead of livelocking (fire → cannot reduce → re-fire). + SessionCompaction.notePinCompaction(sessionID, msgs) + // altimate_change end await SessionCompaction.create({ sessionID, agent: lastUser.agent, @@ -1384,7 +1420,7 @@ export namespace SessionPrompt { const validatorCount = ValidatorRegistry.list().length // Always emit to opencode's file log. Mirror to stderr only when // ALTIMATE_VALIDATORS_DEBUG=1 — needed during framework bring-up so - // benchmark harness logs capture the hook signal, but noisy enough + // automated harness logs capture the hook signal, but noisy enough // that we keep it off by default for normal sessions. const diag = { kind: "validator_hook_reached", @@ -1537,6 +1573,10 @@ export namespace SessionPrompt { // altimate_change start — track compaction count compactionCount++ // altimate_change end + // altimate_change start — task-pin livelock guard (see the + // proactive-overflow site above for rationale). + SessionCompaction.notePinCompaction(sessionID, msgs) + // altimate_change end await SessionCompaction.create({ sessionID, agent: lastUser.agent, @@ -2381,6 +2421,246 @@ export namespace SessionPrompt { } // altimate_change end + // altimate_change start — pin the original task + // verbatim through compaction. + // + // After compaction the model sees only a lossy summary of the task; the + // summarizer can drop or mutate literal contract terms + // (hallucinated table names, renamed output files). The pin re-injects the + // task instruction VERBATIM as a trusted reminder, labeled authoritative over + // any summary, and is hoisted into the system prompt on non-Anthropic models + // via the trustedReminderParts path below. + // + // Mode-aware pin selection: run mode (`run` CLI, CI/headless — signaled + // by the ALTIMATE_RUN_MODE marker, with ALTIMATE_NON_INTERACTIVE=1 — the same + // signal question.ts uses — as a fallback) pins the + // FIRST non-synthetic user message (the CLI task). Interactive sessions pin + // the MOST RECENT substantive user instruction — users pivot mid-session, and + // hoisting message #1 as "authoritative" would fight later redirections in + // exactly the long sessions that compact. A RESUMED run (`--continue`, + // `--session`, `--fork`) uses the interactive rule too: its history begins + // with an earlier invocation's task, so "first" would pin a stale request + // over the one this run supplied (see resolvePinRunMode). + // + // Budget: SessionCompaction.pinBudget — min(4k, ~17.5% of the post-overhead + // usable window), hard invariant pin + reserved + ≥2k slack < compaction + // threshold, halved by the livelock guard. When a task exceeds the budget we + // keep verbatim head+tail plus a deterministic ≤500-token contract card of + // regex-extracted literals. Never paraphrase. + + /** Exported for unit tests. Selects the message whose text gets pinned. */ + export function selectPinSource( + history: MessageV2.WithParts[], + runMode: boolean, + ): { id: MessageID; text: string } | undefined { + const candidates: { id: MessageID; text: string }[] = [] + for (const msg of history) { + if (msg.info.role !== "user") continue + if (msg.parts.some((p) => p.type === "compaction")) continue + const text = msg.parts + .filter((p): p is MessageV2.TextPart => p.type === "text" && !p.synthetic) + .map((p) => p.text) + .join("\n\n") + .trim() + if (!text) continue + candidates.push({ id: msg.info.id, text }) + } + if (!candidates.length) return undefined + return runMode ? candidates[0] : candidates[candidates.length - 1] + } + + // Deterministic contract card: regex-extracted literals from the task text + // (paths, identifier-shaped names, code spans, quoted terms, constraint + // lines), every entry a verbatim substring of the original — never a + // paraphrase. Patterns are GENERIC lexical shapes only; no vertical (dbt/ + // warehouse) tokens. Budget enforced by tail-truncation: + // stop adding once the cap is reached. + export function extractContractCard(text: string, capTokens: number): string { + if (capTokens <= 0) return "" + const seen = new Set() + const take = (raw: string | undefined) => { + const v = raw?.trim() + if (!v || v.length > 200 || seen.has(v)) return undefined + seen.add(v) + return v + } + const collect = (re: RegExp, group = 0) => { + const out: string[] = [] + for (const m of text.matchAll(re)) { + const v = take(m[group]) + if (v) out.push(v) + } + return out + } + // Paths: contain a slash, or bare filename with a dot-extension. + const paths = collect(/(?:[\w.@~-]+\/)+[\w.-]+|\b[\w-]+\.[A-Za-z]\w{0,7}\b/g) + // snake_case identifier shape (column/model/variable names) — generic. + const identifiers = collect(/\b[a-z][a-z0-9]*(?:_[a-z0-9]+)+\b/g) + // Inline code spans (commands, expressions). + const codeSpans = collect(/`([^`\n]{1,160})`/g, 1) + // Quoted terms. + const quoted: string[] = [] + for (const m of text.matchAll(/"([^"\n]{2,120})"|'([^'\n]{2,120})'/g)) { + const v = take(m[1] ?? m[2]) + if (v) quoted.push(v) + } + // Constraint/prohibition lines, kept verbatim in full. + const constraints: string[] = [] + for (const line of text.split("\n")) { + if (/\b(do not|don'?t|never|must(?: not)?|should not|shall not|avoid|only|require[sd]?|forbidden|prohibited)\b/i.test(line)) { + const v = take(line) + if (v) constraints.push(v) + } + } + + const header = "Contract card — literals extracted verbatim from the task (never paraphrased):" + if (Token.estimate(header) > capTokens) return "" + const out: string[] = [header] + // Budget enforced on the ACTUAL rendered card (labels and separators + // included), tail-truncating: stop adding once the cap would be exceeded. + const fits = (candidate: string[]) => Token.estimate(candidate.join("\n")) <= capTokens + const emitList = (label: string, items: string[]) => { + if (!items.length) return + let line = "" + for (const item of items) { + const next = line ? `${line}, ${item}` : `- ${label}: ${item}` + if (!fits([...out, next])) break + line = next + } + if (line) out.push(line) + } + emitList("files/paths", paths) + emitList("identifiers", identifiers) + emitList("code/commands", codeSpans) + emitList("quoted terms", quoted) + if (constraints.length) { + const kept: string[] = ["- constraints (verbatim lines):"] + for (const line of constraints) { + const next = [...kept, ` - ${line}`] + if (!fits([...out, ...next])) break + kept.push(` - ${line}`) + } + if (kept.length > 1) out.push(...kept) + } + if (out.length <= 1) return "" + return out.join("\n") + } + + /** + * Exported for unit tests. Returns the pin body: the task verbatim when it + * fits the cap; otherwise verbatim head+tail plus the contract card. + */ + export function buildPinnedTask(input: { text: string; capTokens: number; cardCapTokens: number }): string | undefined { + if (input.capTokens <= 0) return undefined + const text = input.text + if (Token.estimate(text) <= input.capTokens) return text + // Over cap: middle truncation alone deletes exactly the mid-prompt facts + // the evidence shows decaying — pair verbatim head+tail with the card. + const cardCap = Math.min(input.cardCapTokens, Math.floor(input.capTokens / 2)) + const card = extractContractCard(text, cardCap) + const marker = "\n\n[... middle of the original task truncated — literal terms preserved in the contract card below ...]\n\n" + const bodyBudget = input.capTokens - Token.estimate(card) - Token.estimate(marker) - 8 + if (bodyBudget <= 0) return card || undefined + // Token.estimate is ratio-based; shrink the char budget geometrically until + // the assembled result fits. Deterministic for a given input. + let charBudget = Math.floor(bodyBudget * 3.7) + while (charBudget >= 100) { + const half = Math.floor(charBudget / 2) + const candidate = text.slice(0, half) + marker + text.slice(text.length - half) + (card ? "\n\n" + card : "") + if (Token.estimate(candidate) <= input.capTokens) return candidate + charBudget = Math.floor(charBudget * 0.85) + } + return card || undefined + } + + /** + * Exported for unit tests (mid-session-redirect case is covered here without + * DB fixtures). Pure: assembles the labeled reminder text from the full + * chronological history and the currently visible (compaction-filtered) + * messages, or returns undefined when no pin should be injected. + */ + export function taskPinText(input: { + history: MessageV2.WithParts[] + visible: MessageV2.WithParts[] + runMode: boolean + capTokens: number + cardCapTokens: number + }): string | undefined { + const source = selectPinSource(input.history, input.runMode) + if (!source) return undefined + // Skip while the source message is still in visible context verbatim — the + // pin exists to survive compaction, not to duplicate live messages. + if (input.visible.some((m) => m.info.id === source.id)) return undefined + const body = buildPinnedTask({ text: source.text, capTokens: input.capTokens, cardCapTokens: input.cardCapTokens }) + if (!body) return undefined + return [ + "", + "Original task — authoritative over any summary. The conversation above was compacted into a summary; the task below is the user's own instruction, reproduced verbatim. If the summary and this task conflict, this task wins.", + "", + body, + "", + ].join("\n") + } + + /** + * Run mode = the dedicated ALTIMATE_RUN_MODE marker (set by run.ts, never by + * TUI/serve), with ALTIMATE_NON_INTERACTIVE=1 as a fallback signal for + * headless drivers that predate the marker. An EXPLICITLY set run-mode value + * always wins — a user exporting ALTIMATE_RUN_MODE=0 has opted out of + * run-mode semantics and must not be flipped back by the legacy fallback; + * the fallback applies only when the marker is undefined/blank. + * Exported for unit tests. + */ + export function resolvePinRunMode(env: Record = process.env): boolean { + // A resumed run (`--continue` / `--session` / `--fork`, marked by run.ts) + // carries earlier invocations' messages, so "first user message" is a + // previous task, not this run's. Those sessions use interactive selection — + // the latest substantive instruction — which is the request this invocation + // actually supplied. + if (env["ALTIMATE_RUN_RESUMED"] === "1") return false + if (env["ALTIMATE_RUN_MODE"]?.trim()) return Flag.parseRunModeValue(env["ALTIMATE_RUN_MODE"]) + return env["ALTIMATE_NON_INTERACTIVE"] === "1" + } + + // Compaction-gated entry point used by insertReminders: fires only when the + // visible context already contains a completed summary, the pin budget is + // positive, and the pinned source message is no longer visible. + async function taskPinReminder(input: { + visible: MessageV2.WithParts[] + session: Session.Info + model: Provider.Model + }): Promise { + // Compaction gate FIRST — it is pure message inspection, so never-compacted + // sessions (the common path) pay no Config/DB cost here. + const compacted = input.visible.some( + (m) => m.info.role === "assistant" && m.info.summary && m.info.finish && !m.info.error, + ) + if (!compacted) return undefined + // Fail-safe from here on: the pin is an additive reminder — a session that + // cannot compute it must still run the turn. + try { + const cfg = await Config.get() + if (!SessionCompaction.pinEnabled(cfg)) return undefined + const cap = SessionCompaction.pinBudget({ cfg, model: input.model, sessionID: input.session.id }) + if (cap <= 0) return undefined + // Full chronological history — the pinned source was dropped from the + // compaction-filtered view, which is exactly why it must be re-read here. + const history = [...MessageV2.stream(input.session.id)].reverse() + const runMode = resolvePinRunMode() + return taskPinText({ + history, + visible: input.visible, + runMode, + capTokens: cap, + cardCapTokens: SessionCompaction.pinCardBudget(cfg), + }) + } catch (e) { + log.warn("task pin skipped", { error: e instanceof Error ? e.message : String(e) }) + return undefined + } + } + // altimate_change end + // altimate_change start — return the trusted reminder parts insertReminders just appended // so the caller can hoist them into the system prompt on non-Anthropic models. // The returned-parts contract is the trust boundary: only parts that *this function* @@ -2413,6 +2693,29 @@ export namespace SessionPrompt { const userMessage = input.messages.findLast((msg) => msg.info.role === "user") if (!userMessage) return { messages: input.messages, trustedReminderParts } + // altimate_change start — pin the original task + // verbatim through compaction, hoisted via the trustedReminderParts path + // and labeled "Original task — authoritative over any summary". The pin + // text embeds the user's OWN instruction verbatim — the user's directive, + // not third-party file/resource content — so promoting it through + // trustedReminderParts does not cross the trust boundary documented above. + // Not persisted: recomputed per turn, like the plan reminder below. + const pinText = await taskPinReminder({ visible: input.messages, session: input.session, model: input.model }) + if (pinText) { + const part: MessageV2.TextPart = { + id: PartID.ascending(), + messageID: userMessage.info.id, + sessionID: userMessage.info.sessionID, + type: "text", + text: pinText, + synthetic: true, + ...(nonAnthropic ? { ignored: true } : {}), + } + userMessage.parts.push(part) + trustedReminderParts.push(part) + } + // altimate_change end + // Original logic when experimental plan mode is disabled if (!Flag.OPENCODE_EXPERIMENTAL_PLAN_MODE) { if (input.agent.name === "plan") { diff --git a/packages/opencode/src/session/starvation.ts b/packages/opencode/src/session/starvation.ts new file mode 100644 index 0000000000..b14a0a62ed --- /dev/null +++ b/packages/opencode/src/session/starvation.ts @@ -0,0 +1,551 @@ +// Fork-only module — write-starvation circuit breaker, signature-hash loop +// detection, unchanged-read annotation, and the re-keyed doom-loop escalation +// ladder. +// +// Design constraints: +// - ANNOTATE-ONLY BY DEFAULT: directive injection and any hard consequence are +// config-gated OFF (`mode: "annotate"`) until validation shows no session +// class regresses. In annotate mode the harness only logs +// breaker-would-fire events and appends informational annotations. +// - Directives are OUTCOME-NEUTRAL and always carry a DONE alternative — never +// an unconditional "produce the edit now" (fabricated-edit risk on read-only +// / review / analysis tasks). +// - GENERIC classifiers only. Mutation evidence comes from file-mutation tool +// completions and the harness snapshot diff (patch parts) — no command-string +// matching, no vertical/tool-vendor tokens anywhere in this file (enforced +// by a source-scan guard in the unit tests). +// - Unchanged-read detection is by CONTENT HASH at read time; generated paths +// are exempt; the annotation NEVER suppresses content. +// - Doom-loop counting is keyed on (toolName + normalized args) — the legacy +// per-NAME counter is telemetry only (legitimate multi-step work routinely +// crosses a name-only counter). Escalation ladder: nudge → forced +// status-check → stop; never straight to stop. +// - Armed behavior is run-mode-only and skipped for plan/review-class agents; +// directive delivery goes through the NudgeArbiter (one directive per turn). +import { createHash } from "node:crypto" + +export namespace SessionStarvation { + export type Mode = "off" | "annotate" | "armed" + + export interface ConfigShape { + mode?: Mode + max_turns_without_mutation?: number + repeat_signature_threshold?: number + doom_loop_threshold?: number + polling_threshold_multiplier?: number + polling_pattern?: string + exempt_agents?: string[] + generated_path_patterns?: string[] + } + + export interface ResolvedConfig { + mode: Mode + maxTurnsWithoutMutation: number + repeatSignatureThreshold: number + doomLoopThreshold: number + pollingThresholdMultiplier: number + pollingPattern: string + exemptAgents: string[] + generatedPathPatterns: string[] + } + + // Threshold rationale (config-exposed defaults, never fitted to any one + // workload): + // - doomLoopThreshold = 3: matches the pre-existing upstream DOOM_LOOP_THRESHOLD; + // a legitimate edit→verify cycle takes only a couple of tool calls, so 3 + // consecutive byte-identical (tool+args) calls sits outside any + // legitimate cycle shape. + // - repeatSignatureThreshold = 3: three identical (tool+args+touched-files+ + // failure) signatures means three attempts produced the same failure — + // repeating the call cannot change the outcome. + // - maxTurnsWithoutMutation = 12: counted per GENERATION STEP (onStepFinish + // is called once per model step, and one user message routinely spans + // several read-only steps) — not per user message. Legitimate exploration + // bursts (read/search before a first edit or a final answer) span a + // handful of steps; 12 consecutive steps with zero corroborated file + // mutation is well beyond that regime while still permitting long + // read-only research tasks to proceed (the directive is outcome-neutral). + // - pollingThresholdMultiplier = 5: identical polling commands (sleep/watch/ + // status probes) are legitimately repetitive; raising, not exempting, + // keeps a ceiling on unbounded polling loops. + export const DEFAULTS: ResolvedConfig = { + mode: "annotate", + maxTurnsWithoutMutation: 12, + repeatSignatureThreshold: 3, + doomLoopThreshold: 3, + pollingThresholdMultiplier: 5, + pollingPattern: "\\b(sleep|watch|status)\\b", + exemptAgents: ["plan", "review"], + // Generated/regenerating artifacts: re-reading these is expected to see new + // content on every build, so unchanged-read annotation must not fire. + generatedPathPatterns: [ + "target/", + "dist/", + "build/", + "out/", + "node_modules/", + ".git/", + "__pycache__/", + "*.log", + "*.db", + "*.duckdb", + "*.sqlite", + ], + } + + // altimate_change start — upstream_fix: a configured 0 (commonly meant as + // "off") on any of these made the breaker fire on the very first tool call — + // `consecutiveIdenticalCalls >= threshold * 3` is true at threshold 0, and a + // 0 multiplier zeroes the polling threshold too. Disabling starvation must go + // through `mode: "off"` only; clamp everything else to >= 1. + function positive(value: number | undefined, fallback: number): number { + return typeof value === "number" && Number.isFinite(value) && value >= 1 ? Math.floor(value) : fallback + } + // altimate_change end + + export function resolveConfig(cfg: ConfigShape | undefined): ResolvedConfig { + return { + mode: cfg?.mode ?? DEFAULTS.mode, + maxTurnsWithoutMutation: positive(cfg?.max_turns_without_mutation, DEFAULTS.maxTurnsWithoutMutation), + repeatSignatureThreshold: positive(cfg?.repeat_signature_threshold, DEFAULTS.repeatSignatureThreshold), + doomLoopThreshold: positive(cfg?.doom_loop_threshold, DEFAULTS.doomLoopThreshold), + pollingThresholdMultiplier: positive(cfg?.polling_threshold_multiplier, DEFAULTS.pollingThresholdMultiplier), + pollingPattern: cfg?.polling_pattern ?? DEFAULTS.pollingPattern, + exemptAgents: cfg?.exempt_agents ?? DEFAULTS.exemptAgents, + generatedPathPatterns: cfg?.generated_path_patterns ?? DEFAULTS.generatedPathPatterns, + } + } + + // --------------------------------------------------------------------------- + // Generic classifiers — NO vertical tokens (hard requirement: keep these domain-neutral). + // --------------------------------------------------------------------------- + + // Tools whose successful completion IS file mutation (harness-corroborated by + // construction). Bash-mediated mutations (sed -i, heredocs) are corroborated + // separately via the step snapshot diff (patch part files) in onStepFinish. + const FILE_MUTATION_TOOLS = new Set(["write", "edit", "apply_patch", "patch", "multiedit"]) + + // Tools that can never mutate the workspace. Anything else ("bash", MCP tools, + // unknown tools) classifies as "unknown" — ground truth for those comes from + // the snapshot diff, never from parsing command strings. + const READ_ONLY_TOOLS = new Set([ + "read", + "glob", + "grep", + "list", + "codesearch", + "webfetch", + "websearch", + "skill", + "todoread", + "question", + "lsp", + ]) + + export type CallClass = "mutating" | "read-only" | "unknown" + + export function classifyToolCall(tool: string): CallClass { + if (FILE_MUTATION_TOOLS.has(tool)) return "mutating" + if (READ_ONLY_TOOLS.has(tool)) return "read-only" + return "unknown" + } + + export function isGeneratedPath(filePath: string, patterns: string[]): boolean { + const normalized = filePath.replaceAll("\\", "/") + for (const pattern of patterns) { + if (pattern.endsWith("/")) { + if (normalized.includes(`/${pattern}`) || normalized.startsWith(pattern)) return true + continue + } + if (pattern.startsWith("*.")) { + if (normalized.endsWith(pattern.slice(1))) return true + continue + } + if (normalized.includes(pattern)) return true + } + return false + } + + /** Deterministic, order-insensitive stringification of tool args. */ + export function normalizeArgs(input: unknown): string { + const seen = new Set() + function norm(value: unknown): unknown { + if (value === null || typeof value !== "object") { + if (typeof value === "string") return value.replace(/\s+/g, " ").trim() + return value + } + if (seen.has(value)) return "[circular]" + // altimate_change start — upstream_fix: track the CURRENT recursion path, + // not every object ever visited — a shared (non-circular) reference in a + // DAG-shaped input was mislabeled "[circular]" because it stayed in + // `seen` after its subtree finished. Remove on the way back out. + seen.add(value) + try { + if (Array.isArray(value)) return value.map(norm) + const out: Record = {} + for (const key of Object.keys(value as Record).sort()) { + out[key] = norm((value as Record)[key]) + } + return out + } finally { + seen.delete(value) + } + // altimate_change end + } + return JSON.stringify(norm(input)) + } + + function sha(text: string): string { + return createHash("sha256").update(text).digest("hex") + } + + /** repeat_signature = hash(tool + normalized args + touched files + failure message). + * Catches edit-verify-fail-revert-reedit loops that mutate files every turn but + * make no progress — invisible to zero-mutation counting. */ + export function repeatSignature(input: { + tool: string + args: unknown + touchedFiles?: string[] + failureMessage?: string + }): string { + return sha( + [ + input.tool, + normalizeArgs(input.args), + [...(input.touchedFiles ?? [])].sort().join(","), + (input.failureMessage ?? "").replace(/\s+/g, " ").trim(), + ].join("\u0000"), + ) + } + + // --------------------------------------------------------------------------- + // Directive text — outcome-neutral, always with a DONE alternative. + // --------------------------------------------------------------------------- + + export function starvationDirective(input: { + turnsWithoutMutation: number + topReadPath?: string + topReadCount?: number + }): string { + const readClause = + input.topReadPath && (input.topReadCount ?? 0) > 1 + ? `; you have already read ${input.topReadPath} ${input.topReadCount} times` + : "" + return ( + `You have taken ${input.turnsWithoutMutation} turns without modifying any file${readClause}. ` + + `If this task requires an edit, produce it now; if the correct deliverable is analysis with no ` + + `file changes, state your final answer and say DONE.` + ) + } + + /** + * Run-mode gate for ANY persisted-output mutation. Interactive (TUI/serve) + * sessions must see tool output byte-identical to what the tool produced — + * they get a telemetry-only shadow instead of an appended annotation. + */ + export function applyReadAnnotation(output: string, annotation: string, runMode: boolean): string { + if (!runMode) return output + return `${output}\n\n${annotation}` + } + + export function repeatSignatureDirective(input: { count: number; tool: string }): string { + return ( + `Your last ${input.count} \`${input.tool}\` attempts had identical inputs and identical outcomes. ` + + `Repeating the same call again will not change the result. Diagnose why the previous attempts did ` + + `not achieve the goal and take a different action; if the deliverable is already complete, state ` + + `your final answer and say DONE.` + ) + } + + export function doomLoopNudgeDirective(input: { count: number; tool: string }): string { + return ( + `You have issued the same \`${input.tool}\` call with identical arguments ${input.count} times in a row. ` + + `If a different action is needed, take it now; if the deliverable is already complete, state your ` + + `final answer and say DONE.` + ) + } + + export function doomLoopStatusDirective(input: { count: number; tool: string }): string { + return ( + `You have repeated the same \`${input.tool}\` call ${input.count} times. Before any further tool ` + + `calls, produce a status check: (1) what you are trying to accomplish, (2) what the repeated call ` + + `returned, (3) why the next action will produce a different result. Then take that different ` + + `action — or, if the deliverable is already complete, state your final answer and say DONE.` + ) + } + + // --------------------------------------------------------------------------- + // Tracker — session-scoped state machine. Pure with respect to the harness: + // callers feed it events; it returns what (if anything) would fire. + // --------------------------------------------------------------------------- + + export type DoomEscalation = "nudge" | "status_check" | "stop" + + export interface CallResult { + class: CallClass + /** Present when the (tool + normalized args) consecutive-repeat ladder crossed a rung. */ + doomLoop?: { escalation: DoomEscalation; count: number; threshold: number; directive: string } + } + + export interface ResultOutcome { + /** Informational annotation to APPEND to the tool output (never replaces it). */ + readAnnotation?: string + /** Present when the repeat-signature loop detector crossed its threshold. */ + repeatLoop?: { count: number; signature: string; directive: string } + } + + export interface StepOutcome { + turnsWithoutMutation: number + /** Present when the write-starvation breaker would fire this turn. */ + starvation?: { directive: string } + } + + export interface Stats { + step: number + turnsWithoutMutation: number + firstMutationStep: number | undefined + toolCalls: number + mutatingCalls: number + unchangedReads: number + } + + export function createTracker(config: ResolvedConfig) { + let step = 1 + let turnsWithoutMutation = 0 + let stepSawMutation = false + let firstMutationStep: number | undefined + let toolCalls = 0 + let mutatingCalls = 0 + let unchangedReads = 0 + + // Doom-loop ladder state — keyed on (tool + normalized args). + let lastCallKey: string | undefined + let consecutiveIdenticalCalls = 0 + + // Repeat-signature loop state — consecutive identical signatures. + let lastSignature: string | undefined + let consecutiveIdenticalSignatures = 0 + + // Read tracking: path → content hash + counts. + const reads = new Map() + + const pollingRegex = (() => { + try { + return new RegExp(config.pollingPattern, "i") + } catch { + return new RegExp(DEFAULTS.pollingPattern, "i") + } + })() + + function markMutation() { + stepSawMutation = true + firstMutationStep ??= step + } + + function topRead(): { path: string; count: number } | undefined { + let best: { path: string; count: number } | undefined + for (const [path, entry] of reads) { + if (!best || entry.count > best.count) best = { path, count: entry.count } + } + return best + } + + return { + get config() { + return config + }, + + onToolCall(input: { tool: string; input: unknown }): CallResult { + toolCalls++ + const klass = classifyToolCall(input.tool) + if (klass === "mutating") { + mutatingCalls++ + // No mutation credit here: the call has not succeeded yet. Credit is + // granted on successful completion (onToolResult) or snapshot-diff + // evidence (onStepFinish) — a failed edit must not reset the + // starvation counter. + } + + const key = `${input.tool}\u0000${normalizeArgs(input.input)}` + if (key === lastCallKey) consecutiveIdenticalCalls++ + else { + lastCallKey = key + consecutiveIdenticalCalls = 1 + } + + // Polling patterns (identical sleep/watch/status probes) get a raised + // threshold, not an exemption — a ceiling still exists. + const command = + input.input && typeof input.input === "object" && typeof (input.input as any).command === "string" + ? ((input.input as any).command as string) + : undefined + const polling = command !== undefined && pollingRegex.test(command) + const threshold = polling ? config.doomLoopThreshold * config.pollingThresholdMultiplier : config.doomLoopThreshold + + let escalation: DoomEscalation | undefined + if (consecutiveIdenticalCalls >= threshold * 3) escalation = "stop" + else if (consecutiveIdenticalCalls === threshold * 2) escalation = "status_check" + else if (consecutiveIdenticalCalls === threshold) escalation = "nudge" + + if (escalation === "stop") { + // Latch: the stop fires exactly once per completed ladder run — the + // count resets so (a) further identical calls in the same stopping + // step cannot re-fire it (directive/part spam), and (b) a retried + // session starts with a cleared ladder and full runway instead of an + // instant stop on its first repeated call. + const count = consecutiveIdenticalCalls + consecutiveIdenticalCalls = 0 + return { + class: klass, + doomLoop: { + escalation, + count, + threshold, + directive: doomLoopStatusDirective({ count, tool: input.tool }), + }, + } + } + + if (!escalation) return { class: klass } + // "stop" returned above; only nudge/status_check reach here. + const directive = + escalation === "status_check" + ? doomLoopStatusDirective({ count: consecutiveIdenticalCalls, tool: input.tool }) + : doomLoopNudgeDirective({ count: consecutiveIdenticalCalls, tool: input.tool }) + return { + class: klass, + doomLoop: { escalation, count: consecutiveIdenticalCalls, threshold, directive }, + } + }, + + onToolResult(input: { + tool: string + input: unknown + output?: string + failureMessage?: string + touchedFiles?: string[] + }): ResultOutcome { + const outcome: ResultOutcome = {} + + // Successful file-mutation tool completions are corroborated mutations. + if (classifyToolCall(input.tool) === "mutating" && input.failureMessage === undefined) markMutation() + + // Unchanged-read annotation — content hash at read time; annotate, never + // suppress. Generated paths are exempt (they legitimately change or are + // re-read across builds). + if (input.tool === "read" && input.failureMessage === undefined && typeof input.output === "string") { + const filePath = + input.input && typeof input.input === "object" && typeof (input.input as any).filePath === "string" + ? ((input.input as any).filePath as string) + : undefined + if (filePath !== undefined) { + const hash = sha(input.output) + const prior = reads.get(filePath) + if (prior === undefined) { + reads.set(filePath, { hash, count: 1, lastStep: step, firstStep: step }) + } else { + const unchanged = prior.hash === hash + const priorStep = prior.lastStep + prior.hash = hash + prior.count++ + prior.lastStep = step + if (unchanged && !isGeneratedPath(filePath, config.generatedPathPatterns)) { + unchangedReads++ + outcome.readAnnotation = + `[harness note: ${filePath} is unchanged since you read it at turn ${priorStep} ` + + `(identical content hash); this is read #${prior.count} of this file in this session.]` + } + } + } + } + + // Repeat-signature loop detection. + const signature = repeatSignature({ + tool: input.tool, + args: input.input, + touchedFiles: input.touchedFiles, + failureMessage: input.failureMessage, + }) + if (signature === lastSignature) consecutiveIdenticalSignatures++ + else { + lastSignature = signature + consecutiveIdenticalSignatures = 1 + } + if ( + consecutiveIdenticalSignatures >= config.repeatSignatureThreshold && + (consecutiveIdenticalSignatures - config.repeatSignatureThreshold) % config.repeatSignatureThreshold === 0 + ) { + outcome.repeatLoop = { + count: consecutiveIdenticalSignatures, + signature, + directive: repeatSignatureDirective({ count: consecutiveIdenticalSignatures, tool: input.tool }), + } + } + + return outcome + }, + + /** Called once per assistant step with the snapshot-diff evidence (patch + * part files) — the generic, command-agnostic mutation ground truth that + * also catches bash-mediated writes (sed -i, heredocs). */ + onStepFinish(input: { mutatedFiles: string[] }): StepOutcome { + if (input.mutatedFiles.length > 0) markMutation() + if (stepSawMutation) turnsWithoutMutation = 0 + else turnsWithoutMutation++ + stepSawMutation = false + step++ + + const outcome: StepOutcome = { turnsWithoutMutation } + const t = config.maxTurnsWithoutMutation + // Fire at the threshold, then re-fire every `threshold` turns — not every + // turn (directive spam would drown the model's own reasoning). + if (turnsWithoutMutation >= t && (turnsWithoutMutation - t) % t === 0) { + const top = topRead() + outcome.starvation = { + directive: starvationDirective({ + turnsWithoutMutation, + topReadPath: top?.path, + topReadCount: top?.count, + }), + } + } + return outcome + }, + + stats(): Stats { + return { step, turnsWithoutMutation, firstMutationStep, toolCalls, mutatingCalls, unchangedReads } + }, + } + } + + export type Tracker = ReturnType + + // Session-scoped store — trackers must survive across processor instances + // (SessionProcessor.create runs once per step). Bounded for long-lived servers. + const MAX_SESSIONS = 128 + const trackers = new Map() + + export function forSession(sessionID: string, config: ResolvedConfig): Tracker { + let tracker = trackers.get(sessionID) + if (!tracker) { + if (trackers.size >= MAX_SESSIONS) { + // Evict the LEAST-RECENTLY-USED session (front of the Map after the + // refresh-on-access below), never the oldest-created — the longest-running + // active session is exactly the one accumulating escalation-ladder state + // and must not be silently reset by churn from short-lived sessions. + const oldest = trackers.keys().next().value + if (oldest !== undefined) trackers.delete(oldest) + } + tracker = createTracker(config) + } else { + // Refresh recency: re-insert so Map iteration order tracks last access. + trackers.delete(sessionID) + } + trackers.set(sessionID, tracker) + return tracker + } + + export function clear(sessionID: string): void { + trackers.delete(sessionID) + } +} diff --git a/packages/opencode/src/session/termination.ts b/packages/opencode/src/session/termination.ts new file mode 100644 index 0000000000..3b8feba2ea --- /dev/null +++ b/packages/opencode/src/session/termination.ts @@ -0,0 +1,141 @@ +// Fork-only module — real session termination path. +// +// This module owns the COMPLETION-TOKEN CONTRACT: the post-compaction +// nudge and the idle-done confirm challenge both instruct the model to assert +// completion with a literal trailing `DONE`, and `isExplicitDone()` is the single +// detector every consumer (processor stop-path, run-mode accounting, idle-done +// challenge evaluation) must use, so the instruction and the detection can never +// drift apart. +// +// "Finished naturally" REQUIRES finishReason "stop" PLUS an explicit +// completion assertion in the final text — never bare "stop", which ends nearly +// every ordinary text turn ("Let me now read the schema file." finishes with +// stop). Explicit model DONE is the PRIMARY termination path; the run-mode +// idle-done heuristic (cli/cmd/idle-done.ts) is a fallback only. +// +// Directive texts live here (not at call sites) so any wording-change review +// covers ONE file, and both texts stay consistent with the +// detector. Delivery goes through the NudgeArbiter (session/nudge.ts): +// at most one system-authored directive block per injected turn, +// termination_challenge > starvation_breaker > budget_reminder. + +export namespace SessionTermination { + /** The literal completion token the nudge/challenge instruct the model to emit. */ + export const DONE_TOKEN = "DONE" + + // Standalone final plaintext line only. The earlier trailing-token regex + // accepted code-fenced, inline-code, indented, and quoted text whose content + // happened to end in DONE — demonstration text could be classified as + // completion. The detector now requires the FINAL line (after stripping + // trailing whitespace) to be exactly the token: not inside an unclosed code + // fence, not markdown-indented code (>= 4 leading spaces or a tab), not a + // `>` quote, not wrapped in backticks or other markup, no punctuation. + // Case-sensitive so prose "done" never counts. + const CODE_FENCE_PATTERN = /^ {0,3}(`{3,}|~{3,})/ + + /** True when the text ends with an explicit completion assertion (see module header). */ + export function isExplicitDone(text: string): boolean { + // Normalize line endings FIRST. On CRLF input the interior lines keep a + // trailing `\r`, which fails the closing fence's whitespace-only check and + // leaves every fence permanently open (a genuine DONE is then rejected); + // on bare-CR input the text never splits at all. + const lines = text + .replace(/\r\n?/g, "\n") + .replace(/\s+$/, "") + .split("\n") + const last = lines[lines.length - 1] + if (last === undefined) return false + // Markdown-indented code (4+ spaces or a tab) is demonstration text. + if (/^(?: {4,}|\t)/.test(last)) return false + // Up to 3 leading spaces is plain text in Markdown; anything else must match exactly. + if (last.replace(/^ {0,3}/, "") !== DONE_TOKEN) return false + // Reject a final line inside an unclosed code fence — the block's content is + // quoted material, not an assertion. Fence state follows CommonMark: a fence + // opens with a run of >= 3 backticks or tildes (an info string, e.g. an + // opening ```lang, is permitted); only a run of the SAME character with at + // least the SAME length, followed by nothing but optional whitespace, closes + // it. A fence-looking line with a trailing info string is opener/content, + // never a valid closer — treating it as one would let a still-open fence's + // interior DONE terminate the run. + let open: { char: string; length: number } | undefined + for (let i = 0; i < lines.length - 1; i++) { + const match = CODE_FENCE_PATTERN.exec(lines[i]!) + if (!match) continue + const marker = match[1]! + const rest = lines[i]!.slice(match[0]!.length) + if (!open) { + // CommonMark: a backtick fence's info string may not contain a + // backtick. Such a line is ordinary paragraph text, so treating it as + // an opener would make a later backtick run look like its closer and + // expose the interior — including a demonstration DONE — as an + // assertion. + if (marker[0] === "`" && rest.includes("`")) continue + open = { char: marker[0]!, length: marker.length } + } else if ( + marker[0] === open.char && + marker.length >= open.length && + /^[ \t]*$/.test(rest) + ) { + open = undefined + } + } + return open === undefined + } + + /** + * Stop-path decision: should a turn that would otherwise trigger + * compaction terminate the session instead? True only for an errorless turn + * that finished with "stop" AND asserted completion in its final real + * (non-synthetic) text part. Returning "compact" for such a turn is the + * termination-impossibility triangle: the finished session gets summarized and + * the post-compaction continue message breeds further turns forever. Deferring + * the compaction is safe in every mode — the pre-dispatch overflow check in + * prompt.ts compacts before the next request is sent. + */ + export function explicitDoneStop(input: { + finish: string | undefined + hasError: boolean + parts: readonly { type: string; synthetic?: boolean; text?: string }[] + }): boolean { + if (input.hasError) return false + if (input.finish !== "stop") return false + const lastText = input.parts.findLast((part) => part.type === "text" && part.synthetic !== true) + if (!lastText?.text) return false + return isExplicitDone(lastText.text) + } + + /** + * Three-option completion-aware post-compaction nudge. Replaces the + * two-option "Continue … or stop and ask for clarification" text, which gave a + * finished session no way to terminate. Prompt-visible text — changes need + * extra review. + */ + export const COMPLETION_NUDGE = + "Context was compacted; the summary above is the record of the work so far. Choose exactly one: " + + "(1) if concrete next steps remain toward the original task, continue with them; " + + "(2) if you are blocked or unsure how to proceed, stop and ask for clarification; " + + `(3) if the deliverable is complete and verified, reply with ${DONE_TOKEN} alone on the final line and stop.` + + /** + * One-shot confirm-DONE challenge injected by the run-mode + * idle-done fallback before it may end a session. The session exits as done + * only on confirmation; otherwise the model states what remains and continues. + */ + export const CONFIRM_DONE_CHALLENGE = + "Completion check: the most recent verification succeeded after your last file change and no further " + + "actions have been taken since. If the deliverable is complete and verified, confirm by replying " + + `${DONE_TOKEN} alone on the final line. Otherwise, state specifically what remains and continue working on it.` + + /** + * Mechanism-accurate overflow notice. The previous text blamed "large + * media attachments" — but the overflow flag is set whenever a request exceeded + * the provider's context/size limit before any response was produced + * (prompt.ts sets `overflow: !processor.message.finish`); media is only one + * possible cause, so the old message was usually false. + */ + export const OVERFLOW_NOTICE = + "The previous request exceeded the model's context limit before a response could be generated. Older " + + "messages were compacted into the summary above, and oversized content (large tool outputs or file " + + "attachments) may have been dropped from context. If information you need is missing from the summary, " + + "re-read the relevant files or ask the user to re-supply it." +} diff --git a/packages/opencode/src/session/tool-result-cap.ts b/packages/opencode/src/session/tool-result-cap.ts new file mode 100644 index 0000000000..b336ce9520 --- /dev/null +++ b/packages/opencode/src/session/tool-result-cap.ts @@ -0,0 +1,114 @@ +import { Token } from "@/util/token" +import { TruncateCore } from "@/tool/truncate-core" + +// Per-tool-result dispatch cap: a single tool result must never exceed a +// bounded token estimate when it enters the conversation. The per-tool +// truncation service (tool.ts:wrap → truncate.ts) already middle-truncates +// most outputs, but observed production bypasses let one giant duckdb/query +// dump jump a ~4K-token conversation past a 65K window in a single step. This +// module is the session-side hard cap enforced in processor.ts on every +// completed tool result, sized relative to the EFFECTIVE context limit (the +// declared limit scaled by the estimator safety fraction). +export namespace ToolResultCap { + // Fraction of the effective context limit one tool result may occupy. + export const DEFAULT_LIMIT_FRACTION = 0.15 + + // Densest chars-per-token ratio Token.estimate can return (RATIOS.code = 3.0): + // a string held to capTokens * 3 bytes can never estimate above capTokens. + export const MIN_CHARS_PER_TOKEN = 3.0 + + // Long single-line dumps (minified JSON, one-row query results) are re-chunked + // at this many chars so the middle-truncation byte walk can keep a head and + // tail instead of dropping the entire line. + const LINE_CHUNK_CHARS = 2_000 + + // Conservative bound when the model's limits are unknown: size the cap as if + // the model had the smallest window this cap protects (64K, scaled by the + // default 0.65 safety fraction) rather than trusting the byte-derived cap + // (~17K tokens), which can overwhelm a small window on its own. + export const UNKNOWN_MODEL_CAP_TOKENS = Math.floor(Math.floor(65_536 * 0.65) * DEFAULT_LIMIT_FRACTION) + + /** + * Resolve the per-result token cap: an explicit `tool_output.dispatch_max_tokens` + * config wins; otherwise min(existing byte-cap expressed in tokens, 15% of the + * effective context limit). Unknown or degenerate model limits fall back to a + * conservative small-window bound, never the raw byte-derived cap alone. + */ + export function resolve(input: { + config?: { + tool_output?: { max_bytes?: number; dispatch_max_tokens?: number } + compaction?: { context_safety_fraction?: number } + } + model?: { limit?: { context?: number; input?: number } } + /** Estimator safety fraction; callers pass SessionCompaction.contextSafetyFraction(config). */ + safetyFraction?: number + }): number { + const configured = input.config?.tool_output?.dispatch_max_tokens + if (configured && configured > 0) return configured + + const maxBytes = input.config?.tool_output?.max_bytes ?? TruncateCore.MAX_BYTES + const existingCapTokens = Math.ceil(maxBytes / MIN_CHARS_PER_TOKEN) + + const base = input.model?.limit?.input ?? input.model?.limit?.context ?? 0 + if (base <= 0) return Math.min(existingCapTokens, UNKNOWN_MODEL_CAP_TOKENS) + + // Default to the estimator safety fraction, not 1: an omitted fraction must + // fail conservative (tool outputs are estimate-domain), never fail open. + const fraction = input.safetyFraction ?? 0.65 + const effectiveLimit = Math.floor(base * fraction) + const limitCapTokens = Math.floor(effectiveLimit * DEFAULT_LIMIT_FRACTION) + if (limitCapTokens <= 0) return Math.min(existingCapTokens, UNKNOWN_MODEL_CAP_TOKENS) + return Math.min(existingCapTokens, limitCapTokens) + } + + /** + * Enforce the cap on one tool-result output. Outputs whose token estimate fits + * return unchanged; oversized outputs are middle-truncated (same machinery and + * marker as the tool-level truncation service) with a notice telling the model + * the output was truncated. + */ + export function apply(output: string, capTokens: number): { content: string; truncated: boolean } { + if (capTokens <= 0) return { content: output, truncated: false } + if (Token.estimate(output) <= capTokens) return { content: output, truncated: false } + + const lines: string[] = [] + for (const line of output.split("\n")) { + if (line.length <= LINE_CHUNK_CHARS) { + lines.push(line) + continue + } + for (let i = 0; i < line.length; i += LINE_CHUNK_CHARS) lines.push(line.slice(i, i + LINE_CHUNK_CHARS)) + } + const totalBytes = Buffer.byteLength(output, "utf-8") + const hint = + "The tool call succeeded but the output exceeded the per-result context budget and was truncated before dispatch. Re-run the tool with a narrower query (filters, LIMIT, offset/limit) to view specific sections." + const frame = (bodyBytes: number) => { + const preview = TruncateCore.preview(lines, totalBytes, { + maxLines: Number.MAX_SAFE_INTEGER, + maxBytes: bodyBytes, + direction: "middle", + headRatio: TruncateCore.DEFAULT_HEAD_RATIO, + }) + return TruncateCore.assemble(preview, hint, "middle") + } + + // The marker/hint framing counts against the cap: build, re-measure, and + // shrink the body budget until the ASSEMBLED result fits. Spending the full + // cap on the preview and then appending framing produced results above cap. + let bodyBytes = Math.max(1, Math.floor(capTokens * MIN_CHARS_PER_TOKEN)) + let content = frame(bodyBytes) + for (let i = 0; i < 6; i++) { + const over = Token.estimate(content) - capTokens + if (over <= 0) return { content, truncated: true } + // Remove at least the overage at the loosest chars-per-token ratio (4.0) + // so each pass makes definite progress. + bodyBytes -= Math.ceil(over * 4) + if (bodyBytes <= 0) break + content = frame(bodyBytes) + } + if (Token.estimate(content) <= capTokens) return { content, truncated: true } + // Degenerate caps (smaller than the framing itself): drop the framing and + // hard-slice — a ≤ capTokens * 3-char head can never estimate above the cap. + return { content: output.slice(0, Math.max(1, Math.floor(capTokens * MIN_CHARS_PER_TOKEN))), truncated: true } + } +} diff --git a/packages/opencode/src/tool/truncate-core.ts b/packages/opencode/src/tool/truncate-core.ts new file mode 100644 index 0000000000..de53f65d3e --- /dev/null +++ b/packages/opencode/src/tool/truncate-core.ts @@ -0,0 +1,151 @@ +// Pure truncation-selection algorithm shared by `tool/truncate.ts` (the Effect +// Service every `Tool.define()` output is routed through via `tool.ts:wrap()` — +// this is what the bash tool actually uses in production) and +// `tool/truncation.ts` (the plain-async twin used directly by `bash.ts`'s +// description-text constants, `bootstrap.ts`'s cleanup scheduler, and +// `prompt.ts`'s MCP tool-output truncation). Both call this ONE algorithm so a +// future change to truncation behavior cannot silently apply on one call path +// and not the other, the way the pre-existing hand-duplicated implementations +// could. +export const MAX_LINES = 2000 +export const MAX_BYTES = 50 * 1024 + +// Head:tail split for "middle" (head+tail) truncation. First-principles, not +// fitted to any specific corpus: root-cause errors print FIRST for the +// common compiler/build/test tool families (tsc, pytest, gcc, dbt-compile, +// ...) while verdict/success lines print LAST — weighting toward the tail +// keeps the higher-density trailing content while still guaranteeing the +// command's first error line(s) survive at the head. Callers may override +// per call via `Options.headRatio`. +export const DEFAULT_HEAD_RATIO = 1 / 3 + +// Promoted default (was "head"): pure head truncation silently drops +// trailing content — including the success/verdict line most command +// families print last. "middle" is family-neutral by construction. +export const DEFAULT_DIRECTION: Direction = "middle" + +export type Direction = "head" | "tail" | "middle" + +export interface Options { + maxLines?: number + maxBytes?: number + direction?: Direction + headRatio?: number +} + +export interface ResolvedOptions { + maxLines: number + maxBytes: number + direction: Direction + headRatio: number +} + +export interface Preview { + head: string + tail: string + removed: number + unit: "bytes" | "lines" +} + +export function fits(lines: string[], totalBytes: number, maxLines: number, maxBytes: number): boolean { + return lines.length <= maxLines && totalBytes <= maxBytes +} + +interface Selection { + lines: string[] + bytes: number + hitBytes: boolean +} + +function selectFromHead(lines: string[], maxLines: number, maxBytes: number): Selection { + const out: string[] = [] + let bytes = 0 + let hitBytes = false + for (let i = 0; i < lines.length && out.length < maxLines; i++) { + const size = Buffer.byteLength(lines[i], "utf-8") + (out.length > 0 ? 1 : 0) + if (bytes + size > maxBytes) { + hitBytes = true + break + } + out.push(lines[i]) + bytes += size + } + return { lines: out, bytes, hitBytes } +} + +// `notBefore`: lowest index the tail selection may consume, so a "middle" +// selection can never re-select a line already claimed by the head half. +function selectFromTail(lines: string[], maxLines: number, maxBytes: number, notBefore: number): Selection { + const out: string[] = [] + let bytes = 0 + let hitBytes = false + for (let i = lines.length - 1; i >= notBefore && out.length < maxLines; i--) { + const size = Buffer.byteLength(lines[i], "utf-8") + (out.length > 0 ? 1 : 0) + if (bytes + size > maxBytes) { + hitBytes = true + break + } + out.unshift(lines[i]) + bytes += size + } + return { lines: out, bytes, hitBytes } +} + +/** + * Selects the preview lines to keep for `lines`/`totalBytes` under the given + * direction and budget. Callers must first confirm `fits()` is false — + * `preview()` always assumes at least one line/byte is being removed. + */ +export function preview(lines: string[], totalBytes: number, opts: ResolvedOptions): Preview { + const { maxLines, maxBytes, direction, headRatio } = opts + + // A "middle" split needs at least one head line AND one tail line; with a + // 1-line budget the two mandatory halves would keep 2 lines and exceed + // maxLines. Degrade to tail-only (the verdict/summary line, per the + // tail-weighted design) instead of overrunning the budget. + if (direction === "tail" || (direction === "middle" && maxLines <= 1)) { + const sel = selectFromTail(lines, maxLines, maxBytes, 0) + const removed = sel.hitBytes ? totalBytes - sel.bytes : lines.length - sel.lines.length + return { head: "", tail: sel.lines.join("\n"), removed, unit: sel.hitBytes ? "bytes" : "lines" } + } + + if (direction === "middle") { + const headBudgetLines = Math.max(1, Math.floor(maxLines * headRatio)) + const tailBudgetLines = Math.max(1, maxLines - headBudgetLines) + const headBudgetBytes = Math.max(1, Math.floor(maxBytes * headRatio)) + const tailBudgetBytes = Math.max(1, maxBytes - headBudgetBytes) + + const headSel = selectFromHead(lines, headBudgetLines, headBudgetBytes) + // notBefore = headSel.lines.length: the tail walk stops at the boundary + // of what the head half already claimed, so the two halves never overlap. + const tailSel = selectFromTail(lines, tailBudgetLines, tailBudgetBytes, headSel.lines.length) + + const keptLines = headSel.lines.length + tailSel.lines.length + const keptBytes = headSel.bytes + tailSel.bytes + const linesRemoved = Math.max(0, lines.length - keptLines) + const bytesRemoved = Math.max(0, totalBytes - keptBytes) + const hitBytes = headSel.hitBytes || tailSel.hitBytes + + return { + head: headSel.lines.join("\n"), + tail: tailSel.lines.join("\n"), + removed: hitBytes ? bytesRemoved : linesRemoved, + unit: hitBytes ? "bytes" : "lines", + } + } + + // direction === "head" + const sel = selectFromHead(lines, maxLines, maxBytes) + const removed = sel.hitBytes ? totalBytes - sel.bytes : lines.length - sel.lines.length + return { head: sel.lines.join("\n"), tail: "", removed, unit: sel.hitBytes ? "bytes" : "lines" } +} + +/** Assembles the final tool-output content from a preview, the retrieval hint, and direction. */ +export function assemble(p: Preview, hint: string, direction: Direction): string { + const marker = `...${p.removed} ${p.unit} truncated...` + if (direction === "tail") return `${marker}\n\n${hint}\n\n${p.tail}` + if (direction === "middle") return `${p.head}\n\n${marker}\n\n${hint}\n\n${p.tail}` + return `${p.head}\n\n${marker}\n\n${hint}` +} + +export * as TruncateCore from "./truncate-core" diff --git a/packages/opencode/src/tool/truncate.ts b/packages/opencode/src/tool/truncate.ts index 81fdfa2b3e..224ed050d4 100644 --- a/packages/opencode/src/tool/truncate.ts +++ b/packages/opencode/src/tool/truncate.ts @@ -8,21 +8,21 @@ import { evaluate } from "@/permission/evaluate" import { Config } from "@/config/config" import { ToolID } from "./schema" import { TRUNCATION_DIR } from "./truncation-dir" +// altimate_change start — shared truncation algorithm (see truncate-core.ts +// header) so this Service and the tool/truncation.ts twin can't drift. +import { TruncateCore } from "./truncate-core" +// altimate_change end const RETENTION = Duration.days(7) -export const MAX_LINES = 2000 -export const MAX_BYTES = 50 * 1024 +export const MAX_LINES = TruncateCore.MAX_LINES +export const MAX_BYTES = TruncateCore.MAX_BYTES export const DIR = TRUNCATION_DIR export const GLOB = path.join(TRUNCATION_DIR, "*") export type Result = { content: string; truncated: false } | { content: string; truncated: true; outputPath: string } -export interface Options { - maxLines?: number - maxBytes?: number - direction?: "head" | "tail" -} +export type Options = TruncateCore.Options function hasTaskTool(agent?: Agent.Info) { if (!agent?.permission) return false @@ -92,48 +92,22 @@ export const layer = Layer.effect( } }) + // altimate_change start — default direction "middle" (head+tail, + // tail-weighted elision) via the shared truncate-core.ts algorithm. const output = Effect.fn("Truncate.output")(function* (text: string, options: Options = {}, agent?: Agent.Info) { const resolved = yield* limits() const maxLines = options.maxLines ?? resolved.maxLines const maxBytes = options.maxBytes ?? resolved.maxBytes - const direction = options.direction ?? "head" + const direction = options.direction ?? TruncateCore.DEFAULT_DIRECTION + const headRatio = options.headRatio ?? TruncateCore.DEFAULT_HEAD_RATIO const lines = text.split("\n") const totalBytes = Buffer.byteLength(text, "utf-8") - if (lines.length <= maxLines && totalBytes <= maxBytes) { + if (TruncateCore.fits(lines, totalBytes, maxLines, maxBytes)) { return { content: text, truncated: false } as const } - const out: string[] = [] - let i = 0 - let bytes = 0 - let hitBytes = false - - if (direction === "head") { - for (i = 0; i < lines.length && i < maxLines; i++) { - const size = Buffer.byteLength(lines[i], "utf-8") + (i > 0 ? 1 : 0) - if (bytes + size > maxBytes) { - hitBytes = true - break - } - out.push(lines[i]) - bytes += size - } - } else { - for (i = lines.length - 1; i >= 0 && out.length < maxLines; i--) { - const size = Buffer.byteLength(lines[i], "utf-8") + (out.length > 0 ? 1 : 0) - if (bytes + size > maxBytes) { - hitBytes = true - break - } - out.unshift(lines[i]) - bytes += size - } - } - - const removed = hitBytes ? totalBytes - bytes : lines.length - out.length - const unit = hitBytes ? "bytes" : "lines" - const preview = out.join("\n") + const preview = TruncateCore.preview(lines, totalBytes, { maxLines, maxBytes, direction, headRatio }) const file = yield* write(text) const hint = hasTaskTool(agent) @@ -141,14 +115,12 @@ export const layer = Layer.effect( : `The tool call succeeded but the output was truncated. Full output saved to: ${file}\nUse Grep to search the full content or Read with offset/limit to view specific sections.` return { - content: - direction === "head" - ? `${preview}\n\n...${removed} ${unit} truncated...\n\n${hint}` - : `...${removed} ${unit} truncated...\n\n${hint}\n\n${preview}`, + content: TruncateCore.assemble(preview, hint, direction), truncated: true, outputPath: file, } as const }) + // altimate_change end yield* cleanup().pipe( Effect.catchCause((cause) => Effect.logError("truncation cleanup failed", { cause: Cause.pretty(cause) })), diff --git a/packages/opencode/src/tool/truncation.ts b/packages/opencode/src/tool/truncation.ts index fbd92b1d7c..0b39534086 100644 --- a/packages/opencode/src/tool/truncation.ts +++ b/packages/opencode/src/tool/truncation.ts @@ -7,10 +7,14 @@ import { Scheduler } from "../scheduler" import { Filesystem } from "../util/filesystem" import { Glob } from "../util/glob" import { ToolID } from "./schema" +// altimate_change start — shared truncation algorithm (see truncate-core.ts +// header) so this twin and tool/truncate.ts's Effect Service can't drift. +import { TruncateCore } from "./truncate-core" +// altimate_change end export namespace Truncate { - export const MAX_LINES = 2000 - export const MAX_BYTES = 50 * 1024 + export const MAX_LINES = TruncateCore.MAX_LINES + export const MAX_BYTES = TruncateCore.MAX_BYTES export const DIR = path.join(Global.Path.data, "tool-output") export const GLOB = path.join(DIR, "*") const RETENTION_MS = 7 * 24 * 60 * 60 * 1000 // 7 days @@ -18,11 +22,7 @@ export namespace Truncate { export type Result = { content: string; truncated: false } | { content: string; truncated: true; outputPath: string } - export interface Options { - maxLines?: number - maxBytes?: number - direction?: "head" | "tail" - } + export type Options = TruncateCore.Options export function init() { Scheduler.register({ @@ -60,47 +60,21 @@ export namespace Truncate { return rule.action !== "deny" } + // altimate_change start — default direction "middle" (head+tail, + // tail-weighted elision) via the shared truncate-core.ts algorithm. export async function output(text: string, options: Options = {}, agent?: Agent.Info): Promise { const maxLines = options.maxLines ?? MAX_LINES const maxBytes = options.maxBytes ?? MAX_BYTES - const direction = options.direction ?? "head" + const direction = options.direction ?? TruncateCore.DEFAULT_DIRECTION + const headRatio = options.headRatio ?? TruncateCore.DEFAULT_HEAD_RATIO const lines = text.split("\n") const totalBytes = Buffer.byteLength(text, "utf-8") - if (lines.length <= maxLines && totalBytes <= maxBytes) { + if (TruncateCore.fits(lines, totalBytes, maxLines, maxBytes)) { return { content: text, truncated: false } } - const out: string[] = [] - let i = 0 - let bytes = 0 - let hitBytes = false - - if (direction === "head") { - for (i = 0; i < lines.length && i < maxLines; i++) { - const size = Buffer.byteLength(lines[i], "utf-8") + (i > 0 ? 1 : 0) - if (bytes + size > maxBytes) { - hitBytes = true - break - } - out.push(lines[i]) - bytes += size - } - } else { - for (i = lines.length - 1; i >= 0 && out.length < maxLines; i--) { - const size = Buffer.byteLength(lines[i], "utf-8") + (out.length > 0 ? 1 : 0) - if (bytes + size > maxBytes) { - hitBytes = true - break - } - out.unshift(lines[i]) - bytes += size - } - } - - const removed = hitBytes ? totalBytes - bytes : lines.length - out.length - const unit = hitBytes ? "bytes" : "lines" - const preview = out.join("\n") + const preview = TruncateCore.preview(lines, totalBytes, { maxLines, maxBytes, direction, headRatio }) const id = ToolID.ascending() const filepath = path.join(DIR, id) @@ -109,11 +83,8 @@ export namespace Truncate { const hint = hasTaskTool(agent) ? `The tool call succeeded but the output was truncated. Full output saved to: ${filepath}\nUse the Task tool to have explore agent process this file with Grep and Read (with offset/limit). Do NOT read the full file yourself - delegate to save context.` : `The tool call succeeded but the output was truncated. Full output saved to: ${filepath}\nUse Grep to search the full content or Read with offset/limit to view specific sections.` - const message = - direction === "head" - ? `${preview}\n\n...${removed} ${unit} truncated...\n\n${hint}` - : `...${removed} ${unit} truncated...\n\n${hint}\n\n${preview}` - return { content: message, truncated: true, outputPath: filepath } + return { content: TruncateCore.assemble(preview, hint, direction), truncated: true, outputPath: filepath } } + // altimate_change end } diff --git a/packages/opencode/test/cli/idle-done.test.ts b/packages/opencode/test/cli/idle-done.test.ts new file mode 100644 index 0000000000..41230bf9c4 --- /dev/null +++ b/packages/opencode/test/cli/idle-done.test.ts @@ -0,0 +1,462 @@ +// Harness reliability (c) unit gates — idle-done detection, the run-mode-only +// FALLBACK termination path. Every hard precondition is exercised: +// (i) green verify temporally AFTER the last file mutation (event-stream order) +// (ii) generic verify classification (configured command or side-effecting bash; +// classifier contains no vertical tokens — leak-lens hard requirement) +// (iii) suppression while tools/subagents/permissions are outstanding +// (iv) compaction-gated + N consecutive post-compaction text-only turns +// (v) one-shot recursion guard +import { describe, expect, test } from "bun:test" +import { IdleDone } from "../../src/cli/cmd/idle-done" + +const OPTS: IdleDone.Options = { enabled: true, minCompactions: 2, idleTurns: 3 } +const deps = (compactionIDs: string[] = []) => ({ + isCompactionStep: (id: string) => compactionIDs.includes(id), +}) + +let partCounter = 0 +function pid() { + return `prt_${++partCounter}` +} + +function bashPart(messageID: string, command: string, exit: number): IdleDone.PartSlice { + return { + id: pid(), + messageID, + type: "tool", + tool: "bash", + state: { status: "completed", input: { command }, metadata: { exit } }, + } +} +function editPart(messageID: string): IdleDone.PartSlice { + return { id: pid(), messageID, type: "tool", tool: "edit", state: { status: "completed", input: {} } } +} +function patchPart(messageID: string): IdleDone.PartSlice { + return { id: pid(), messageID, type: "patch" } +} +function stepFinish(messageID: string, reason = "stop"): IdleDone.PartSlice { + return { id: pid(), messageID, type: "step-finish", reason } +} + +/** Drive a detector into the fully-satisfied state, returning it. */ +function satisfied(options: IdleDone.Options = OPTS) { + const d = IdleDone.create(options, deps(["cmp_1", "cmp_2"])) + // Work turn: edit, then (in a LATER step) a green side-effecting verify. + d.observePart(editPart("m_work")) + d.observePart(patchPart("m_work")) + d.observePart(stepFinish("m_work")) + d.observePart(bashPart("m_verify", "./scripts/verify.sh --all", 0)) + d.observePart(stepFinish("m_verify")) + // Two completed compaction cycles. + d.observePart(stepFinish("cmp_1")) + d.observePart(stepFinish("cmp_2")) + // Three post-compaction text-only turns (the churn signature). + d.observePart(stepFinish("m_idle1")) + d.observePart(stepFinish("m_idle2")) + d.observePart(stepFinish("m_idle3")) + return d +} + +describe("IdleDone.optionsFromEnv (config-exposed thresholds)", () => { + test("defaults: enabled, minCompactions=2, idleTurns=3, no verify command", () => { + expect(IdleDone.optionsFromEnv({})).toEqual({ + enabled: true, + minCompactions: 2, + idleTurns: 3, + verifyCommand: undefined, + }) + }) + + test("env overrides win; ALTIMATE_RUN_IDLE_DONE=0 disables", () => { + const opts = IdleDone.optionsFromEnv({ + ALTIMATE_RUN_IDLE_DONE: "0", + ALTIMATE_IDLE_DONE_MIN_COMPACTIONS: "5", + ALTIMATE_IDLE_DONE_IDLE_TURNS: "7", + ALTIMATE_RUN_VERIFY_COMMAND: "make check", + }) + expect(opts).toEqual({ enabled: false, minCompactions: 5, idleTurns: 7, verifyCommand: "make check" }) + }) + + test("garbage threshold values fall back to defaults", () => { + const opts = IdleDone.optionsFromEnv({ + ALTIMATE_IDLE_DONE_MIN_COMPACTIONS: "zero", + ALTIMATE_IDLE_DONE_IDLE_TURNS: "-3", + }) + expect(opts.minCompactions).toBe(2) + expect(opts.idleTurns).toBe(3) + }) +}) + +describe("IdleDone.armedOptions (run-mode/attach arming gate)", () => { + const base: IdleDone.Options = { enabled: true, minCompactions: 2, idleTurns: 3 } + + test("armed only for a local run with run mode active", () => { + expect(IdleDone.armedOptions(base, { attach: false, runMode: true }).enabled).toBe(true) + }) + + test("regression: explicit ALTIMATE_RUN_MODE=0 opt-out disarms idle-done", () => { + // The run handler preserves an explicit "0" (applyRunModeDefault) so the + // flag reads false — idle-done must never arm in that session. + expect(IdleDone.armedOptions(base, { attach: false, runMode: false }).enabled).toBe(false) + }) + + test("regression: --attach disarms idle-done regardless of run mode", () => { + // The remote (possibly shared/interactive) session must never be aborted + // by the local idle-done challenge. + expect(IdleDone.armedOptions(base, { attach: true, runMode: true }).enabled).toBe(false) + expect(IdleDone.armedOptions(base, { attach: true, runMode: false }).enabled).toBe(false) + }) + + test("an env-disabled fallback can never be re-enabled by the gate", () => { + const disabled: IdleDone.Options = { ...base, enabled: false } + expect(IdleDone.armedOptions(disabled, { attach: false, runMode: true }).enabled).toBe(false) + }) + + test("a disarmed detector never challenges even when every precondition holds", () => { + // Same event sequence that arms the fully-satisfied detector above. + expect(satisfied().shouldChallenge()).toBe(true) + expect(satisfied(IdleDone.armedOptions(base, { attach: false, runMode: false })).shouldChallenge()).toBe(false) + expect(satisfied(IdleDone.armedOptions(base, { attach: true, runMode: true })).shouldChallenge()).toBe(false) + }) +}) + +describe("IdleDone.isReadOnlyCommand (generic classifier, .ii)", () => { + test("plain read-only commands are read-only", () => { + for (const cmd of ["ls -la", "cat file.txt", "grep -r pattern .", "pwd", "git status", "git log --oneline -5"]) { + expect(IdleDone.isReadOnlyCommand(cmd)).toBe(true) + } + }) + + test("pipelines of read-only heads stay read-only", () => { + expect(IdleDone.isReadOnlyCommand("cat log.txt | grep ERROR | wc -l")).toBe(true) + expect(IdleDone.isReadOnlyCommand("ls && pwd; git status")).toBe(true) + }) + + test("build/test/run-shaped commands are side-effecting", () => { + for (const cmd of ["make check", "npm test", "python3 run_tests.py", "./verify.sh", "cargo build"]) { + expect(IdleDone.isReadOnlyCommand(cmd)).toBe(false) + } + }) + + test("a read-only head with a mutating tail statement is side-effecting", () => { + expect(IdleDone.isReadOnlyCommand("ls && rm -rf build")).toBe(false) + }) + + test("mutating git subcommands are side-effecting", () => { + expect(IdleDone.isReadOnlyCommand("git commit -m x")).toBe(false) + expect(IdleDone.isReadOnlyCommand("git push")).toBe(false) + }) + + test("leading env assignments are skipped when classifying the head", () => { + expect(IdleDone.isReadOnlyCommand("FOO=1 cat x")).toBe(true) + expect(IdleDone.isReadOnlyCommand("FOO=1 make check")).toBe(false) + }) + + // Classifying by command head alone misses in-place and redirection forms. + // Snapshot patch parts normally catch bash writes, but `snapshot: false` + // emits none, and an unrecorded write leaves the mutation watermark stale. + test("in-place editor flags are mutating even though the head is read-only", () => { + for (const cmd of ["sed -i '' 's/a/b/' src/app.ts", "sed -i.bak s/a/b/ f.txt"]) { + // `sed` is on the read-only allowlist, so the head alone says "no write". + expect(IdleDone.isReadOnlyCommand(cmd)).toBe(true) + expect(IdleDone.isMutatingCommand(cmd)).toBe(true) + } + expect(IdleDone.isMutatingCommand("perl -i -pe 's/a/b/' f.txt")).toBe(true) + }) + + test("output redirection is mutating; fd duplication is not", () => { + expect(IdleDone.isMutatingCommand("cat a.txt > b.txt")).toBe(true) + expect(IdleDone.isMutatingCommand("echo hi >> log.txt")).toBe(true) + expect(IdleDone.isMutatingCommand("ls | tee out.txt")).toBe(true) + expect(IdleDone.isMutatingCommand("make check 2>&1")).toBe(false) + }) + + test("always-writing heads are mutating anywhere in the pipeline", () => { + expect(IdleDone.isMutatingCommand("ls && rm -rf build")).toBe(true) + expect(IdleDone.isMutatingCommand("mkdir -p out")).toBe(true) + expect(IdleDone.isMutatingCommand("FOO=1 mv a b")).toBe(true) + }) + + test("plain read-only commands are not mutating", () => { + for (const cmd of ["ls -la", "cat file.txt", "grep -r pattern .", "git status", "sed s/a/b/ f.txt"]) { + expect(IdleDone.isMutatingCommand(cmd)).toBe(false) + } + }) + + test("classifier and module contain no vertical/product tokens (leak-lens hard requirement)", async () => { + const source = await Bun.file(new URL("../../src/cli/cmd/idle-done.ts", import.meta.url).pathname).text() + // No dbt/vertical string matching inside the generic mechanism, and no bench + // task command strings in product code. + expect(/\bdbt\b/i.test(source)).toBe(false) + expect(source).not.toContain("--profiles-dir") + }) +}) + +describe("IdleDone hard preconditions", () => { + test("fully-satisfied signature arms the challenge", () => { + expect(satisfied().shouldChallenge()).toBe(true) + }) + + // altimate_change start — upstream_fix regression: lastMutationSeq starts at + // -1, so a session that never mutated a file (read-only/explore, or one that + // only ever ran verify commands) satisfied "verify after last write" + // vacuously — there was no completed work for the green verify to confirm. + test("(i) NEVER fires when no mutation was ever observed, even with a green verify", () => { + const d = IdleDone.create(OPTS, deps(["cmp_1", "cmp_2"])) + d.observePart(bashPart("m_verify", "./scripts/verify.sh --all", 0)) + d.observePart(stepFinish("m_verify")) + d.observePart(stepFinish("cmp_1")) + d.observePart(stepFinish("cmp_2")) + d.observePart(stepFinish("m_idle1")) + d.observePart(stepFinish("m_idle2")) + d.observePart(stepFinish("m_idle3")) + expect(d.shouldChallenge()).toBe(false) + }) + // altimate_change end + + // Snapshots off (`snapshot: false`) means no patch part reports a + // bash-mediated write, so the in-place edit below is the only evidence that + // the session changed a file after its last green verification. + test("(i) an in-place bash edit after the verify blocks the challenge with no patch part", () => { + const d = IdleDone.create(OPTS, deps(["cmp_1", "cmp_2"])) + d.observePart(editPart("m_work")) + d.observePart(patchPart("m_work")) + d.observePart(stepFinish("m_work")) + d.observePart(bashPart("m_verify", "./scripts/verify.sh --all", 0)) + d.observePart(stepFinish("m_verify")) + // A write that produces no patch part and whose head is on the read-only list. + d.observePart(bashPart("m_sed", "sed -i '' 's/a/b/' src/app.ts", 0)) + d.observePart(stepFinish("m_sed")) + d.observePart(stepFinish("cmp_1")) + d.observePart(stepFinish("cmp_2")) + for (const m of ["m_idle1", "m_idle2", "m_idle3"]) d.observePart(stepFinish(m)) + expect(d.shouldChallenge()).toBe(false) + // The write was recorded, and it postdates the verification. + const snap = d.snapshot() + expect(snap.last_mutation_seq).toBeGreaterThan(snap.last_verify_seq) + }) + + test("(ii) a configured verify command that redirects its output is still the verification", () => { + const opts: IdleDone.Options = { ...OPTS, verifyCommand: "make check" } + const d = IdleDone.create(opts, deps(["cmp_1", "cmp_2"])) + d.observePart(editPart("m_work")) + d.observePart(patchPart("m_work")) + d.observePart(stepFinish("m_work")) + d.observePart(bashPart("m_verify", "make check > build.log", 0)) + d.observePart(stepFinish("m_verify")) + d.observePart(stepFinish("cmp_1")) + d.observePart(stepFinish("cmp_2")) + for (const m of ["m_idle1", "m_idle2", "m_idle3"]) d.observePart(stepFinish(m)) + expect(d.shouldChallenge()).toBe(true) + }) + + test("(iv) NEVER fires in a never-compacted session", () => { + const d = IdleDone.create(OPTS, deps([])) + d.observePart(editPart("m1")) + d.observePart(bashPart("m2", "make check", 0)) + d.observePart(stepFinish("m2")) + for (const m of ["m3", "m4", "m5", "m6"]) d.observePart(stepFinish(m)) + expect(d.shouldChallenge()).toBe(false) + }) + + test("(iv) one compaction is not enough at minCompactions=2", () => { + const d = IdleDone.create(OPTS, deps(["cmp_1"])) + d.observePart(bashPart("m_verify", "make check", 0)) + d.observePart(stepFinish("m_verify")) + d.observePart(stepFinish("cmp_1")) + for (const m of ["m3", "m4", "m5"]) d.observePart(stepFinish(m)) + expect(d.shouldChallenge()).toBe(false) + }) + + test("(iv) fewer than idleTurns consecutive text-only turns is not enough", () => { + const d = IdleDone.create(OPTS, deps(["cmp_1", "cmp_2"])) + d.observePart(bashPart("m_verify", "make check", 0)) + d.observePart(stepFinish("m_verify")) + d.observePart(stepFinish("cmp_1")) + d.observePart(stepFinish("cmp_2")) + d.observePart(stepFinish("m_idle1")) + d.observePart(stepFinish("m_idle2")) + expect(d.shouldChallenge()).toBe(false) + }) + + test("a tool-using turn resets the consecutive idle-turn counter", () => { + const d = satisfied() + expect(d.shouldChallenge()).toBe(true) + // A turn with tool activity breaks the streak… + d.observePart(bashPart("m_active", "grep -r foo .", 0)) + d.observePart(stepFinish("m_active")) + expect(d.shouldChallenge()).toBe(false) + // …and three more idle turns re-arm it. + for (const m of ["m_i4", "m_i5", "m_i6"]) d.observePart(stepFinish(m)) + expect(d.shouldChallenge()).toBe(true) + }) + + test("(i) verify BEFORE the last mutation does not certify: build-after-last-write", () => { + const d = IdleDone.create(OPTS, deps(["cmp_1", "cmp_2"])) + d.observePart(bashPart("m1", "make check", 0)) // green verify… + d.observePart(stepFinish("m1")) + d.observePart(editPart("m2")) // …then a mutation AFTER it + d.observePart(patchPart("m2")) + d.observePart(stepFinish("m2")) + d.observePart(stepFinish("cmp_1")) + d.observePart(stepFinish("cmp_2")) + for (const m of ["m3", "m4", "m5"]) d.observePart(stepFinish(m)) + expect(d.shouldChallenge()).toBe(false) + }) + + test("(i) a patch part (bash-mediated mutation ground truth) after the verify suppresses", () => { + const d = IdleDone.create(OPTS, deps(["cmp_1", "cmp_2"])) + // Verify and mutation land in the SAME step: the step's patch part postdates + // the verify in stream order, so ordering within the step cannot be proven + // and the detector conservatively suppresses. + d.observePart(editPart("m1")) + d.observePart(bashPart("m1", "make check", 0)) + d.observePart(patchPart("m1")) + d.observePart(stepFinish("m1")) + d.observePart(stepFinish("cmp_1")) + d.observePart(stepFinish("cmp_2")) + for (const m of ["m2", "m3", "m4"]) d.observePart(stepFinish(m)) + expect(d.shouldChallenge()).toBe(false) + }) + + // Production emits the step-finish part FIRST and the snapshot `patch` part + // immediately after it (session/processor.ts writes step-finish, then diffs the + // snapshot), so the two tests below drive the detector in that real order — the + // helpers above emit the patch before step-finish, which would mask an ordering + // regression in the mutation/verify comparison. + test("(i) production ordering: patch AFTER its own step-finish still precedes a later verify", () => { + const d = IdleDone.create(OPTS, deps(["cmp_1", "cmp_2"])) + d.observePart(editPart("m_work")) + d.observePart(stepFinish("m_work")) + d.observePart(patchPart("m_work")) // step-end snapshot diff, as production emits it + d.observePart(bashPart("m_verify", "./scripts/verify.sh --all", 0)) + d.observePart(stepFinish("m_verify")) + d.observePart(stepFinish("cmp_1")) + d.observePart(stepFinish("cmp_2")) + for (const m of ["m_idle1", "m_idle2", "m_idle3"]) d.observePart(stepFinish(m)) + expect(d.shouldChallenge()).toBe(true) + }) + + test("(i) production ordering: a same-step patch emitted after step-finish still suppresses", () => { + const d = IdleDone.create(OPTS, deps(["cmp_1", "cmp_2"])) + d.observePart(editPart("m1")) + d.observePart(bashPart("m1", "make check", 0)) // verify inside the mutating step + d.observePart(stepFinish("m1")) + d.observePart(patchPart("m1")) // snapshot diff lands after step-finish + d.observePart(stepFinish("cmp_1")) + d.observePart(stepFinish("cmp_2")) + for (const m of ["m2", "m3", "m4"]) d.observePart(stepFinish(m)) + expect(d.shouldChallenge()).toBe(false) + }) + + test("(i)/(ii) a FAILING most-recent verify blocks the challenge", () => { + const d = IdleDone.create(OPTS, deps(["cmp_1", "cmp_2"])) + d.observePart(editPart("m1")) + d.observePart(stepFinish("m1")) + d.observePart(bashPart("m2", "make check", 0)) + d.observePart(stepFinish("m2")) + d.observePart(bashPart("m3", "make check", 2)) // most recent verify is RED + d.observePart(stepFinish("m3")) + d.observePart(stepFinish("cmp_1")) + d.observePart(stepFinish("cmp_2")) + for (const m of ["m4", "m5", "m6"]) d.observePart(stepFinish(m)) + expect(d.shouldChallenge()).toBe(false) + }) + + test("(ii) read-only bash (ls/git status) never counts as a green verify", () => { + const d = IdleDone.create(OPTS, deps(["cmp_1", "cmp_2"])) + d.observePart(editPart("m1")) + d.observePart(stepFinish("m1")) + d.observePart(bashPart("m2", "ls -la", 0)) + d.observePart(bashPart("m2", "git status", 0)) + d.observePart(stepFinish("m2")) + d.observePart(stepFinish("cmp_1")) + d.observePart(stepFinish("cmp_2")) + for (const m of ["m3", "m4", "m5"]) d.observePart(stepFinish(m)) + expect(d.shouldChallenge()).toBe(false) + }) + + test("(ii) configured verify command restricts candidates to that command", () => { + const opts: IdleDone.Options = { ...OPTS, verifyCommand: "./scripts/verify.sh" } + const d = IdleDone.create(opts, deps(["cmp_1", "cmp_2"])) + d.observePart(editPart("m1")) + d.observePart(stepFinish("m1")) + // A green side-effecting command that is NOT the configured verify: ignored. + d.observePart(bashPart("m2", "make check", 0)) + d.observePart(stepFinish("m2")) + d.observePart(stepFinish("cmp_1")) + d.observePart(stepFinish("cmp_2")) + for (const m of ["m3", "m4", "m5"]) d.observePart(stepFinish(m)) + expect(d.shouldChallenge()).toBe(false) + // The configured command going green in a later step satisfies (i)+(ii). + d.observePart(bashPart("m6", "./scripts/verify.sh --all", 0)) + d.observePart(stepFinish("m6")) + for (const m of ["m7", "m8", "m9"]) d.observePart(stepFinish(m)) + expect(d.shouldChallenge()).toBe(true) + }) + + test("(iii) an outstanding running tool (e.g. a task subagent) suppresses", () => { + const d = satisfied() + d.observePart({ id: "prt_task", messageID: "m_bg", type: "tool", tool: "task", state: { status: "running" } }) + expect(d.shouldChallenge()).toBe(false) + d.observePart({ id: "prt_task", messageID: "m_bg", type: "tool", tool: "task", state: { status: "completed" } }) + // The completing subagent turn resets the idle streak; re-idle to re-arm. + for (const m of ["m_i7", "m_i8", "m_i9"]) d.observePart(stepFinish(m)) + expect(d.shouldChallenge()).toBe(true) + }) + + test("(iii) a pending permission request suppresses until resolved", () => { + const d = satisfied() + d.onPermissionAsked("perm_1") + expect(d.shouldChallenge()).toBe(false) + d.onPermissionResolved("perm_1") + expect(d.shouldChallenge()).toBe(true) + }) + + test("(v) one-shot: after the challenge is issued it can never re-arm", () => { + const d = satisfied() + expect(d.shouldChallenge()).toBe(true) + d.markChallengeIssued() + expect(d.shouldChallenge()).toBe(false) + for (const m of ["m_x1", "m_x2", "m_x3", "m_x4"]) d.observePart(stepFinish(m)) + expect(d.shouldChallenge()).toBe(false) + expect(d.challengeIssued).toBe(true) + }) + + test("disabled via config: never arms even when fully satisfied", () => { + const d = satisfied({ ...OPTS, enabled: false }) + expect(d.shouldChallenge()).toBe(false) + }) + + test("compaction step-finishes reset the idle streak (idle turns are per-cycle)", () => { + const d = IdleDone.create(OPTS, deps(["cmp_1", "cmp_2"])) + // A mutation must exist before a green verify can satisfy the + // build-after-last-write precondition (i) — this test isolates the + // idle-streak-reset behavior, not the mutation precondition itself. + d.observePart(editPart("m_work")) + d.observePart(stepFinish("m_work")) + d.observePart(bashPart("m_verify", "make check", 0)) + d.observePart(stepFinish("m_verify")) + d.observePart(stepFinish("cmp_1")) + d.observePart(stepFinish("m_idle1")) + d.observePart(stepFinish("m_idle2")) + d.observePart(stepFinish("cmp_2")) // another compaction mid-streak + d.observePart(stepFinish("m_idle3")) + expect(d.shouldChallenge()).toBe(false) // streak restarted after cmp_2 + d.observePart(stepFinish("m_idle4")) + d.observePart(stepFinish("m_idle5")) + expect(d.shouldChallenge()).toBe(true) + }) + + test("non-stop finish reasons do not count as idle turns", () => { + const d = IdleDone.create(OPTS, deps(["cmp_1", "cmp_2"])) + d.observePart(bashPart("m_verify", "make check", 0)) + d.observePart(stepFinish("m_verify")) + d.observePart(stepFinish("cmp_1")) + d.observePart(stepFinish("cmp_2")) + d.observePart(stepFinish("m1", "length")) + d.observePart(stepFinish("m2", "length")) + d.observePart(stepFinish("m3", "length")) + expect(d.shouldChallenge()).toBe(false) + }) +}) diff --git a/packages/opencode/test/cli/run-accounting.test.ts b/packages/opencode/test/cli/run-accounting.test.ts new file mode 100644 index 0000000000..386a34ab07 --- /dev/null +++ b/packages/opencode/test/cli/run-accounting.test.ts @@ -0,0 +1,383 @@ +// — honest turn accounting: compaction-machinery steps must not consume the +// --max-turns budget. (E4) — dual-attribution termination logging: every run +// records why_model_stopped AND why_harness_stopped as independent fields. +// — real error serialization (never a bare name, "[object Object]", or "{}"). +import { describe, expect, test } from "bun:test" +import { RunAccounting } from "../../src/cli/cmd/run-accounting" + +describe("RunAccounting turn accounting", () => { + test("counts ordinary assistant steps", () => { + const acc = RunAccounting.create() + acc.onAssistantMessage({ id: "msg_1", agent: "build" }) + expect(acc.onStepStart("msg_1")).toBe(true) + expect(acc.onStepStart("msg_1")).toBe(true) + expect(acc.turnCount).toBe(2) + }) + + test("excludes compaction-machinery steps from turnCount", () => { + const acc = RunAccounting.create() + acc.onAssistantMessage({ id: "msg_work", agent: "build" }) + acc.onAssistantMessage({ id: "msg_compact", agent: "compaction" }) + expect(acc.onStepStart("msg_work")).toBe(true) + expect(acc.onStepStart("msg_compact")).toBe(false) + expect(acc.onStepStart("msg_compact")).toBe(false) + expect(acc.onStepStart("msg_work")).toBe(true) + expect(acc.turnCount).toBe(2) + }) + + test("a step whose owning message is unknown is counted (conservative default)", () => { + const acc = RunAccounting.create() + expect(acc.onStepStart("msg_unknown")).toBe(true) + expect(acc.turnCount).toBe(1) + }) + + test("compaction steps do not perturb termination attribution", () => { + const acc = RunAccounting.create() + acc.onAssistantMessage({ id: "msg_work", agent: "build" }) + acc.onAssistantMessage({ id: "msg_compact", agent: "compaction" }) + acc.onStepFinish("msg_work", "stop") + // compaction machinery finishing later must not overwrite the model's reason + acc.onStepFinish("msg_compact", "tool-calls") + acc.onText("msg_compact", "summary text\nDONE") + expect(acc.termination().why_model_stopped).toBe("stop") + }) +}) + +describe("RunAccounting termination attribution (E4)", () => { + test("both fields are always present with valid enum values", () => { + const acc = RunAccounting.create() + const t = acc.termination() + expect(["stop", "tool-call", "explicit-done"]).toContain(t.why_model_stopped) + expect(["budget-exhausted", "timeout", "error", "idle-done", "none"]).toContain(t.why_harness_stopped) + }) + + test("natural finish: model=stop, harness=none", () => { + const acc = RunAccounting.create() + acc.onAssistantMessage({ id: "m1", agent: "build" }) + acc.onStepStart("m1") + acc.onStepFinish("m1", "stop") + expect(acc.termination()).toEqual({ why_model_stopped: "stop", why_harness_stopped: "none", done_reason: "none" }) + expect(acc.fatal).toBe(false) + }) + + test("model still tool-calling when harness exhausts the budget", () => { + const acc = RunAccounting.create() + acc.onAssistantMessage({ id: "m1", agent: "build" }) + acc.onStepStart("m1") + acc.onStepFinish("m1", "tool-calls") + acc.onBudgetExhausted() + expect(acc.termination()).toEqual({ why_model_stopped: "tool-call", why_harness_stopped: "budget-exhausted", done_reason: "none" }) + expect(acc.fatal).toBe(true) + }) + + test("explicit DONE assertion in the final text classifies as explicit-done", () => { + const acc = RunAccounting.create() + acc.onAssistantMessage({ id: "m1", agent: "build" }) + acc.onText("m1", "All checks pass.\nDONE") + acc.onStepFinish("m1", "stop") + expect(acc.termination().why_model_stopped).toBe("explicit-done") + }) + + test("a later non-DONE text clears the explicit-done classification", () => { + const acc = RunAccounting.create() + acc.onAssistantMessage({ id: "m1", agent: "build" }) + acc.onText("m1", "DONE") + acc.onText("m1", "actually, one more thing") + acc.onStepFinish("m1", "stop") + expect(acc.termination().why_model_stopped).toBe("stop") + }) + + test("fatal session error attributes harness=error and flips fatal", () => { + const acc = RunAccounting.create() + acc.onSessionError("APIError", "boom") + expect(acc.termination().why_harness_stopped).toBe("error") + expect(acc.fatal).toBe(true) + }) + + test("timeout-shaped session error attributes harness=timeout", () => { + const acc = RunAccounting.create() + acc.onSessionError("UnknownError", "request timed out waiting for provider") + expect(acc.termination().why_harness_stopped).toBe("timeout") + }) + + test("recoverable ContextOverflowError does not flip fatal or the attribution", () => { + // Auto-compaction recovers overflow; the error event alone must not change rc. + const acc = RunAccounting.create() + acc.onSessionError("ContextOverflowError", "context window exceeded") + expect(acc.fatal).toBe(false) + expect(acc.termination().why_harness_stopped).toBe("none") + }) + + test("terminal message with abnormal finish (error/other) is fatal (swallowed transport failure)", () => { + for (const finish of ["error", "other"]) { + const acc = RunAccounting.create() + acc.onPromptResult({ finish }) + expect(acc.fatal).toBe(true) + expect(acc.termination().why_harness_stopped).toBe("error") + } + }) + + test("terminal message with a normal finish is not fatal", () => { + for (const finish of ["stop", "length", "tool-calls", "content-filter", "unknown", undefined]) { + const acc = RunAccounting.create() + acc.onPromptResult({ finish }) + expect(acc.fatal).toBe(false) + } + }) + + test("terminal message carrying an error field is fatal via the session-error path", () => { + const acc = RunAccounting.create() + acc.onPromptResult({ finish: "stop", error: { name: "APIError", data: { message: "boom" } } }) + expect(acc.fatal).toBe(true) + expect(acc.termination().why_harness_stopped).toBe("error") + }) + + test("budget exhaustion takes precedence over a subsequent abort error", () => { + const acc = RunAccounting.create() + acc.onBudgetExhausted() + acc.onSessionError("MessageAbortedError", "aborted") + expect(acc.termination().why_harness_stopped).toBe("budget-exhausted") + }) +}) + +describe("RunAccounting.serializeSessionError", () => { + test("composes name, status, and message", () => { + expect( + RunAccounting.serializeSessionError({ name: "APIError", data: { message: "upstream broke", status: 502 } }), + ).toBe("APIError (status 502): upstream broke") + }) + + test("statusCode variant is picked up", () => { + expect(RunAccounting.serializeSessionError({ name: "APIError", data: { message: "x", statusCode: 500 } })).toBe( + "APIError (status 500): x", + ) + }) + + test("never returns a literal {} or [object Object]", () => { + for (const input of [{}, { name: "", data: {} }, { name: "E", data: { message: { nested: true } } }, null, 7]) { + const out = RunAccounting.serializeSessionError(input) + expect(out).not.toBe("{}") + expect(out).not.toContain("[object Object]") + expect(out.length).toBeGreaterThan(0) + } + }) + + test("name-only errors serialize to the name", () => { + expect(RunAccounting.serializeSessionError({ name: "MessageOutputLengthError", data: {} })).toBe( + "MessageOutputLengthError", + ) + }) + + // altimate_change start — upstream_fix regression: a native thrown Error + // (e.g. a network/transport failure) has `.message` at the top level, not + // nested under `.data`, and previously serialized to the bare error name. + test("falls back to the top-level message on a native Error with no data.message", () => { + expect(RunAccounting.serializeSessionError(new TypeError("fetch failed"))).toBe("TypeError: fetch failed") + }) + + test("data.message still wins over the top-level message when both are present", () => { + expect( + RunAccounting.serializeSessionError({ name: "APIError", message: "generic", data: { message: "specific" } }), + ).toBe("APIError: specific") + }) + // altimate_change end +}) + +describe("RunAccounting retry classification", () => { + test("5xx statuses are retryable; 4xx and non-numbers are not", () => { + expect(RunAccounting.isRetryableStatus(500)).toBe(true) + expect(RunAccounting.isRetryableStatus(503)).toBe(true) + expect(RunAccounting.isRetryableStatus(400)).toBe(false) + expect(RunAccounting.isRetryableStatus(404)).toBe(false) + expect(RunAccounting.isRetryableStatus(undefined)).toBe(false) + }) + + test("timeouts and dropped connections are retryable thrown errors", () => { + expect(RunAccounting.isRetryableThrown(new Error("request timed out"))).toBe(true) + expect(RunAccounting.isRetryableThrown(Object.assign(new Error("io"), { code: "ECONNRESET" }))).toBe(true) + expect(RunAccounting.isRetryableThrown(new Error("fetch failed"))).toBe(true) + expect(RunAccounting.isRetryableThrown(new Error("model not found"))).toBe(false) + expect(RunAccounting.isRetryableThrown(undefined)).toBe(false) + }) + + test("backoff grows exponentially but never exceeds the timer ceiling", () => { + expect(RunAccounting.retryDelayMs(1000, 0)).toBe(1000) + expect(RunAccounting.retryDelayMs(1000, 3)).toBe(8000) + // The accepted env maximums (base 60s, 20 retries) compound past the signed + // 32-bit limit; an unclamped delay there is scheduled for ~1ms, which turns + // the backoff into a tight retry loop. + expect(60_000 * 2 ** 19).toBeGreaterThan(RunAccounting.MAX_TIMER_MS) + expect(RunAccounting.retryDelayMs(60_000, 19)).toBe(RunAccounting.MAX_TIMER_MS) + for (let attempt = 0; attempt <= 20; attempt++) { + expect(RunAccounting.retryDelayMs(60_000, attempt)).toBeLessThanOrEqual(RunAccounting.MAX_TIMER_MS) + } + }) +}) + +describe("RunAccounting done_reason + idle-done bookkeeping", () => { + test("bare finishReason stop is NEVER reported as done", () => { + const acc = RunAccounting.create() + acc.onAssistantMessage({ id: "m1", agent: "build" }) + acc.onText("m1", "Let me now read the schema file.") + acc.onStepFinish("m1", "stop") + expect(acc.termination().done_reason).toBe("none") + }) + + test("unprompted stop+DONE reports done_reason=explicit_done", () => { + const acc = RunAccounting.create() + acc.onAssistantMessage({ id: "m1", agent: "build" }) + acc.onText("m1", "All checks green.\nDONE") + acc.onStepFinish("m1", "stop") + const t = acc.termination() + expect(t.done_reason).toBe("explicit_done") + expect(t.why_model_stopped).toBe("explicit-done") + expect(t.why_harness_stopped).toBe("none") + }) + + test("DONE elicited by the idle-done challenge reports idle_heuristic + harness idle-done", () => { + const acc = RunAccounting.create() + acc.onAssistantMessage({ id: "m1", agent: "build" }) + acc.onIdleDoneChallengeIssued() + acc.onText("m1", "Confirmed.\nDONE") + acc.onStepFinish("m1", "stop") + const t = acc.termination() + expect(t.done_reason).toBe("idle_heuristic") + expect(t.why_harness_stopped).toBe("idle-done") + expect(acc.fatal).toBe(false) + }) + + test("challenge issued but model continues without DONE: done_reason=none, harness=none", () => { + const acc = RunAccounting.create() + acc.onAssistantMessage({ id: "m1", agent: "build" }) + acc.onIdleDoneChallengeIssued() + acc.onText("m1", "Remaining: wire the config flag. Continuing.") + acc.onStepFinish("m1", "stop") + const t = acc.termination() + expect(t.done_reason).toBe("none") + expect(t.why_harness_stopped).toBe("none") + }) + + test("the harness-initiated challenge abort is not scored as a fatal error", () => { + const acc = RunAccounting.create() + acc.onIdleDoneChallengeIssued() + acc.onSessionError("MessageAbortedError", "aborted") + acc.onPromptResult({ finish: "error" }) + expect(acc.fatal).toBe(false) + expect(acc.termination().why_harness_stopped).toBe("none") + }) + + test("regression: an exhausted challenge send is fatal — the run must not exit 0", () => { + // The confirm-DONE challenge never reached the model: completion was NOT + // confirmed, so the run exits nonzero with harness attribution "error". + const acc = RunAccounting.create() + acc.onAssistantMessage({ id: "m1", agent: "build" }) + acc.onText("m1", "still churning") + acc.onStepFinish("m1", "stop") + acc.onIdleDoneChallengeIssued() + acc.onSessionError("IdleDoneChallengeFailed", "idle-done challenge prompt failed: ProviderError(503)") + expect(acc.fatal).toBe(true) + const t = acc.termination() + expect(t.why_harness_stopped).toBe("error") + expect(t.done_reason).toBe("none") + }) + + test("an abort BEFORE any challenge is still fatal (guard is challenge-scoped)", () => { + const acc = RunAccounting.create() + acc.onSessionError("MessageAbortedError", "aborted") + expect(acc.fatal).toBe(true) + }) + + test("unprompted DONE generations after a declined challenge report explicit_done", () => { + // Challenge at turn N; the model declines, does two more full turns of + // work, then asserts DONE on its own — that is honest explicit_done, not + // the heuristic's doing. + const acc = RunAccounting.create() + acc.onAssistantMessage({ id: "m1", agent: "build" }) + acc.onStepStart("m1") + acc.onIdleDoneChallengeIssued() + acc.onText("m1", "Remaining: wire the config flag. Continuing.") + acc.onStepFinish("m1", "stop") + acc.onAssistantMessage({ id: "m2", agent: "build" }) + acc.onStepStart("m2") + acc.onStepFinish("m2", "tool-calls") + acc.onAssistantMessage({ id: "m3", agent: "build" }) + acc.onStepStart("m3") + acc.onText("m3", "All checks green.\nDONE") + acc.onStepFinish("m3", "stop") + const t = acc.termination() + expect(t.done_reason).toBe("explicit_done") + expect(t.why_harness_stopped).toBe("none") + }) + + // altimate_change start — upstream_fix regression: onText/onStepFinish are + // independently overwritten by whichever message last fired each event. A + // DONE-bearing message that finishes "tool-calls" followed by a textless + // message that finishes "stop" must NOT pair the stale DONE flag from the + // FIRST message with the finish reason of the SECOND. + test("stale DONE from a tool-calls message is not paired with a later textless stop", () => { + const acc = RunAccounting.create() + acc.onAssistantMessage({ id: "m1", agent: "build" }) + acc.onStepStart("m1") + acc.onText("m1", "Wrapping up.\nDONE") + acc.onStepFinish("m1", "tool-calls") + acc.onAssistantMessage({ id: "m2", agent: "build" }) + acc.onStepStart("m2") + acc.onStepFinish("m2", "stop") // no onText for m2 — no text part at all + const t = acc.termination() + expect(t.done_reason).toBe("none") + expect(t.why_model_stopped).toBe("stop") + }) + // altimate_change end + + test("only ONE harness abort is forgiven per challenge — a second abort is fatal", () => { + const acc = RunAccounting.create() + acc.onIdleDoneChallengeIssued() + acc.onSessionError("MessageAbortedError", "aborted") + expect(acc.fatal).toBe(false) + acc.onSessionError("MessageAbortedError", "aborted again") + expect(acc.fatal).toBe(true) + }) + + test("a real error during the challenge continuation still wins over idle-done", () => { + const acc = RunAccounting.create() + acc.onAssistantMessage({ id: "m1", agent: "build" }) + acc.onIdleDoneChallengeIssued() + acc.onText("m1", "DONE") + acc.onStepFinish("m1", "stop") + acc.onSessionError("APIError", "boom") + expect(acc.termination().why_harness_stopped).toBe("error") + expect(acc.fatal).toBe(true) + }) + + // altimate_change start — upstream_fix regression: the interrupted prompt's + // abort can surface as EITHER onSessionError(MessageAbortedError) or an + // abnormal onPromptResult (or both) — both suppressions above are scoped to + // that one abort. Once the challenge reply itself has been sent, a genuine + // failure of the reply must not be absorbed by whichever suppression the + // interrupted prompt's abort left unused. + test("a genuine challenge-reply failure is fatal even if the interrupted prompt's abort only used one suppression channel", () => { + const acc = RunAccounting.create() + acc.onIdleDoneChallengeIssued() + // The interrupted prompt's abort surfaces ONLY via onSessionError — its + // own onPromptResult never reports an abnormal finish (e.g. it resolved + // "stop" before the abort landed), so challengeFinishSuppressed is never + // consumed here. + acc.onSessionError("MessageAbortedError", "aborted") + expect(acc.fatal).toBe(false) + // The challenge reply is now sent; its own errorless abnormal finish is a + // real failure of the confirmation, not a leftover of the abort above. + acc.onIdleDoneChallengeReplySent() + acc.onPromptResult({ finish: "other" }) + expect(acc.fatal).toBe(true) + expect(acc.termination().why_harness_stopped).toBe("error") + }) + + test("both suppression channels still forgive the same interrupted-prompt abort before the reply is sent", () => { + const acc = RunAccounting.create() + acc.onIdleDoneChallengeIssued() + acc.onSessionError("MessageAbortedError", "aborted") + acc.onPromptResult({ finish: "error" }) + expect(acc.fatal).toBe(false) + }) + // altimate_change end +}) diff --git a/packages/opencode/test/cli/run/before-exit.test.ts b/packages/opencode/test/cli/run/before-exit.test.ts new file mode 100644 index 0000000000..07f69b8bdc --- /dev/null +++ b/packages/opencode/test/cli/run/before-exit.test.ts @@ -0,0 +1,52 @@ +// Mirrors the beforeExit crash-handler contract from cli/cmd/run.ts: +// - the handler marks the run failed (exitCode 1) only while the run is +// still in flight (event loop drained before completion); +// - a run that completes normally sets runFinished, restores exitCode, and +// removes the listener, so a premature/spurious firing can never poison a +// successful run's rc; +// - fatal accounting remains the single authority for a nonzero rc afterwards. +// If the handler logic in run.ts changes, update this mirror to match. +import { describe, expect, test } from "bun:test" + +function makeRun() { + const proc = { exitCode: undefined as number | undefined, listeners: new Set<() => void>() } + let runFinished = false + const onBeforeExit = () => { + if (!runFinished) proc.exitCode = 1 + } + proc.listeners.add(onBeforeExit) + const fireBeforeExit = () => { + for (const listener of proc.listeners) listener() + } + const finish = (fatal: boolean) => { + runFinished = true + proc.exitCode = 0 + proc.listeners.delete(onBeforeExit) + if (fatal) proc.exitCode = 1 + } + return { proc, fireBeforeExit, finish } +} + +describe("run beforeExit rc stickiness", () => { + test("abandoned run (loop drains mid-flight) exits nonzero", () => { + const run = makeRun() + run.fireBeforeExit() + expect(run.proc.exitCode).toBe(1) + }) + + test("spurious firing before a successful completion does not poison rc 0", () => { + const run = makeRun() + run.fireBeforeExit() // event-loop gap while SSE/challenge promises pend + run.finish(false) + expect(run.proc.exitCode).toBe(0) + // any later firing is a no-op: listener removed + run.fireBeforeExit() + expect(run.proc.exitCode).toBe(0) + }) + + test("fatal accounting still exits nonzero after a normal drain", () => { + const run = makeRun() + run.finish(true) + expect(run.proc.exitCode).toBe(1) + }) +}) diff --git a/packages/opencode/test/cli/run/run-mode.test.ts b/packages/opencode/test/cli/run/run-mode.test.ts new file mode 100644 index 0000000000..ad094df148 --- /dev/null +++ b/packages/opencode/test/cli/run/run-mode.test.ts @@ -0,0 +1,145 @@ +import { afterEach, beforeEach, describe, expect, test } from "bun:test" +import { applyRunModeDefault } from "@/cli/cmd/run/run-mode" +import { Flag } from "@/flag/flag" + +// ─── `altimate-code run` implies run mode ─────────────────────── +// External drivers (harbor, CI) invoke `run` without exporting +// ALTIMATE_RUN_MODE; the run command applies the default itself, with an +// explicit ALTIMATE_RUN_MODE=0 opt-out. Interactive TUI/serve entrypoints +// never call applyRunModeDefault, so their behavior is untouched. + +describe("applyRunModeDefault", () => { + test("sets ALTIMATE_RUN_MODE=1 when unset", () => { + const env: Record = {} + applyRunModeDefault(env) + expect(env["ALTIMATE_RUN_MODE"]).toBe("1") + }) + + test("blank/whitespace value is treated as unset", () => { + for (const blank of ["", " "]) { + const env: Record = { ALTIMATE_RUN_MODE: blank } + applyRunModeDefault(env) + expect(env["ALTIMATE_RUN_MODE"]).toBe("1") + } + }) + + test("explicit opt-out ALTIMATE_RUN_MODE=0 is preserved", () => { + const env: Record = { ALTIMATE_RUN_MODE: "0" } + applyRunModeDefault(env) + expect(env["ALTIMATE_RUN_MODE"]).toBe("0") + }) + + test("explicit opt-out ALTIMATE_RUN_MODE=false is preserved", () => { + const env: Record = { ALTIMATE_RUN_MODE: "false" } + applyRunModeDefault(env) + expect(env["ALTIMATE_RUN_MODE"]).toBe("false") + }) + + test("explicit ALTIMATE_RUN_MODE=1 stays set", () => { + const env: Record = { ALTIMATE_RUN_MODE: "1" } + applyRunModeDefault(env) + expect(env["ALTIMATE_RUN_MODE"]).toBe("1") + }) + + test("--attach leaves the env untouched", () => { + const env: Record = {} + applyRunModeDefault(env, { attach: true }) + expect(env["ALTIMATE_RUN_MODE"]).toBeUndefined() + }) + + // A resumed run's history starts with an earlier invocation's task, so the + // pin selector must not treat "first user message" as this run's request. + test("a fresh run sets no resumed marker", () => { + const env: Record = {} + applyRunModeDefault(env) + expect(env["ALTIMATE_RUN_RESUMED"]).toBeUndefined() + }) + + test("a resumed run marks the session", () => { + const env: Record = {} + applyRunModeDefault(env, { resumed: true }) + expect(env["ALTIMATE_RUN_MODE"]).toBe("1") + expect(env["ALTIMATE_RUN_RESUMED"]).toBe("1") + }) + + test("the resumed marker is set even when run mode was exported explicitly", () => { + const env: Record = { ALTIMATE_RUN_MODE: "1" } + applyRunModeDefault(env, { resumed: true }) + expect(env["ALTIMATE_RUN_RESUMED"]).toBe("1") + }) + + test("--attach sets no resumed marker either — the agent runs remotely", () => { + const env: Record = {} + applyRunModeDefault(env, { attach: true, resumed: true }) + expect(env["ALTIMATE_RUN_RESUMED"]).toBeUndefined() + }) +}) + +describe("Flag.ALTIMATE_RUN_MODE integration", () => { + const saved = process.env["ALTIMATE_RUN_MODE"] + + beforeEach(() => { + delete process.env["ALTIMATE_RUN_MODE"] + }) + afterEach(() => { + if (saved === undefined) delete process.env["ALTIMATE_RUN_MODE"] + else process.env["ALTIMATE_RUN_MODE"] = saved + }) + + test("run implies run mode: default application arms the flag", () => { + expect(Flag.ALTIMATE_RUN_MODE).toBe(false) + applyRunModeDefault(process.env) + expect(Flag.ALTIMATE_RUN_MODE).toBe(true) + }) + + test("opt-out: ALTIMATE_RUN_MODE=0 keeps the flag disarmed", () => { + process.env["ALTIMATE_RUN_MODE"] = "0" + applyRunModeDefault(process.env) + expect(Flag.ALTIMATE_RUN_MODE).toBe(false) + }) + + test("--attach never arms the flag", () => { + applyRunModeDefault(process.env, { attach: true }) + expect(Flag.ALTIMATE_RUN_MODE).toBe(false) + }) +}) + +describe("Flag.parseRunModeValue (strict trimmed boolean parser)", () => { + test("trimmed truthy values arm", () => { + expect(Flag.parseRunModeValue("1")).toBe(true) + expect(Flag.parseRunModeValue(" 1 ")).toBe(true) + expect(Flag.parseRunModeValue("true")).toBe(true) + expect(Flag.parseRunModeValue(" TRUE ")).toBe(true) + }) + + test("falsy and blank values disarm", () => { + expect(Flag.parseRunModeValue("0")).toBe(false) + expect(Flag.parseRunModeValue(" false ")).toBe(false) + expect(Flag.parseRunModeValue("")).toBe(false) + expect(Flag.parseRunModeValue(" ")).toBe(false) + expect(Flag.parseRunModeValue(undefined)).toBe(false) + }) + + test("invalid values warn and disarm rather than silently flipping", () => { + const warnings: string[] = [] + const original = console.warn + console.warn = (...args: unknown[]) => warnings.push(args.join(" ")) + try { + expect(Flag.parseRunModeValue("yes-please")).toBe(false) + expect(warnings.some((w) => w.includes("ALTIMATE_RUN_MODE"))).toBe(true) + } finally { + console.warn = original + } + }) + + test("whitespace-padded env value arms the flag end to end", () => { + const saved = process.env["ALTIMATE_RUN_MODE"] + process.env["ALTIMATE_RUN_MODE"] = " 1 " + try { + expect(Flag.ALTIMATE_RUN_MODE).toBe(true) + } finally { + if (saved === undefined) delete process.env["ALTIMATE_RUN_MODE"] + else process.env["ALTIMATE_RUN_MODE"] = saved + } + }) +}) diff --git a/packages/opencode/test/cli/run/run-process.test.ts b/packages/opencode/test/cli/run/run-process.test.ts index bfb21aae2d..0d19a25426 100644 --- a/packages/opencode/test/cli/run/run-process.test.ts +++ b/packages/opencode/test/cli/run/run-process.test.ts @@ -73,21 +73,21 @@ describe("opencode run (non-interactive subprocess)", () => { 30_000, ) - // Locks in the current behavior: when the LLM stream errors mid-response - // (the prompt was accepted, then the upstream provider failed), opencode - // emits a session.error event and the process exits 0 today. - // - // This is debatable — a future cleanup might flip it to exit 1. If you're - // changing this expectation, do it deliberately and say so in the PR. + // (harness-improvement plan): a run that ends with an unrecovered session + // error is a fatal abort and must exit nonzero — an honest rc is the contract + // automation needs. This deliberately flips the previous "exits 0 today" + // contract lock-in (its comment asked for exactly this kind of deliberate + // change). Recoverable errors (context overflow handled by auto-compaction) + // still exit 0; see RunAccounting.onSessionError. cliIt.concurrent( - "mid-stream LLM error still exits 0 today (contract lock-in)", + "mid-stream LLM error exits nonzero (honest rc on fatal abort)", ({ llm, opencode }) => Effect.gen(function* () { yield* llm.fail("upstream provider exploded mid-stream") // bunRun: the compiled binary hangs handling a mid-stream stream error in the isolated test env - // (never exits); `bun run src` exits 0 in ~1s. See cliCommand in test/lib/cli-process.ts. + // (never exits); `bun run src` exits promptly. See cliCommand in test/lib/cli-process.ts. const result = yield* opencode.run("trigger midstream error", { timeoutMs: 30_000, bunRun: true }) - expect(result.exitCode).toBe(0) + expect(result.exitCode).toBe(1) }), 60_000, ) diff --git a/packages/opencode/test/session/compaction-fithead.test.ts b/packages/opencode/test/session/compaction-fithead.test.ts new file mode 100644 index 0000000000..e5ef9d52e2 --- /dev/null +++ b/packages/opencode/test/session/compaction-fithead.test.ts @@ -0,0 +1,122 @@ +import { describe, expect, test } from "bun:test" + +import { SessionCompaction } from "../../src/session/compaction" +import type { MessageV2 } from "../../src/session/message-v2" +import type { Provider } from "../../src/provider/provider" + +function userMessage(id: string, text: string): MessageV2.WithParts { + return { + info: { + id, + sessionID: "session-1", + role: "user", + time: { created: 1000 }, + model: { providerID: "local", modelID: "local-test-model" }, + }, + parts: [ + { + id: `${id}-part`, + sessionID: "session-1", + messageID: id, + type: "text", + text, + }, + ], + } as unknown as MessageV2.WithParts +} + +function assistantMessage(id: string, text: string): MessageV2.WithParts { + return { + info: { + id, + sessionID: "session-1", + role: "assistant", + time: { created: 1000 }, + model: { providerID: "local", modelID: "local-test-model" }, + }, + parts: [ + { + id: `${id}-part`, + sessionID: "session-1", + messageID: id, + type: "text", + text, + }, + ], + } as unknown as MessageV2.WithParts +} + +function model(context: number, output = 16384): Provider.Model { + return { + id: "local-test-model", + providerID: "local", + api: { npm: "@ai-sdk/openai-compatible" }, + limit: { context, output }, + } as unknown as Provider.Model +} + +describe("SessionCompaction.fitHead", () => { + test("leaves a small head untouched", async () => { + const head = [userMessage("m1", "short"), userMessage("m2", "also short")] + const result = await SessionCompaction.fitHead({ head, model: model(131072) }) + expect(result.dropped).toBe(0) + expect(result.head.length).toBe(2) + }) + + test("drops oldest messages until an oversized head fits the window", async () => { + // ~64 chars/token estimate baseline: 40 messages x 20k chars ≈ 200k tokens, + // far over a 32k window minus output reserve. + const head = Array.from({ length: 40 }, (_, i) => userMessage(`m${i}`, "x".repeat(20_000))) + const result = await SessionCompaction.fitHead({ head, model: model(32768, 8192) }) + expect(result.dropped).toBeGreaterThan(0) + expect(result.head.length).toBeLessThan(40) + expect(result.head.length).toBeGreaterThanOrEqual(1) + // survivors are the NEWEST messages (front of head is oldest) + expect(result.head.at(-1)).toBe(head.at(-1)!) + }) + + test("budget derives from the shared safety-fraction helper, not the raw limit", async () => { + // A head sized to fit the RAW window (minus output + prompt allowance) but + // NOT the safety-fraction-scaled window must still be trimmed — the + // summarization request itself would otherwise overflow under estimator error. + // 32k ctx, 8k out: raw budget ≈ 22.5k tokens; effective (0.65) ≈ 11.1k. + const head = Array.from({ length: 30 }, (_, i) => userMessage(`m${i}`, "x".repeat(2_400))) + const result = await SessionCompaction.fitHead({ head, model: model(32768, 8192), fraction: 0.65 }) + expect(result.dropped).toBeGreaterThan(0) + // At fraction 1 the same head fits the raw window untrimmed. + const raw = await SessionCompaction.fitHead({ head, model: model(32768, 8192), fraction: 1 }) + expect(raw.dropped).toBe(0) + }) + + test("cuts only at user boundaries — a single-leading-user head fails closed", async () => { + // One user turn followed by only assistant messages, far over budget. + // There is no later user boundary to cut at; the fallback must NOT slice + // mid-turn (an assistant-leading head draws a provider 400) — it returns + // the head unchanged, still led by the user message. + const head = [ + userMessage("m0", "x".repeat(20_000)), + ...Array.from({ length: 15 }, (_, i) => assistantMessage(`a${i}`, "x".repeat(20_000))), + ] + const result = await SessionCompaction.fitHead({ head, model: model(32768, 8192) }) + expect(result.dropped).toBe(0) + expect(result.head[0]!.info.role).toBe("user") + expect(result.head.length).toBe(16) + }) + + test("cut rounds forward to the next user boundary, never mid-turn", async () => { + // Alternating user/assistant turns over budget: every survivor head must + // start with a user message. + const head = Array.from({ length: 20 }, (_, i) => + i % 2 === 0 ? userMessage(`m${i}`, "x".repeat(20_000)) : assistantMessage(`a${i}`, "x".repeat(20_000)), + ) + const result = await SessionCompaction.fitHead({ head, model: model(32768, 8192) }) + expect(result.dropped).toBeGreaterThan(0) + expect(result.head[0]!.info.role).toBe("user") + }) + + test("zero-context models pass through unchanged", async () => { + const head = [userMessage("m1", "x".repeat(100_000))] + const result = await SessionCompaction.fitHead({ head, model: model(0) }) + expect(result.dropped).toBe(0) + }) +}) diff --git a/packages/opencode/test/session/compaction-ledger.test.ts b/packages/opencode/test/session/compaction-ledger.test.ts new file mode 100644 index 0000000000..fca6685e49 --- /dev/null +++ b/packages/opencode/test/session/compaction-ledger.test.ts @@ -0,0 +1,488 @@ +import { describe, test, expect } from "bun:test" +import { SessionCompaction } from "../../src/session/compaction" +import { Token } from "../../src/util/token" +import type { MessageV2 } from "../../src/session/message-v2" + +// Harness reliability / item 5 — unit gate: ledger determinism (5a) + append-only carry (5b). + +// ─── Helpers ──────────────────────────────────────────────────────────────── + +let partCounter = 0 + +function toolPart(overrides: { + tool: string + status?: "completed" | "error" | "pending" | "running" + input?: Record + output?: string + metadata?: Record + end?: number +}): any { + partCounter++ + const status = overrides.status ?? "completed" + const base = { + id: `part-${partCounter}`, + sessionID: "session-1", + messageID: `msg-${partCounter}`, + type: "tool", + callID: `call-${partCounter}`, + tool: overrides.tool, + } + if (status === "completed") + return { + ...base, + state: { + status, + input: overrides.input ?? {}, + output: overrides.output ?? "", + title: "t", + metadata: overrides.metadata ?? {}, + time: { start: 1000, end: overrides.end ?? 2000 }, + }, + } + if (status === "error") + return { + ...base, + state: { + status, + input: overrides.input ?? {}, + error: "boom", + metadata: overrides.metadata, + time: { start: 1000, end: overrides.end ?? 2000 }, + }, + } + if (status === "running") + return { ...base, state: { status, input: overrides.input ?? {}, time: { start: 1000 } } } + return { ...base, state: { status, input: overrides.input ?? {}, raw: "{}" } } +} + +function assistantMsg(parts: any[], info?: Partial>): MessageV2.WithParts { + partCounter++ + return { + info: { + id: `msg-a-${partCounter}`, + role: "assistant", + sessionID: "session-1", + ...info, + }, + parts, + } as unknown as MessageV2.WithParts +} + +function summaryMsg(text: string): MessageV2.WithParts { + partCounter++ + return { + info: { + id: `msg-s-${partCounter}`, + role: "assistant", + sessionID: "session-1", + summary: true, + finish: "stop", + }, + parts: [ + { + id: `part-s-${partCounter}`, + sessionID: "session-1", + messageID: `msg-s-${partCounter}`, + type: "text", + text, + }, + ], + } as unknown as MessageV2.WithParts +} + +// ─── 5a: buildLedger ──────────────────────────────────────────────────────── + +describe("SessionCompaction.buildLedger", () => { + test("records write/edit tool events with event-time mtimes (no fs access)", () => { + const messages = [ + assistantMsg([ + toolPart({ tool: "write", input: { filePath: "/repo/a.ts", content: "x" }, end: 5000 }), + toolPart({ tool: "edit", input: { filePath: "/repo/b.sql", oldString: "x", newString: "y" }, end: 6000 }), + ]), + ] + const ledger = SessionCompaction.buildLedger(messages) + expect(ledger.writes).toEqual([ + { path: "/repo/b.sql", mtime: 6000, tool: "edit" }, + { path: "/repo/a.ts", mtime: 5000, tool: "write" }, + ]) + }) + + test("last write wins per path and newest-first ordering is deterministic", () => { + const messages = [ + assistantMsg([ + toolPart({ tool: "write", input: { filePath: "/repo/a.ts" }, end: 5000 }), + toolPart({ tool: "edit", input: { filePath: "/repo/a.ts" }, end: 9000 }), + toolPart({ tool: "write", input: { filePath: "/repo/z.ts" }, end: 9000 }), + ]), + ] + const ledger = SessionCompaction.buildLedger(messages) + expect(ledger.writes).toEqual([ + { path: "/repo/a.ts", mtime: 9000, tool: "edit" }, + { path: "/repo/z.ts", mtime: 9000, tool: "write" }, + ]) + }) + + test("captures bash exit codes from metadata, command-agnostic", () => { + const messages = [ + assistantMsg([ + toolPart({ tool: "bash", input: { command: "bun test" }, metadata: { exit: 0 } }), + toolPart({ tool: "bash", input: { command: "some-arbitrary-cmd --flag" }, metadata: { exit: 1 } }), + ]), + ] + const ledger = SessionCompaction.buildLedger(messages) + expect(ledger.sawBash).toBe(true) + expect(ledger.calls).toEqual([ + { tool: "bash", detail: "bun test", exit: 0, errored: false }, + { tool: "bash", detail: "some-arbitrary-cmd --flag", exit: 1, errored: false }, + ]) + }) + + test("bash does NOT produce verified write entries (shell writes are unverifiable)", () => { + const messages = [ + assistantMsg([toolPart({ tool: "bash", input: { command: "sed -i s/a/b/ /repo/a.ts" }, metadata: { exit: 0 } })]), + ] + const ledger = SessionCompaction.buildLedger(messages) + expect(ledger.writes).toEqual([]) + expect(ledger.sawBash).toBe(true) + }) + + test("errored tool calls are recorded as errored and never count as writes", () => { + const messages = [ + assistantMsg([toolPart({ tool: "edit", status: "error", input: { filePath: "/repo/a.ts" } })]), + ] + const ledger = SessionCompaction.buildLedger(messages) + expect(ledger.writes).toEqual([]) + expect(ledger.calls[0]).toEqual({ tool: "edit", detail: "/repo/a.ts", exit: undefined, errored: true }) + }) + + test("errored bash still sets sawBash (it may have written before failing)", () => { + const messages = [assistantMsg([toolPart({ tool: "bash", status: "error", input: { command: "cp a b" } })])] + expect(SessionCompaction.buildLedger(messages).sawBash).toBe(true) + }) + + test("apply_patch writes come from result metadata files", () => { + const messages = [ + assistantMsg([ + toolPart({ + tool: "apply_patch", + input: { patchText: "..." }, + metadata: { files: [{ filePath: "/repo/c.py" }, { filePath: "/repo/d.py" }] }, + end: 7000, + }), + ]), + ] + const ledger = SessionCompaction.buildLedger(messages) + expect(ledger.writes.map((w) => w.path).sort()).toEqual(["/repo/c.py", "/repo/d.py"]) + expect(ledger.writes.every((w) => w.mtime === 7000 && w.tool === "apply_patch")).toBe(true) + }) + + test("apply_patch moves record the DESTINATION, and deletes record nothing", () => { + // `filePath` is the move SOURCE; the content lands at `movePath`. Naming the + // source sends the continuing agent back to the path that was removed. + const messages = [ + assistantMsg([ + toolPart({ + tool: "apply_patch", + input: { patchText: "..." }, + metadata: { + files: [ + { filePath: "/repo/old.py", movePath: "/repo/new.py", type: "update" }, + { filePath: "/repo/gone.py", type: "delete" }, + { filePath: "/repo/kept.py", type: "update" }, + ], + }, + end: 7000, + }), + ]), + ] + const ledger = SessionCompaction.buildLedger(messages) + expect(ledger.writes.map((w) => w.path).sort()).toEqual(["/repo/kept.py", "/repo/new.py"]) + }) + + test("pending and running parts are ignored (facts only)", () => { + const messages = [ + assistantMsg([ + toolPart({ tool: "write", status: "pending", input: { filePath: "/repo/a.ts" } }), + toolPart({ tool: "bash", status: "running", input: { command: "sleep 5" } }), + ]), + ] + const ledger = SessionCompaction.buildLedger(messages) + expect(ledger.writes).toEqual([]) + expect(ledger.calls).toEqual([]) + expect(ledger.sawBash).toBe(false) + }) + + test("deterministic: identical input yields identical ledger and rendering", () => { + const make = () => [ + assistantMsg([ + toolPart({ tool: "write", input: { filePath: "/repo/a.ts" }, end: 5000 }), + toolPart({ tool: "bash", input: { command: "make check" }, metadata: { exit: 0 } }), + toolPart({ tool: "read", input: { filePath: "/repo/a.ts" } }), + ]), + ] + const first = SessionCompaction.buildLedger(make()) + const second = SessionCompaction.buildLedger(make()) + expect(second).toEqual(first) + expect(SessionCompaction.renderLedger(second)).toBe(SessionCompaction.renderLedger(first)) + }) +}) + +// ─── 5a: renderLedger ─────────────────────────────────────────────────────── + +describe("SessionCompaction.renderLedger", () => { + const sample = () => + SessionCompaction.buildLedger([ + assistantMsg([ + toolPart({ tool: "write", input: { filePath: "/repo/models/orders.sql" }, end: 1_700_000_000_000 }), + toolPart({ tool: "bash", input: { command: "run-all-checks" }, metadata: { exit: 0 } }), + ]), + ]) + + test("empty ledger renders empty string", () => { + expect(SessionCompaction.renderLedger({ writes: [], calls: [], sawBash: false })).toBe("") + }) + + test("a budget too small for even the header renders nothing, not a bare header", () => { + // `ledger_max_tokens: 0` is accepted by the schema; truncation used to stop + // at the header and return it, injecting text the tail calculation had + // budgeted at zero. + expect(SessionCompaction.renderLedger(sample(), { maxTokens: 0 })).toBe("") + expect(SessionCompaction.renderLedger(sample(), { maxTokens: 3 })).toBe("") + }) + + test("contains verified writes with ISO event time, advisory wording, and unverified-shell note", () => { + const text = SessionCompaction.renderLedger(sample()) + expect(text).toContain("/repo/models/orders.sql") + expect(text).toContain(new Date(1_700_000_000_000).toISOString()) + expect(text).toContain("last written by you at") + expect(text).toContain("possible but unverified") + // advisory, never an absolute prohibition + expect(text).toContain("re-read a file only if") + expect(text).toContain("IDE edits") + expect(text).not.toMatch(/never re-read|do not read/i) + }) + + test("lists at most recentCalls tool calls, newest first", () => { + const parts = [] + for (let i = 1; i <= 15; i++) parts.push(toolPart({ tool: "bash", input: { command: `cmd-${i}` }, metadata: { exit: 0 } })) + const ledger = SessionCompaction.buildLedger([assistantMsg(parts)]) + const text = SessionCompaction.renderLedger(ledger, { recentCalls: 10 }) + expect(text).toContain("cmd-15") + expect(text).toContain("cmd-6") + expect(text).not.toContain("cmd-5\n") + expect(text).not.toContain("cmd-1 ") + // newest first + expect(text.indexOf("cmd-15")).toBeLessThan(text.indexOf("cmd-6")) + expect(text).toContain("last 10 of 15") + }) + + test("tail-truncates to the token cap, preserving the header and writes section", () => { + const parts = [toolPart({ tool: "write", input: { filePath: "/repo/first.ts" }, end: 1000 })] + for (let i = 0; i < 50; i++) + parts.push(toolPart({ tool: "bash", input: { command: `x`.repeat(90) + `-${i}` }, metadata: { exit: 0 } })) + const ledger = SessionCompaction.buildLedger([assistantMsg(parts)]) + const capped = SessionCompaction.renderLedger(ledger, { maxTokens: 120, recentCalls: 50 }) + expect(Token.estimate(capped)).toBeLessThanOrEqual(120) + expect(capped.split("\n")[0]).toContain("Session state ledger") + expect(capped).toContain("/repo/first.ts") + }) + + test("default cap is 500 tokens (config default, plan provenance)", () => { + const parts = [] + for (let i = 0; i < 200; i++) + parts.push(toolPart({ tool: "bash", input: { command: "y".repeat(95) + i }, metadata: { exit: 0 } })) + const ledger = SessionCompaction.buildLedger([assistantMsg(parts)]) + const text = SessionCompaction.renderLedger(ledger, { recentCalls: 200 }) + expect(SessionCompaction.LEDGER_MAX_TOKENS).toBe(500) + expect(Token.estimate(text)).toBeLessThanOrEqual(500) + }) + + test("null exit code renders as unknown, error state as errored", () => { + const ledger = SessionCompaction.buildLedger([ + assistantMsg([ + toolPart({ tool: "bash", input: { command: "killed-cmd" }, metadata: { exit: null } }), + toolPart({ tool: "glob", status: "error", input: { pattern: "**/*.ts" } }), + ]), + ]) + const text = SessionCompaction.renderLedger(ledger) + expect(text).toContain("bash (exit ?) — killed-cmd") + expect(text).toContain("glob (errored) — **/*.ts") + }) +}) + +// ─── 5b: extractAccomplished / corroborateCarry / renderCarryAnchors ──────── + +describe("SessionCompaction.extractAccomplished", () => { + test("parses bullets under ## Accomplished only, stopping at the next heading", () => { + const summary = [ + "## Goal", + "- not this", + "## Accomplished", + "- built /repo/models/orders.sql", + "* verified row counts", + "not a bullet", + "## Relevant files / directories", + "- /repo/models", + ].join("\n") + expect(SessionCompaction.extractAccomplished(summary)).toEqual([ + { text: "built /repo/models/orders.sql", priorStatus: undefined }, + { text: "verified row counts", priorStatus: undefined }, + ]) + }) + + test("preserves prior carry tags", () => { + const summary = ["## Accomplished", "- [verified] created a.sql", "- [claimed, unverified] fixed the test"].join( + "\n", + ) + expect(SessionCompaction.extractAccomplished(summary)).toEqual([ + { text: "created a.sql", priorStatus: "verified" }, + { text: "fixed the test", priorStatus: "claimed, unverified" }, + ]) + }) + + test("returns empty for summaries without the section", () => { + expect(SessionCompaction.extractAccomplished("## Goal\n- stuff")).toEqual([]) + }) +}) + +describe("SessionCompaction.corroborateCarry", () => { + const ledger = SessionCompaction.buildLedger([ + assistantMsg([ + toolPart({ tool: "write", input: { filePath: "/repo/models/orders.sql" }, end: 5000 }), + toolPart({ tool: "bash", input: { command: "python scripts/export.py --out report.csv" }, metadata: { exit: 0 } }), + toolPart({ tool: "bash", input: { command: "validate broken_thing.json" }, metadata: { exit: 1 } }), + ]), + ]) + + test("item naming a verified-written file carries as verified", () => { + const out = SessionCompaction.corroborateCarry([{ text: "created models/orders.sql with dedup logic" }], ledger) + expect(out).toEqual([{ text: "created models/orders.sql with dedup logic", status: "verified" }]) + }) + + test("item with no corroborating event carries as claimed, unverified", () => { + const out = SessionCompaction.corroborateCarry([{ text: "generated final_report.pdf and emailed it" }], ledger) + expect(out[0]!.status).toBe("claimed, unverified") + }) + + test("a directory-qualified artifact is not corroborated by a same-basename write elsewhere", () => { + // Basename fallback exists for bare filenames. Applying it to a qualified + // token lets an unrelated `test/orders.sql` verify `src/orders.sql`, and + // because carry status is append-only that wrong fact never gets corrected. + const out = SessionCompaction.corroborateCarry([{ text: "created test/orders.sql fixtures" }], ledger) + expect(out[0]!.status).toBe("claimed, unverified") + }) + + test("a bare filename still matches the write's basename", () => { + const out = SessionCompaction.corroborateCarry([{ text: "created orders.sql" }], ledger) + expect(out[0]!.status).toBe("verified") + }) + + test("zero-exit command naming the artifact corroborates; failed command does not", () => { + const out = SessionCompaction.corroborateCarry( + [{ text: "exported report.csv" }, { text: "validated broken_thing.json" }], + ledger, + ) + expect(out[0]!.status).toBe("verified") + expect(out[1]!.status).toBe("claimed, unverified") + }) + + test("append-only: a prior [verified] tag is preserved even without current evidence", () => { + const out = SessionCompaction.corroborateCarry( + [{ text: "shipped ancient_artifact.xyz", priorStatus: "verified" }], + ledger, + ) + expect(out[0]!.status).toBe("verified") + }) + + test("a prior unverified claim can be promoted when evidence appears", () => { + const out = SessionCompaction.corroborateCarry( + [{ text: "created models/orders.sql", priorStatus: "claimed, unverified" }], + ledger, + ) + expect(out[0]!.status).toBe("verified") + }) + + test("prose without artifact tokens never matches spuriously", () => { + const out = SessionCompaction.corroborateCarry([{ text: "discussed the approach with the user" }], ledger) + expect(out[0]!.status).toBe("claimed, unverified") + }) +}) + +describe("SessionCompaction.renderCarryAnchors", () => { + test("empty items render empty string", () => { + expect(SessionCompaction.renderCarryAnchors([])).toBe("") + }) + + test("renders tagged items with carry-forward instructions", () => { + const text = SessionCompaction.renderCarryAnchors([ + { text: "built a.sql", status: "verified" }, + { text: "wrote docs", status: "claimed, unverified" }, + ]) + expect(text).toContain("append-only carry") + expect(text).toContain("- [verified] built a.sql") + expect(text).toContain("- [claimed, unverified] wrote docs") + expect(text).toContain("keep the tag") + }) + + test("over budget drops the OLDEST items first and stays under the cap", () => { + const items = [] + for (let i = 0; i < 60; i++) items.push({ text: `item-${i} ` + "z".repeat(80), status: "verified" as const }) + const text = SessionCompaction.renderCarryAnchors(items, 300) + expect(Token.estimate(text)).toBeLessThanOrEqual(300) + expect(text).not.toContain("item-0 ") + expect(text).toContain("item-59 ") + }) + + test("deterministic rendering", () => { + const items = [ + { text: "one", status: "verified" as const }, + { text: "two", status: "claimed, unverified" as const }, + ] + expect(SessionCompaction.renderCarryAnchors(items)).toBe(SessionCompaction.renderCarryAnchors(items)) + }) +}) + +// ─── latestSummaryText ────────────────────────────────────────────────────── + +describe("SessionCompaction.latestSummaryText", () => { + test("returns the most recent finished, non-errored summary", () => { + const messages = [ + summaryMsg("## Accomplished\n- old item"), + assistantMsg([toolPart({ tool: "read", input: { filePath: "/x" } })]), + summaryMsg("## Accomplished\n- new item"), + ] + expect(SessionCompaction.latestSummaryText(messages)).toContain("new item") + }) + + test("skips errored or unfinished summaries", () => { + const errored = summaryMsg("## Accomplished\n- bad") as any + errored.info.error = { name: "x" } + const unfinished = summaryMsg("## Accomplished\n- incomplete") as any + delete unfinished.info.finish + const messages = [summaryMsg("## Accomplished\n- good"), unfinished, errored] + expect(SessionCompaction.latestSummaryText(messages)).toContain("good") + }) + + test("undefined when no summary exists", () => { + expect(SessionCompaction.latestSummaryText([assistantMsg([])])).toBeUndefined() + }) +}) + +// ─── Leak guard: no vertical tokens in the generic mechanism (hard requirement) ─ + +describe("leak guard", () => { + test("ledger output for a dbt-style command is treated identically to any other command", () => { + const mk = (cmd: string) => + SessionCompaction.renderLedger( + SessionCompaction.buildLedger([ + assistantMsg([toolPart({ tool: "bash", input: { command: cmd }, metadata: { exit: 0 } })]), + ]), + ) + const a = mk("dbt build --select orders") + const b = mk("qqq build --select orders".replace("build", "frobnicate")) + // Same structure: swapping the command text is the ONLY difference (no classifier). + expect(a.replace("dbt build --select orders", "CMD")).toBe( + b.replace("qqq frobnicate --select orders", "CMD"), + ) + }) +}) diff --git a/packages/opencode/test/session/compaction-loop.test.ts b/packages/opencode/test/session/compaction-loop.test.ts index e762ca205e..6325ff6ce3 100644 --- a/packages/opencode/test/session/compaction-loop.test.ts +++ b/packages/opencode/test/session/compaction-loop.test.ts @@ -1,4 +1,4 @@ -import { describe, expect, test } from "bun:test" +import { afterAll, beforeAll, describe, expect, test } from "bun:test" import { SessionCompaction } from "../../src/session/compaction" import { Instance } from "../../src/project/instance" import { Log } from "../../src/util/log" @@ -404,6 +404,16 @@ function createModel(opts: { } describe("session.compaction.isOverflow boundary conditions", () => { + // These tests pin the RAW-limit boundary math, so disable the estimator + // safety margin (fraction 1 = raw limit). Default-margin behavior is covered + // in compaction-safety-fraction.test.ts. + beforeAll(() => { + process.env["ALTIMATE_CONTEXT_SAFETY_FRACTION"] = "1" + }) + afterAll(() => { + delete process.env["ALTIMATE_CONTEXT_SAFETY_FRACTION"] + }) + test("tokens exactly at usable limit triggers overflow", async () => { await using tmp = await tmpdir() await Instance.provide({ @@ -601,3 +611,74 @@ describe("session.compaction.prune with disabled config", () => { }) }) }) + +describe("session.compaction.process circuit breaker", () => { + // Real-module gate for the attempt>3 breaker: it must return "stop" (never + // undefined — the prompt loop treats non-"continue" as stop, and undefined + // previously fell through to `continue`, re-entering process() in a busy + // loop) and must clear the per-session counter so a later prompt gets a + // fresh bounded set of attempts. + test("attempt>3 returns 'stop' and resets the attempt counter", async () => { + await using tmp = await tmpdir() + await Instance.provide({ + directory: tmp.path, + fn: async () => { + const sessionID = "ses_breaker_test" as any + const input = () => ({ + messages: [] as any[], + parentID: "msg_missing" as any, + abort: new AbortController().signal, + sessionID, + auto: true, + }) + // Attempts 1-3: breaker not yet tripped; the missing parent throws. + for (let i = 0; i < 3; i++) { + await expect(SessionCompaction.process(input())).rejects.toThrow(/Compaction parent/) + } + // Attempt 4: breaker trips BEFORE the parent lookup and returns "stop". + expect(await SessionCompaction.process(input())).toBe("stop") + // Counter was cleared: the next call is attempt 1 again (throws, not "stop"). + await expect(SessionCompaction.process(input())).rejects.toThrow(/Compaction parent/) + }, + }) + }) +}) + +describe("small-window retained-content clamp", () => { + // Regression: on small-window models the verbatim tail budget used to be + // sized from the raw limit (floor 2000, cap 8000) and could exceed the + // overflow trigger by itself, so every compaction immediately re-triggered + // (per-turn summarization churn). Tail + ledger must stay well below the + // trigger threshold. + const threshold = (model: Provider.Model, cfg: any = {}) => { + const maxOutput = model.limit.output ?? 4096 + const headroom = Math.max(cfg.compaction?.reserved ?? 20_000, maxOutput) + const base = model.limit.input ?? model.limit.context + return SessionCompaction.overflowThreshold({ base, headroom, fraction: 1 }) + } + + test("32K model: tail + ledger can never alone reach the overflow trigger", () => { + const model = createModel({ context: 32_768, output: 8_192 }) + const cfg = {} as any + const budget = SessionCompaction.preserveRecentBudget({ cfg, model }) + const trigger = threshold(model) + expect(trigger).toBe(12_768) + expect(budget + SessionCompaction.LEDGER_MAX_TOKENS).toBeLessThanOrEqual(Math.floor(trigger / 2)) + expect(budget).toBeGreaterThan(0) + }) + + test("explicit preserve_recent_tokens config is clamped too — the invariant is unconditional", () => { + const model = createModel({ context: 32_768, output: 8_192 }) + const cfg = { compaction: { preserve_recent_tokens: 20_000 } } as any + const budget = SessionCompaction.preserveRecentBudget({ cfg, model }) + const trigger = threshold(model) + expect(budget + SessionCompaction.LEDGER_MAX_TOKENS).toBeLessThanOrEqual(Math.floor(trigger / 2)) + }) + + test("large-window models keep the normal tail budget", () => { + const model = createModel({ context: 200_000, output: 32_000 }) + const budget = SessionCompaction.preserveRecentBudget({ cfg: {} as any, model }) + // usable-derived candidate caps at 8000 and the clamp does not bind. + expect(budget).toBe(8_000) + }) +}) diff --git a/packages/opencode/test/session/compaction-safety-fraction.test.ts b/packages/opencode/test/session/compaction-safety-fraction.test.ts new file mode 100644 index 0000000000..9a86255627 --- /dev/null +++ b/packages/opencode/test/session/compaction-safety-fraction.test.ts @@ -0,0 +1,213 @@ +import { afterEach, beforeEach, describe, expect, test } from "bun:test" +import { SessionCompaction } from "../../src/session/compaction" +import { Instance } from "../../src/project/instance" +import { Log } from "../../src/util/log" +import { tmpdir } from "../fixture/fixture" +import type { Provider } from "../../src/provider/provider" + +Log.init({ print: false }) + +// ─── estimator safety margin ───────────────────────────────────── +// Token.estimate (chars-based) undercounts real tokenization of dense +// SQL/JSON by up to ~1.55x. The safety fraction corrects for that ONLY where +// estimates are involved: estimate-derived budgets use base * fraction, and +// the estimated component passed to isOverflow is inflated by 1/fraction. +// Provider-reported usage is exact and compares against the raw limit minus +// headroom. + +function createModel(opts: { context: number; output: number; input?: number }): Provider.Model { + return { + id: "test-model", + providerID: "test" as any, + name: "Test", + limit: { + context: opts.context, + input: opts.input, + output: opts.output, + }, + cost: { input: 0, output: 0, cache: { read: 0, write: 0 } }, + capabilities: { + toolcall: true, + attachment: false, + reasoning: false, + temperature: true, + input: { text: true, image: false, audio: false, video: false }, + output: { text: true, image: false, audio: false, video: false }, + }, + api: { npm: "@ai-sdk/anthropic" }, + options: {}, + } as Provider.Model +} + +function tokens(input: number) { + return { input, output: 0, reasoning: 0, cache: { read: 0, write: 0 } } +} + +beforeEach(() => { + delete process.env["ALTIMATE_CONTEXT_SAFETY_FRACTION"] +}) +afterEach(() => { + delete process.env["ALTIMATE_CONTEXT_SAFETY_FRACTION"] +}) + +describe("contextSafetyFraction resolution", () => { + test("defaults to 0.65", () => { + expect(SessionCompaction.contextSafetyFraction(undefined)).toBe(0.65) + expect(SessionCompaction.contextSafetyFraction({})).toBe(0.65) + }) + + test("config key overrides the default", () => { + expect(SessionCompaction.contextSafetyFraction({ compaction: { context_safety_fraction: 0.8 } })).toBe(0.8) + }) + + test("env var overrides config", () => { + process.env["ALTIMATE_CONTEXT_SAFETY_FRACTION"] = "0.9" + expect(SessionCompaction.contextSafetyFraction({ compaction: { context_safety_fraction: 0.5 } })).toBe(0.9) + }) + + test("non-numeric env var is ignored", () => { + process.env["ALTIMATE_CONTEXT_SAFETY_FRACTION"] = "banana" + expect(SessionCompaction.contextSafetyFraction(undefined)).toBe(0.65) + }) + + test("numeric-prefix garbage is ignored (Number over the full value, not parseFloat)", () => { + process.env["ALTIMATE_CONTEXT_SAFETY_FRACTION"] = "0.9junk" + expect(SessionCompaction.contextSafetyFraction(undefined)).toBe(0.65) + expect(SessionCompaction.contextSafetyFraction({ compaction: { context_safety_fraction: 0.5 } })).toBe(0.5) + }) + + test("surrounding whitespace is tolerated", () => { + process.env["ALTIMATE_CONTEXT_SAFETY_FRACTION"] = " 0.9 " + expect(SessionCompaction.contextSafetyFraction(undefined)).toBe(0.9) + }) + + test("clamps to [0.1, 1]", () => { + expect(SessionCompaction.contextSafetyFraction({ compaction: { context_safety_fraction: 2 } })).toBe(1) + expect(SessionCompaction.contextSafetyFraction({ compaction: { context_safety_fraction: 0.01 } })).toBe(0.1) + process.env["ALTIMATE_CONTEXT_SAFETY_FRACTION"] = "-3" + expect(SessionCompaction.contextSafetyFraction(undefined)).toBe(0.1) + }) +}) + +describe("effectiveContextLimit", () => { + test("floors base * fraction", () => { + expect(SessionCompaction.effectiveContextLimit(65_536, 0.65)).toBe(42_598) + expect(SessionCompaction.effectiveContextLimit(100_000, 0.65)).toBe(65_000) + expect(SessionCompaction.effectiveContextLimit(100_000, 1)).toBe(100_000) + }) + + test("worst-observed 1.55x underestimate still fits inside the raw window", () => { + // The 65K-context incident model: estimated 45.8K = real >65K → provider 400. + // With the default fraction, the compaction trigger sits low enough that + // 1.55x the trigger PLUS the 20K headroom stays inside the raw context. + const context = 65_536 + const headroom = 20_000 + const effective = SessionCompaction.effectiveContextLimit(context, 0.65) + const threshold = effective - headroom + expect(Math.ceil(threshold * 1.55) + headroom).toBeLessThanOrEqual(context) + }) +}) + +describe("isOverflow two regimes: exact provider counts vs estimated components", () => { + test("provider-reported usage compares against the RAW limit minus headroom", async () => { + await using tmp = await tmpdir() + await Instance.provide({ + directory: tmp.path, + fn: async () => { + // context=100K, output=32K → headroom = max(20K, 32K) = 32K → raw usable = 68K. + // Exact counts must NOT be scaled by the default 0.65 fraction — that + // forfeited ~35% of every window for estimate-free sessions. + const model = createModel({ context: 100_000, output: 32_000 }) + expect(await SessionCompaction.isOverflow({ tokens: tokens(68_000), model })).toBe(true) + expect(await SessionCompaction.isOverflow({ tokens: tokens(67_999), model })).toBe(false) + // Well above the old fraction-scaled trigger (33K) but under the raw + // boundary: still no overflow. + expect(await SessionCompaction.isOverflow({ tokens: tokens(50_000), model })).toBe(false) + }, + }) + }) + + test("estimated component is inflated by 1/fraction (default 0.65)", async () => { + await using tmp = await tmpdir() + await Instance.provide({ + directory: tmp.path, + fn: async () => { + // Raw usable = 68K. Provider count 60K + estimated tail 6K: + // adjusted = 60K + ceil(6000 / 0.65) = 60K + 9,231 = 69,231 → overflow. + const model = createModel({ context: 100_000, output: 32_000 }) + expect( + await SessionCompaction.isOverflow({ tokens: tokens(60_000), estimatedTokens: 6_000, model }), + ).toBe(true) + // 60K + ceil(5000 / 0.65) = 67,693 < 68K → no overflow. + expect( + await SessionCompaction.isOverflow({ tokens: tokens(60_000), estimatedTokens: 5_000, model }), + ).toBe(false) + }, + }) + }) + + test("fraction 1 disables estimate inflation", async () => { + process.env["ALTIMATE_CONTEXT_SAFETY_FRACTION"] = "1" + await using tmp = await tmpdir() + await Instance.provide({ + directory: tmp.path, + fn: async () => { + const model = createModel({ context: 100_000, output: 32_000 }) + expect( + await SessionCompaction.isOverflow({ tokens: tokens(60_000), estimatedTokens: 8_000, model }), + ).toBe(true) + expect( + await SessionCompaction.isOverflow({ tokens: tokens(60_000), estimatedTokens: 7_999, model }), + ).toBe(false) + }, + }) + }) + + test("config key context_safety_fraction scales the estimated component only", async () => { + await using tmp = await tmpdir({ + init: async (dir) => { + await Bun.write(`${dir}/opencode.json`, JSON.stringify({ compaction: { context_safety_fraction: 0.5 } })) + }, + }) + await Instance.provide({ + directory: tmp.path, + fn: async () => { + const model = createModel({ context: 100_000, output: 32_000 }) + // Exact counts still use the raw boundary despite fraction 0.5. + expect(await SessionCompaction.isOverflow({ tokens: tokens(67_999), model })).toBe(false) + // Estimated component doubles: 64K + 4000/0.5 = 72K → overflow; + // 63.9K + 2000/0.5 = 67.9K → no overflow. + expect( + await SessionCompaction.isOverflow({ tokens: tokens(64_000), estimatedTokens: 4_000, model }), + ).toBe(true) + expect( + await SessionCompaction.isOverflow({ tokens: tokens(63_900), estimatedTokens: 2_000, model }), + ).toBe(false) + }, + }) + }) + + test("small-context models keep the full raw usable window for exact counts", async () => { + await using tmp = await tmpdir() + await Instance.provide({ + directory: tmp.path, + fn: async () => { + // context=32,768, output=5K → headroom = max(20K, 5K) = 20K → raw usable = 12,768. + const model = createModel({ context: 32_768, output: 5_000 }) + expect(await SessionCompaction.isOverflow({ tokens: tokens(12_768), model })).toBe(true) + expect(await SessionCompaction.isOverflow({ tokens: tokens(12_767), model })).toBe(false) + }, + }) + }) + + test("base <= headroom still disables compaction entirely", async () => { + await using tmp = await tmpdir() + await Instance.provide({ + directory: tmp.path, + fn: async () => { + const model = createModel({ context: 16_000, output: 32_000 }) + expect(await SessionCompaction.isOverflow({ tokens: tokens(1_000_000), model })).toBe(false) + }, + }) + }) +}) diff --git a/packages/opencode/test/session/compaction-summarizer-integrity.test.ts b/packages/opencode/test/session/compaction-summarizer-integrity.test.ts new file mode 100644 index 0000000000..90b3e2a3b0 --- /dev/null +++ b/packages/opencode/test/session/compaction-summarizer-integrity.test.ts @@ -0,0 +1,418 @@ +import { afterAll, beforeEach, describe, expect, mock, spyOn, test } from "bun:test" +import { SessionCompaction } from "../../src/session/compaction" +import { Session } from "../../src/session" +import { MessageV2 } from "../../src/session/message-v2" +import { SessionProcessor } from "../../src/session/processor" +import { Provider } from "../../src/provider/provider" +import { Agent } from "../../src/agent/agent" +import { Config } from "../../src/config/config" +import { Plugin } from "../../src/plugin" +import { Telemetry } from "../../src/telemetry" +import { Bus } from "../../src/bus" +import { Instance } from "../../src/project/instance" +import { Log } from "../../src/util/log" +import { MessageID, PartID, SessionID } from "../../src/session/schema" +import { ModelID, ProviderID } from "../../src/provider/schema" +import { NudgeArbiter } from "../../src/session/nudge" +import { SessionTermination } from "../../src/session/termination" + +Log.init({ print: false }) + +// ─── Harness reliability + (item 3) unit gates ─────────────────── +// the auto-compaction continue message must carry the original user +// message's format/tools/system/variant (like the replay branch), so the +// first auto-compaction cannot silently widen the permission surface. +// the summarizer call passes explicit toolChoice "none", and a "continue" +// result with no non-empty summary text is retried ONCE, then errored — +// never committed. +// +// SessionCompaction.process wires imperative singletons directly (SessionProcessor, +// Provider, Agent, Session, ...), so these tests use the spy-based mocking pattern +// (see test/altimate/enhance-prompt.test.ts) — never mock.module() for shared +// infrastructure modules. + +const ref = { providerID: ProviderID.make("test"), modelID: ModelID.make("test-model") } + +const fakeModel = { + id: "test-model", + providerID: "test", + name: "Test", + limit: { context: 100_000, output: 32_000 }, + cost: { input: 0, output: 0, cache: { read: 0, write: 0 } }, + capabilities: { + toolcall: true, + attachment: false, + reasoning: false, + temperature: true, + input: { text: true, image: false, audio: false, video: false }, + output: { text: true, image: false, audio: false, video: false }, + }, + api: { npm: "@ai-sdk/anthropic" }, + options: {}, +} as unknown as Provider.Model + +// In-memory message/part store standing in for the session database. +const store = { + messages: [] as any[], + parts: [] as any[], +} + +type ProcessBehavior = (streamInput: any, message: any) => Promise<"continue" | "stop" | "compact"> +let processCalls: any[] = [] +let processBehaviors: ProcessBehavior[] = [] + +function writeSummary(text: string): ProcessBehavior { + return async (_streamInput, message) => { + store.parts.push({ + id: PartID.ascending(), + messageID: message.id, + sessionID: message.sessionID, + type: "text", + text, + }) + return "continue" + } +} + +const noSummary: ProcessBehavior = async () => "continue" + +// Instance.directory / Instance.worktree are getters that require ambient +// instance context; override them for the duration of this file. +const instanceDescriptors = { + directory: Object.getOwnPropertyDescriptor(Instance, "directory")!, + worktree: Object.getOwnPropertyDescriptor(Instance, "worktree")!, +} +Object.defineProperty(Instance, "directory", { configurable: true, get: () => "/tmp/compaction-test" }) +Object.defineProperty(Instance, "worktree", { configurable: true, get: () => "/tmp/compaction-test" }) + +spyOn(Config, "get").mockImplementation(async () => ({}) as any) +spyOn(Provider, "getModel").mockImplementation(async () => fakeModel) +spyOn(Agent, "get").mockImplementation( + async () => ({ name: "compaction", mode: "primary", options: {}, permission: [] }) as any, +) +spyOn(Plugin, "trigger").mockImplementation(async (_name: any, _input: any, output: any) => output) +spyOn(Telemetry, "track").mockImplementation((() => {}) as any) +spyOn(Bus, "publish").mockImplementation(async () => {}) +spyOn(MessageV2, "toModelMessages").mockImplementation(async () => []) +spyOn(MessageV2, "get").mockImplementation( + (input: any) => + ({ + info: store.messages.find((m) => m.id === input.messageID), + parts: store.parts.filter((p) => p.messageID === input.messageID), + }) as any, +) +spyOn(Session, "updateMessage").mockImplementation((async (msg: any) => { + store.messages.push(msg) + return msg +}) as any) +spyOn(Session, "updatePart").mockImplementation((async (part: any) => { + store.parts.push(part) + return part +}) as any) +spyOn(SessionProcessor, "create").mockImplementation((input: any) => { + const message = input.assistantMessage + return { + get message() { + return message + }, + partFromToolCall: () => undefined, + async process(streamInput: any) { + processCalls.push(streamInput) + const behavior = processBehaviors.shift() ?? writeSummary("summary") + return behavior(streamInput, message) + }, + } as any +}) + +afterAll(() => { + mock.restore() + Object.defineProperty(Instance, "directory", instanceDescriptors.directory) + Object.defineProperty(Instance, "worktree", instanceDescriptors.worktree) +}) + +beforeEach(() => { + store.messages = [] + store.parts = [] + processCalls = [] + processBehaviors = [] +}) + +let counter = 0 +function freshSessionID() { + counter += 1 + return SessionID.make(`ses_summarizer_test_${counter}`) +} + +function history(sessionID: SessionID, opts?: { userFields?: Record }) { + const userID = MessageID.ascending() + const assistantID = MessageID.ascending() + const markerID = MessageID.ascending() + const messages = [ + { + info: { + id: userID, + sessionID, + role: "user", + time: { created: 1 }, + agent: "build", + model: ref, + ...(opts?.userFields ?? {}), + }, + parts: [{ id: PartID.ascending(), messageID: userID, sessionID, type: "text", text: "do the task" }], + }, + { + info: { + id: assistantID, + sessionID, + role: "assistant", + parentID: userID, + time: { created: 2 }, + mode: "build", + agent: "build", + path: { cwd: "/tmp/compaction-test", root: "/tmp/compaction-test" }, + cost: 0, + tokens: { input: 1, output: 1, reasoning: 0, cache: { read: 0, write: 0 } }, + modelID: ref.modelID, + providerID: ref.providerID, + finish: "end_turn", + }, + parts: [{ id: PartID.ascending(), messageID: assistantID, sessionID, type: "text", text: "working on it" }], + }, + { + info: { + id: markerID, + sessionID, + role: "user", + time: { created: 3 }, + agent: "build", + model: ref, + }, + parts: [{ id: PartID.ascending(), messageID: markerID, sessionID, type: "compaction", auto: true }], + }, + ] as any[] + return { messages, markerID } +} + +function run(input: { sessionID: SessionID; messages: any[]; markerID: MessageID }) { + return SessionCompaction.process({ + sessionID: input.sessionID, + messages: input.messages, + parentID: input.markerID, + abort: new AbortController().signal, + auto: true, + }) +} + +describe("session.compaction continue-message contract (/ item 12)", () => { + test("continue message carries original tools/system/format/variant through auto-compaction", async () => { + const sessionID = freshSessionID() + const { messages, markerID } = history(sessionID, { + userFields: { + tools: { bash: true, edit: false }, + system: "custom system prompt", + variant: "high", + format: { type: "json" }, + }, + }) + processBehaviors = [writeSummary("a real summary")] + + const result = await run({ sessionID, messages, markerID }) + + expect(result).toBe("continue") + const continueMsg = store.messages.filter((m) => m.role === "user").at(-1) + expect(continueMsg).toBeDefined() + expect(continueMsg.tools).toEqual({ bash: true, edit: false }) + expect(continueMsg.system).toBe("custom system prompt") + expect(continueMsg.variant).toBe("high") + expect(continueMsg.format).toEqual({ type: "json" }) + // (b): the continue prompt is the three-option completion-aware nudge. + const continuePart = store.parts.find((p) => p.messageID === continueMsg.id && p.type === "text") + expect(continuePart?.synthetic).toBe(true) + expect(continuePart?.text).toContain("reply with DONE") + }) + + test("continue message leaves fields unset when the original user message never set them", async () => { + const sessionID = freshSessionID() + const { messages, markerID } = history(sessionID) + processBehaviors = [writeSummary("a real summary")] + + const result = await run({ sessionID, messages, markerID }) + + expect(result).toBe("continue") + const continueMsg = store.messages.filter((m) => m.role === "user").at(-1) + expect(continueMsg.tools).toBeUndefined() + expect(continueMsg.system).toBeUndefined() + expect(continueMsg.variant).toBeUndefined() + expect(continueMsg.format).toBeUndefined() + }) +}) + +describe("session.compaction summarizer integrity (/ item 3)", () => { + test("summarizer call passes explicit toolChoice 'none' and no tools", async () => { + const sessionID = freshSessionID() + const { messages, markerID } = history(sessionID) + processBehaviors = [writeSummary("a real summary")] + + await run({ sessionID, messages, markerID }) + + expect(processCalls.length).toBe(1) + expect(processCalls[0].toolChoice).toBe("none") + expect(processCalls[0].tools).toEqual({}) + }) + + // altimate_change start — upstream_fix regression: PIN_SUMMARY_ADDITION told + // the summarizer to skip the task ("it's pinned separately") based only on + // pinEnabled(cfg) — but pinBudget can independently return 0 on a small + // window, so no pin would actually be injected and the task got dropped + // from both places. + function summarizerPromptText() { + const lastMessage = processCalls[0].messages.at(-1) + return lastMessage.content[0].text as string + } + + test("PIN_SUMMARY_ADDITION is included when the session's pin budget is positive", async () => { + const sessionID = freshSessionID() + const { messages, markerID } = history(sessionID) + processBehaviors = [writeSummary("a real summary")] + + await run({ sessionID, messages, markerID }) + + expect(summarizerPromptText()).toContain(SessionCompaction.PIN_SUMMARY_ADDITION) + }) + + test("PIN_SUMMARY_ADDITION is omitted when the session's pin budget is zero (tiny window)", async () => { + const sessionID = freshSessionID() + const { messages, markerID } = history(sessionID) + processBehaviors = [writeSummary("a real summary")] + const tinyModel = { + ...fakeModel, + limit: { context: 15_000, output: 1_000 }, + } as unknown as Provider.Model + spyOn(Provider, "getModel").mockImplementationOnce(async () => tinyModel) + + await run({ sessionID, messages, markerID }) + + expect(summarizerPromptText()).not.toContain(SessionCompaction.PIN_SUMMARY_ADDITION) + }) + // altimate_change end + + test("does not retry when the first attempt produces summary text", async () => { + const sessionID = freshSessionID() + const { messages, markerID } = history(sessionID) + processBehaviors = [writeSummary("a real summary")] + + const result = await run({ sessionID, messages, markerID }) + + expect(result).toBe("continue") + expect(processCalls.length).toBe(1) + }) + + test("retries once with identical input when the summary step has no text", async () => { + const sessionID = freshSessionID() + const { messages, markerID } = history(sessionID) + processBehaviors = [noSummary, writeSummary("recovered summary")] + + const result = await run({ sessionID, messages, markerID }) + + expect(result).toBe("continue") + expect(processCalls.length).toBe(2) + // Retry uses the identical summarizer input. + expect(processCalls[1]).toBe(processCalls[0]) + // No error was committed. + const summaryMsg = store.messages.find((m) => m.role === "assistant" && m.summary) + expect(summaryMsg.error).toBeUndefined() + }) + + test("whitespace-only summary text counts as empty", async () => { + const sessionID = freshSessionID() + const { messages, markerID } = history(sessionID) + processBehaviors = [writeSummary(" \n\t "), writeSummary("recovered summary")] + + const result = await run({ sessionID, messages, markerID }) + + expect(result).toBe("continue") + expect(processCalls.length).toBe(2) + }) + + test("marks error and stops instead of committing when retry also produces no text", async () => { + const sessionID = freshSessionID() + const { messages, markerID } = history(sessionID) + processBehaviors = [noSummary, noSummary] + + const result = await run({ sessionID, messages, markerID }) + + expect(result).toBe("stop") + expect(processCalls.length).toBe(2) + const summaryMsg = store.messages.find((m) => m.role === "assistant" && m.summary) + expect(summaryMsg.finish).toBe("error") + expect(summaryMsg.error?.name).toBe("UnknownError") + expect(JSON.stringify(summaryMsg.error)).toContain("no summary text") + // The failed summary must NOT be committed as a compaction continue turn. + const continueTurn = store.parts.find((p) => p.type === "text" && p.synthetic) + expect(continueTurn).toBeUndefined() + }) +}) + +// ─── Harness reliability (b)+(d) (item 1): completion-aware continue nudge via the +// nudge arbiter, and the mechanism-accurate overflow notice ────────────────────── +describe("session.compaction continue-nudge termination path (/d)", () => { + test("continue message carries the three-option completion-aware nudge", async () => { + const sessionID = freshSessionID() + const { messages, markerID } = history(sessionID) + processBehaviors = [writeSummary("a real summary")] + + const result = await run({ sessionID, messages, markerID }) + + expect(result).toBe("continue") + const continuePart = store.parts.find((p) => p.type === "text" && p.synthetic) + expect(continuePart?.text).toContain(SessionTermination.COMPLETION_NUDGE) + expect(continuePart?.text).toContain("reply with DONE") + expect(continuePart?.text).toContain("ask for clarification") + }) + + test("one-directive-per-turn contract: exactly ONE directive block — pending lower-precedence directives are consumed", async () => { + const sessionID = freshSessionID() + const { messages, markerID } = history(sessionID) + processBehaviors = [writeSummary("a real summary")] + // A starvation-breaker directive is pending from an earlier step. + NudgeArbiter.register(sessionID, { + source: "starvation_breaker", + kind: "starvation", + text: "STARVATION-DIRECTIVE-TEXT", + }) + + const result = await run({ sessionID, messages, markerID }) + + expect(result).toBe("continue") + const continuePart = store.parts.find((p) => p.type === "text" && p.synthetic) + // The termination nudge (top precedence) wins; the starvation text is NOT + // stacked into the same injected turn… + expect(continuePart?.text).toContain(SessionTermination.COMPLETION_NUDGE) + expect(continuePart?.text).not.toContain("STARVATION-DIRECTIVE-TEXT") + // …and nothing is left pending to leak into the next generation. + expect(NudgeArbiter.pending(sessionID)).toHaveLength(0) + }) + + test("(d): overflow notice is mechanism-accurate — never blames media attachments", async () => { + const sessionID = freshSessionID() + const { messages, markerID } = history(sessionID) + processBehaviors = [writeSummary("a real summary")] + + const result = await SessionCompaction.process({ + sessionID, + messages, + parentID: markerID, + abort: new AbortController().signal, + auto: true, + overflow: true, + }) + + expect(result).toBe("continue") + const continuePart = store.parts.find((p) => p.type === "text" && p.synthetic) + expect(continuePart?.text).toContain(SessionTermination.OVERFLOW_NOTICE) + expect(continuePart?.text).not.toContain("large media attachments") + expect(continuePart?.text).toContain("context limit") + // The completion nudge still follows the notice. + expect(continuePart?.text).toContain("reply with DONE") + }) +}) diff --git a/packages/opencode/test/session/compaction.test.ts b/packages/opencode/test/session/compaction.test.ts index 145c525c79..2bc5be1921 100644 --- a/packages/opencode/test/session/compaction.test.ts +++ b/packages/opencode/test/session/compaction.test.ts @@ -1,4 +1,4 @@ -import { afterEach, describe, expect, mock, test } from "bun:test" +import { afterAll, afterEach, beforeAll, describe, expect, mock, test } from "bun:test" import { ConfigV1 } from "@opencode-ai/core/v1/config/config" import { SessionV1 } from "@opencode-ai/core/v1/session" import { Database } from "@opencode-ai/core/database/database" @@ -463,6 +463,16 @@ function autocontinue(enabled: boolean) { } describe("session.compaction.isOverflow", () => { + // These tests pin the RAW-limit boundary math, so disable the estimator + // safety margin (fraction 1 = raw limit). Default-margin behavior is covered + // in compaction-safety-fraction.test.ts. + beforeAll(() => { + process.env["ALTIMATE_CONTEXT_SAFETY_FRACTION"] = "1" + }) + afterAll(() => { + delete process.env["ALTIMATE_CONTEXT_SAFETY_FRACTION"] + }) + it.live( "returns true when token count exceeds usable context", provideTmpdirInstance(() => @@ -1057,7 +1067,7 @@ describe("session.compaction.process", () => { metadata: { compaction_continue: true }, }) if (last?.parts[0]?.type === "text") { - expect(last.parts[0].text).toContain("Continue if you have next steps") + expect(last.parts[0].text).toContain("reply with DONE") } }), ) @@ -1248,7 +1258,7 @@ describe("session.compaction.process", () => { (msg) => msg.info.role === "user" && msg.parts.some( - (part) => part.type === "text" && part.synthetic && part.text.includes("Continue if you have next steps"), + (part) => part.type === "text" && part.synthetic && part.text.includes("reply with DONE"), ), ), ).toBe(false) diff --git a/packages/opencode/test/session/llm.test.ts b/packages/opencode/test/session/llm.test.ts index 148529ad64..6f01766a71 100644 --- a/packages/opencode/test/session/llm.test.ts +++ b/packages/opencode/test/session/llm.test.ts @@ -1,6 +1,6 @@ import { afterAll, beforeAll, beforeEach, describe, expect, test } from "bun:test" import path from "path" -import type { ModelMessage } from "ai" +import type { ModelMessage, Tool } from "ai" import { LLM } from "../../src/session/llm" import { Global } from "../../src/global" import { Instance } from "../../src/project/instance" @@ -83,6 +83,45 @@ describe("session.llm.toolNamesFromMessages", () => { }) }) +// Harness reliability / item 3: stub injection must be skipped entirely when the call +// exposes zero real tools AND uses the explicit toolChoice "none" no-tool-call +// contract (e.g. the compaction summarizer) — the provider-compat fallback path. +// A normal turn that happens to have zero real tools (allowlist/permissions +// stripped everything) must still get historical stubs so referenced tool_use +// blocks in history don't trip provider validation. +describe("session.llm.addHistoricalToolStubs", () => { + test("skips stub injection when there are zero real tools AND toolChoice is none", () => { + const tools: Record = {} + const result = LLM.addHistoricalToolStubs(tools, new Set(["bash", "read"]), "none") + expect(result).toBe(tools) + expect(Object.keys(tools)).toEqual([]) + }) + + test("still injects stubs for zero real tools when toolChoice is not none", () => { + const tools: Record = {} + LLM.addHistoricalToolStubs(tools, new Set(["bash", "read"])) + expect(Object.keys(tools).sort()).toEqual(["bash", "read"]) + }) + + test("injects stubs for referenced tools missing from a non-empty tool set", () => { + const real = { description: "real bash" } as Tool + const tools: Record = { bash: real } + LLM.addHistoricalToolStubs(tools, new Set(["bash", "old_mcp_tool"])) + expect(Object.keys(tools).sort()).toEqual(["bash", "old_mcp_tool"]) + // Existing real tools are never overwritten. + expect(tools.bash).toBe(real) + expect(tools.old_mcp_tool.description).toContain("[Historical]") + }) + + test("is a no-op when every referenced tool already has a definition", () => { + const real = { description: "real bash" } as Tool + const tools: Record = { bash: real } + LLM.addHistoricalToolStubs(tools, new Set(["bash"])) + expect(Object.keys(tools)).toEqual(["bash"]) + expect(tools.bash).toBe(real) + }) +}) + type Capture = { url: URL headers: Headers diff --git a/packages/opencode/test/session/nudge-arbiter.test.ts b/packages/opencode/test/session/nudge-arbiter.test.ts new file mode 100644 index 0000000000..e85386b7f9 --- /dev/null +++ b/packages/opencode/test/session/nudge-arbiter.test.ts @@ -0,0 +1,104 @@ +// One-directive-per-turn contract unit gates: the nudge arbiter guarantees at +// most ONE system-authored directive block per injected turn, with precedence +// termination_challenge (item 1) > starvation_breaker (item 4) > budget_reminder +// (item 9). Items register candidates; the injection site takes the single winner. +import { beforeEach, describe, expect, test } from "bun:test" +import { NudgeArbiter } from "../../src/session/nudge" + +const SID = "ses_arbiter_test" + +beforeEach(() => { + NudgeArbiter.clear(SID) +}) + +describe("NudgeArbiter precedence (one-directive-per-turn contract)", () => { + test("termination challenge beats starvation breaker and budget reminder", () => { + NudgeArbiter.register(SID, { source: "budget_reminder", kind: "budget", text: "budget text" }) + NudgeArbiter.register(SID, { source: "starvation_breaker", kind: "starvation", text: "starvation text" }) + NudgeArbiter.register(SID, { source: "termination_challenge", kind: "completion_nudge", text: "termination text" }) + const winner = NudgeArbiter.take(SID) + expect(winner?.source).toBe("termination_challenge") + expect(winner?.text).toBe("termination text") + }) + + test("starvation breaker beats budget reminder", () => { + NudgeArbiter.register(SID, { source: "budget_reminder", kind: "budget", text: "budget text" }) + NudgeArbiter.register(SID, { source: "starvation_breaker", kind: "starvation", text: "starvation text" }) + expect(NudgeArbiter.take(SID)?.source).toBe("starvation_breaker") + }) + + test("registration order does not matter, only precedence", () => { + NudgeArbiter.register(SID, { source: "termination_challenge", kind: "confirm_done", text: "t" }) + NudgeArbiter.register(SID, { source: "budget_reminder", kind: "budget", text: "b" }) + expect(NudgeArbiter.take(SID)?.source).toBe("termination_challenge") + }) +}) + +describe("NudgeArbiter one-directive-per-turn contract", () => { + test("take() returns exactly one directive and clears ALL pending — losers are dropped", () => { + NudgeArbiter.register(SID, { source: "starvation_breaker", kind: "starvation", text: "s" }) + NudgeArbiter.register(SID, { source: "budget_reminder", kind: "budget", text: "b" }) + const winner = NudgeArbiter.take(SID) + expect(winner).toBeDefined() + // Nothing left for the same injected turn — a second take yields nothing. + expect(NudgeArbiter.take(SID)).toBeUndefined() + expect(NudgeArbiter.pending(SID)).toHaveLength(0) + }) + + test("take() on an empty registry returns undefined", () => { + expect(NudgeArbiter.take(SID)).toBeUndefined() + }) + + test("same source+kind re-registration replaces, not stacks", () => { + NudgeArbiter.register(SID, { source: "budget_reminder", kind: "budget", text: "old" }) + NudgeArbiter.register(SID, { source: "budget_reminder", kind: "budget", text: "new" }) + expect(NudgeArbiter.pending(SID)).toHaveLength(1) + expect(NudgeArbiter.take(SID)?.text).toBe("new") + }) + + test("sessions are isolated", () => { + const other = "ses_arbiter_other" + NudgeArbiter.clear(other) + NudgeArbiter.register(SID, { source: "budget_reminder", kind: "budget", text: "mine" }) + expect(NudgeArbiter.take(other)).toBeUndefined() + expect(NudgeArbiter.take(SID)?.text).toBe("mine") + }) +}) + +describe("NudgeArbiter injection-site contract (item 1 usage)", () => { + test("register-then-take at an injection point consumes pending lower-precedence directives", () => { + // A starvation directive is pending from a previous step; the compaction + // continue-message injection point registers its termination nudge and takes + // the winner — the injected turn carries ONE directive and the starvation + // candidate is consumed, not deferred into the same turn. + NudgeArbiter.register(SID, { source: "starvation_breaker", kind: "starvation", text: "produce the edit or DONE" }) + NudgeArbiter.register(SID, { source: "termination_challenge", kind: "completion_nudge", text: "three options" }) + const winner = NudgeArbiter.take(SID) + expect(winner?.kind).toBe("completion_nudge") + expect(NudgeArbiter.pending(SID)).toHaveLength(0) + }) +}) + +describe("NudgeArbiter LRU eviction", () => { + test("eviction removes the least-recently-USED session, not the oldest-created", () => { + const prefix = "ses_lru_nudge_" + const directive = { source: "budget_reminder" as const, kind: "budget", text: "d" } + // Cleanup runs even when an assertion throws — otherwise a failure here + // leaves 129 sessions in the module-global table and the 128-session bound + // starts evicting other suites' pending directives. + try { + // Fill the table (the 128-session bound) with fresh sessions. + for (let i = 0; i < 128; i++) NudgeArbiter.register(`${prefix}${i}`, directive) + // Refresh the OLDEST-created session by using it again. + NudgeArbiter.register(`${prefix}0`, { ...directive, text: "refreshed" }) + // A new session must evict the least-recently-used (#1), not #0. + NudgeArbiter.register(`${prefix}new`, directive) + expect(NudgeArbiter.pending(`${prefix}0`).length).toBeGreaterThan(0) + expect(NudgeArbiter.pending(`${prefix}1`)).toHaveLength(0) + expect(NudgeArbiter.pending(`${prefix}new`).length).toBeGreaterThan(0) + } finally { + for (let i = 0; i < 128; i++) NudgeArbiter.clear(`${prefix}${i}`) + NudgeArbiter.clear(`${prefix}new`) + } + }) +}) diff --git a/packages/opencode/test/session/processor.test.ts b/packages/opencode/test/session/processor.test.ts index d6d460b992..e131329b3a 100644 --- a/packages/opencode/test/session/processor.test.ts +++ b/packages/opencode/test/session/processor.test.ts @@ -885,3 +885,53 @@ describe("processor state tracking", () => { expect(attempt).toBe(2) }) }) + +// --------------------------------------------------------------------------- +// Finish-outcome ordering (mirrors the decision block at the end of +// processor.ts finish handling). Terminal outcomes (blocked / error / +// doom-loop stop) must win over "compact"; explicit DONE with a pending +// compaction still stops. If the ordering in processor.ts changes, update +// this mirror to match. +// --------------------------------------------------------------------------- +describe("finish outcome ordering", () => { + function resolveOutcome(state: { + needsCompaction: boolean + explicitDone: boolean + blocked: boolean + error: boolean + starvationStop: boolean + }): "stop" | "compact" | "continue" { + if (state.needsCompaction && state.explicitDone) return "stop" + if (state.blocked) return "stop" + if (state.error) return "stop" + if (state.starvationStop) return "stop" + if (state.needsCompaction) return "compact" + return "continue" + } + + const base = { needsCompaction: false, explicitDone: false, blocked: false, error: false, starvationStop: false } + + test("explicit DONE overrides a pending compaction", () => { + expect(resolveOutcome({ ...base, needsCompaction: true, explicitDone: true })).toBe("stop") + }) + + test("blocked wins over compact", () => { + expect(resolveOutcome({ ...base, needsCompaction: true, blocked: true })).toBe("stop") + }) + + test("error wins over compact", () => { + expect(resolveOutcome({ ...base, needsCompaction: true, error: true })).toBe("stop") + }) + + test("doom-loop stop wins over compact", () => { + expect(resolveOutcome({ ...base, needsCompaction: true, starvationStop: true })).toBe("stop") + }) + + test("plain overflow still compacts", () => { + expect(resolveOutcome({ ...base, needsCompaction: true })).toBe("compact") + }) + + test("nothing pending continues", () => { + expect(resolveOutcome(base)).toBe("continue") + }) +}) diff --git a/packages/opencode/test/session/starvation.test.ts b/packages/opencode/test/session/starvation.test.ts new file mode 100644 index 0000000000..e4f75c83e9 --- /dev/null +++ b/packages/opencode/test/session/starvation.test.ts @@ -0,0 +1,523 @@ +// — write-starvation circuit breaker + loop detection (corrected mechanism). +// Gates covered here (unit level): +// - ships ANNOTATE-ONLY by default (resolveConfig default mode is "annotate") +// - read-only-deliverable task NON-FIRING probe (the misfire class the bench +// cannot see: analysis-only sessions must not be pushed into fabricated edits +// below threshold, and above threshold the directive must carry the DONE +// alternative — never an unconditional "produce the edit now") +// - generic mutating-call classifier with NO vertical tokens (source-scan guard) +// - content-hash unchanged-read annotation: annotate never suppress; generated +// paths exempt +// - repeat_signature = hash(tool + normalized args + touched files + failure +// message) loop detection +// - doom-loop guard re-keyed on (toolName + normalized args) with the +// nudge → status-check → stop escalation ladder; varied-args repetition +// (the old per-NAME false-positive class) never climbs the ladder +// - polling patterns get a raised threshold, not an exemption +// - nudge arbiter: at most ONE directive per turn, fixed precedence +import { describe, expect, test } from "bun:test" +import { readFileSync } from "node:fs" +import path from "node:path" +import { SessionStarvation } from "../../src/session/starvation" +import { NudgeArbiter } from "../../src/session/nudge" + +const cfg = SessionStarvation.resolveConfig(undefined) + +function tracker(overrides: Partial = {}) { + return SessionStarvation.createTracker({ ...cfg, ...overrides }) +} + +// altimate_change start — upstream_fix regression tests +describe("resolveConfig clamps non-positive thresholds to their default", () => { + test("doom_loop_threshold: 0 does not immediately trip the breaker on the first call", () => { + const resolved = SessionStarvation.resolveConfig({ doom_loop_threshold: 0 }) + expect(resolved.doomLoopThreshold).toBe(SessionStarvation.resolveConfig(undefined).doomLoopThreshold) + }) + + test("negative and non-finite values also fall back to the default", () => { + expect(SessionStarvation.resolveConfig({ polling_threshold_multiplier: -3 }).pollingThresholdMultiplier).toBe( + SessionStarvation.resolveConfig(undefined).pollingThresholdMultiplier, + ) + expect(SessionStarvation.resolveConfig({ max_turns_without_mutation: Number.NaN }).maxTurnsWithoutMutation).toBe( + SessionStarvation.resolveConfig(undefined).maxTurnsWithoutMutation, + ) + }) + + test("a valid positive override is still honored", () => { + expect(SessionStarvation.resolveConfig({ doom_loop_threshold: 7 }).doomLoopThreshold).toBe(7) + }) + + test("mode: 'off' remains the only way to disable starvation", () => { + expect(SessionStarvation.resolveConfig({ mode: "off" }).mode).toBe("off") + }) +}) + +describe("normalizeArgs — shared (non-circular) references are not mislabeled circular", () => { + test("a DAG-shaped object (same reference reused, not nested in itself) normalizes both occurrences", () => { + const shared = { a: 1 } + const result = SessionStarvation.normalizeArgs({ x: shared, y: shared }) + expect(result).toBe(JSON.stringify({ x: { a: 1 }, y: { a: 1 } })) + }) + + test("a genuinely circular reference is still caught", () => { + const circular: Record = { a: 1 } + circular.self = circular + expect(SessionStarvation.normalizeArgs(circular)).toBe(JSON.stringify({ a: 1, self: "[circular]" })) + }) +}) +// altimate_change end + +describe("config defaults (annotate-only ships by default)", () => { + test("default mode is annotate — directives and hard consequences are OFF until bench-validated", () => { + expect(cfg.mode).toBe("annotate") + }) + + test("plan/review agents are exempt by default", () => { + expect(cfg.exemptAgents).toContain("plan") + expect(cfg.exemptAgents).toContain("review") + }) + + test("thresholds are config-exposed and overridable", () => { + const resolved = SessionStarvation.resolveConfig({ + mode: "armed", + max_turns_without_mutation: 5, + repeat_signature_threshold: 2, + doom_loop_threshold: 4, + polling_threshold_multiplier: 10, + exempt_agents: ["explore"], + generated_path_patterns: ["gen/"], + }) + expect(resolved.mode).toBe("armed") + expect(resolved.maxTurnsWithoutMutation).toBe(5) + expect(resolved.repeatSignatureThreshold).toBe(2) + expect(resolved.doomLoopThreshold).toBe(4) + expect(resolved.pollingThresholdMultiplier).toBe(10) + expect(resolved.exemptAgents).toEqual(["explore"]) + expect(resolved.generatedPathPatterns).toEqual(["gen/"]) + }) +}) + +describe("applyReadAnnotation — output mutation is run-mode-only", () => { + test("run mode appends the annotation to the tool output", () => { + const out = SessionStarvation.applyReadAnnotation("file contents", "[harness note: unchanged]", true) + expect(out).toBe("file contents\n\n[harness note: unchanged]") + }) + + test("interactive session: output stays byte-identical (telemetry-only shadow)", () => { + const out = SessionStarvation.applyReadAnnotation("file contents", "[harness note: unchanged]", false) + expect(out).toBe("file contents") + }) +}) + +describe("no vertical tokens in generic classifiers (leak-lens hard requirement)", () => { + test("starvation.ts contains no dbt/warehouse vertical tokens", () => { + const source = readFileSync(path.join(import.meta.dir, "../../src/session/starvation.ts"), "utf8") + // Hard requirement: no dbt/altimate-dbt string matching inside any generic + // classifier, and no bench task command strings in product code. + expect(/\bdbt\b/i.test(source)).toBe(false) + expect(/snowflake|bigquery|redshift|databricks/i.test(source)).toBe(false) + expect(source.includes("--profiles-dir")).toBe(false) + }) + + test("classifier is generic: unknown tools are 'unknown', never assumed mutating or read-only", () => { + expect(SessionStarvation.classifyToolCall("some_mcp_tool")).toBe("unknown") + expect(SessionStarvation.classifyToolCall("bash")).toBe("unknown") + expect(SessionStarvation.classifyToolCall("write")).toBe("mutating") + expect(SessionStarvation.classifyToolCall("edit")).toBe("mutating") + expect(SessionStarvation.classifyToolCall("apply_patch")).toBe("mutating") + expect(SessionStarvation.classifyToolCall("read")).toBe("read-only") + expect(SessionStarvation.classifyToolCall("grep")).toBe("read-only") + }) +}) + +describe("read-only-deliverable task: non-firing probe", () => { + test("a varied read-only session below threshold never fires anything", () => { + const t = tracker() + for (let step = 0; step < cfg.maxTurnsWithoutMutation - 1; step++) { + const call = t.onToolCall({ tool: "read", input: { filePath: `/repo/file${step}.sql` } }) + expect(call.doomLoop).toBeUndefined() + const result = t.onToolResult({ + tool: "read", + input: { filePath: `/repo/file${step}.sql` }, + output: `content ${step}`, + }) + expect(result.readAnnotation).toBeUndefined() + expect(result.repeatLoop).toBeUndefined() + const stepOutcome = t.onStepFinish({ mutatedFiles: [] }) + expect(stepOutcome.starvation).toBeUndefined() + } + }) + + test("at threshold the directive is outcome-neutral with a DONE alternative — never an unconditional edit demand", () => { + const t = tracker() + let fired: string | undefined + for (let step = 0; step < cfg.maxTurnsWithoutMutation; step++) { + const out = t.onStepFinish({ mutatedFiles: [] }) + if (out.starvation) fired = out.starvation.directive + } + expect(fired).toBeDefined() + expect(fired!).toContain("If this task requires an edit") + expect(fired!).toContain("analysis with no file changes") + expect(fired!).toContain("DONE") + // must not be the unconditional form + expect(fired!.startsWith("Produce the edit")).toBe(false) + }) + + test("re-fires every threshold turns, not every turn (no directive spam)", () => { + const t = tracker({ maxTurnsWithoutMutation: 3 }) + const firedAt: number[] = [] + for (let step = 1; step <= 9; step++) { + const out = t.onStepFinish({ mutatedFiles: [] }) + if (out.starvation) firedAt.push(step) + } + expect(firedAt).toEqual([3, 6, 9]) + }) +}) + +describe("mutation evidence resets the starvation counter (command-agnostic)", () => { + test("write/edit tool completion counts as mutation", () => { + const t = tracker({ maxTurnsWithoutMutation: 3 }) + t.onStepFinish({ mutatedFiles: [] }) + t.onStepFinish({ mutatedFiles: [] }) + t.onToolResult({ tool: "write", input: { filePath: "/repo/model.sql", content: "x" } }) + const out = t.onStepFinish({ mutatedFiles: [] }) + expect(out.turnsWithoutMutation).toBe(0) + expect(out.starvation).toBeUndefined() + }) + + test("bash-mediated mutation (snapshot patch files) counts — no command parsing needed", () => { + const t = tracker({ maxTurnsWithoutMutation: 3 }) + t.onStepFinish({ mutatedFiles: [] }) + t.onStepFinish({ mutatedFiles: [] }) + // e.g. `sed -i` via bash: no edit event, but the step snapshot diff sees it + const out = t.onStepFinish({ mutatedFiles: ["models/some_file.sql"] }) + expect(out.turnsWithoutMutation).toBe(0) + expect(out.starvation).toBeUndefined() + }) +}) + +describe("mutation credit requires success, never the call alone", () => { + test("a FAILED mutating tool call does not reset the starvation counter", () => { + const t = tracker({ maxTurnsWithoutMutation: 3 }) + t.onStepFinish({ mutatedFiles: [] }) + t.onStepFinish({ mutatedFiles: [] }) + // Call is issued but the result fails: no snapshot diff, no success. + t.onToolCall({ tool: "edit", input: { filePath: "/repo/model.sql" } }) + t.onToolResult({ tool: "edit", input: { filePath: "/repo/model.sql" }, failureMessage: "oldString not found" }) + const out = t.onStepFinish({ mutatedFiles: [] }) + expect(out.turnsWithoutMutation).toBe(3) + expect(out.starvation).toBeDefined() + }) + + test("a successful mutating tool result still resets the counter", () => { + const t = tracker({ maxTurnsWithoutMutation: 3 }) + t.onStepFinish({ mutatedFiles: [] }) + t.onStepFinish({ mutatedFiles: [] }) + t.onToolCall({ tool: "edit", input: { filePath: "/repo/model.sql" } }) + t.onToolResult({ tool: "edit", input: { filePath: "/repo/model.sql" } }) + const out = t.onStepFinish({ mutatedFiles: [] }) + expect(out.turnsWithoutMutation).toBe(0) + expect(out.starvation).toBeUndefined() + }) +}) + +describe("doom-loop stop latch", () => { + test("stop fires exactly once per completed ladder run, then the ladder resets", () => { + const t = tracker({ doomLoopThreshold: 3 }) + const input = { command: "make check" } + const stops: number[] = [] + for (let i = 1; i <= 20; i++) { + const call = t.onToolCall({ tool: "bash", input }) + if (call.doomLoop?.escalation === "stop") stops.push(i) + } + // First full run stops at 9; the ladder then restarts from zero, so the + // next stop needs another full run (9 more calls, with nudge/status-check + // rungs in between) — never a stop on every subsequent call. + expect(stops).toEqual([9, 18]) + }) + + test("after a latched stop, a retried session climbs the full ladder again", () => { + const t = tracker({ doomLoopThreshold: 3 }) + const input = { command: "make check" } + for (let i = 1; i <= 9; i++) t.onToolCall({ tool: "bash", input }) + // Retry: first repeated call after the stop is NOT an instant stop. + const call = t.onToolCall({ tool: "bash", input }) + expect(call.doomLoop).toBeUndefined() + // The nudge rung comes back at the threshold, as in a fresh session. + t.onToolCall({ tool: "bash", input }) + const third = t.onToolCall({ tool: "bash", input }) + expect(third.doomLoop?.escalation).toBe("nudge") + }) +}) + +describe("unchanged-read annotation (content hash; annotate never suppress)", () => { + test("re-reading identical content yields an informational annotation", () => { + const t = tracker() + const input = { filePath: "/repo/models/a.sql" } + expect(t.onToolResult({ tool: "read", input, output: "select 1" }).readAnnotation).toBeUndefined() + const second = t.onToolResult({ tool: "read", input, output: "select 1" }) + expect(second.readAnnotation).toBeDefined() + expect(second.readAnnotation!).toContain("/repo/models/a.sql") + expect(second.readAnnotation!).toContain("unchanged") + // annotation is informational — it must not instruct suppression or forbid re-reading + expect(/do not (re-)?read/i.test(second.readAnnotation!)).toBe(false) + }) + + test("changed content does not annotate", () => { + const t = tracker() + const input = { filePath: "/repo/models/a.sql" } + t.onToolResult({ tool: "read", input, output: "select 1" }) + const second = t.onToolResult({ tool: "read", input, output: "select 2" }) + expect(second.readAnnotation).toBeUndefined() + }) + + test("generated paths are exempt (they regenerate across builds)", () => { + const t = tracker() + for (const filePath of ["/repo/target/compiled/model.sql", "/repo/dev.duckdb", "/repo/logs/run.log"]) { + const input = { filePath } + t.onToolResult({ tool: "read", input, output: "same" }) + const second = t.onToolResult({ tool: "read", input, output: "same" }) + expect(second.readAnnotation).toBeUndefined() + } + }) + + test("failed reads never annotate", () => { + const t = tracker() + const input = { filePath: "/repo/models/a.sql" } + t.onToolResult({ tool: "read", input, output: "select 1" }) + const failed = t.onToolResult({ tool: "read", input, failureMessage: "EACCES" }) + expect(failed.readAnnotation).toBeUndefined() + }) +}) + +describe("repeat_signature loop detection", () => { + test("three identical (tool+args+failure) signatures fire the diagnostic directive", () => { + const t = tracker() + const attempt = { tool: "edit", input: { filePath: "/repo/a.sql", oldString: "x", newString: "y" } } + const fail = "oldString not found in file" + expect(t.onToolResult({ ...attempt, failureMessage: fail }).repeatLoop).toBeUndefined() + expect(t.onToolResult({ ...attempt, failureMessage: fail }).repeatLoop).toBeUndefined() + const third = t.onToolResult({ ...attempt, failureMessage: fail }) + expect(third.repeatLoop).toBeDefined() + expect(third.repeatLoop!.count).toBe(3) + expect(third.repeatLoop!.directive).toContain("DONE") + expect(third.repeatLoop!.directive).toContain("different action") + }) + + test("a different failure message breaks the signature chain (progress is being made)", () => { + const t = tracker() + const attempt = { tool: "edit", input: { filePath: "/repo/a.sql", oldString: "x", newString: "y" } } + t.onToolResult({ ...attempt, failureMessage: "error A" }) + t.onToolResult({ ...attempt, failureMessage: "error B" }) + const third = t.onToolResult({ ...attempt, failureMessage: "error C" }) + expect(third.repeatLoop).toBeUndefined() + }) + + test("signature includes touched files and normalized args (order-insensitive, whitespace-insensitive)", () => { + const a = SessionStarvation.repeatSignature({ + tool: "edit", + args: { filePath: "/a.sql", oldString: "select 1" }, + touchedFiles: ["/a.sql"], + failureMessage: "not found", + }) + const b = SessionStarvation.repeatSignature({ + tool: "edit", + args: { oldString: "select 1", filePath: "/a.sql" }, + touchedFiles: ["/a.sql"], + failureMessage: "not found", + }) + expect(a).toBe(b) + const c = SessionStarvation.repeatSignature({ + tool: "edit", + args: { oldString: "select 1", filePath: "/a.sql" }, + touchedFiles: ["/b.sql"], + failureMessage: "not found", + }) + expect(c).not.toBe(a) + }) +}) + +describe("doom-loop escalation ladder — re-keyed on (toolName + normalized args)", () => { + test("varied-args repetition of the SAME tool never climbs the ladder (the old per-NAME false positive)", () => { + const t = tracker() + for (let i = 0; i < 50; i++) { + const call = t.onToolCall({ tool: "bash", input: { command: `echo ${i}` } }) + expect(call.doomLoop).toBeUndefined() + } + }) + + test("identical (tool+args) calls escalate nudge → status_check → stop, never straight to stop", () => { + const t = tracker({ doomLoopThreshold: 3 }) + const input = { command: "make check" } + const rungs: Array<[number, string]> = [] + for (let i = 1; i <= 9; i++) { + const call = t.onToolCall({ tool: "bash", input }) + if (call.doomLoop) rungs.push([i, call.doomLoop.escalation]) + } + expect(rungs).toEqual([ + [3, "nudge"], + [6, "status_check"], + [9, "stop"], + ]) + }) + + test("a different call resets the consecutive counter", () => { + const t = tracker({ doomLoopThreshold: 3 }) + t.onToolCall({ tool: "bash", input: { command: "make check" } }) + t.onToolCall({ tool: "bash", input: { command: "make check" } }) + t.onToolCall({ tool: "bash", input: { command: "ls" } }) + const call = t.onToolCall({ tool: "bash", input: { command: "make check" } }) + expect(call.doomLoop).toBeUndefined() + }) + + test("normalized-args keying: key order and whitespace do not defeat the counter", () => { + const t = tracker({ doomLoopThreshold: 3 }) + t.onToolCall({ tool: "grep", input: { pattern: "foo", path: "/repo" } }) + t.onToolCall({ tool: "grep", input: { path: "/repo", pattern: "foo" } }) + const call = t.onToolCall({ tool: "grep", input: { pattern: " foo ", path: "/repo" } }) + expect(call.doomLoop).toBeDefined() + expect(call.doomLoop!.escalation).toBe("nudge") + }) + + test("polling patterns raise the threshold (multiplier), not an exemption", () => { + const t = tracker({ doomLoopThreshold: 3, pollingThresholdMultiplier: 5 }) + const input = { command: "sleep 5 && curl -s localhost:8080/health" } + let firstRung: number | undefined + for (let i = 1; i <= 20; i++) { + const call = t.onToolCall({ tool: "bash", input }) + if (call.doomLoop && firstRung === undefined) firstRung = i + } + expect(firstRung).toBe(15) // 3 * 5, not 3 — and a ceiling still exists + }) + + test("directive text at every rung carries the DONE alternative", () => { + const t = tracker({ doomLoopThreshold: 3 }) + const input = { command: "make check" } + for (let i = 1; i <= 9; i++) { + const call = t.onToolCall({ tool: "bash", input }) + if (call.doomLoop) expect(call.doomLoop.directive).toContain("DONE") + } + }) +}) + +describe("armed gating logic (run-mode-only, exempt agents)", () => { + // Mirrors the gate expression in processor.ts: + // sbExempt = exemptAgents.includes(agent) || assistantMessage.summary + // sbArmed = mode === "armed" && runMode && !sbExempt + // starvation tracker is created only when mode !== "off" && !sbExempt + function exempt(resolved: SessionStarvation.ResolvedConfig, agent: string, summary: boolean) { + return resolved.exemptAgents.includes(agent) || summary + } + function armed(mode: SessionStarvation.Mode, runMode: boolean, agent: string, summary = false) { + const resolved = SessionStarvation.resolveConfig({ mode }) + return mode === "armed" && runMode && !exempt(resolved, agent, summary) + } + /** Whether the per-session tracker is wired at all (and so can accumulate steps). */ + function tracks(mode: SessionStarvation.Mode, agent: string, summary = false) { + const resolved = SessionStarvation.resolveConfig({ mode }) + return mode !== "off" && !exempt(resolved, agent, summary) + } + + test("annotate mode (the default) never arms — even in run mode", () => { + expect(armed("annotate", true, "build")).toBe(false) + }) + test("armed mode outside run mode (TUI/serve) never arms", () => { + expect(armed("armed", false, "build")).toBe(false) + }) + test("armed + run mode arms for build agents", () => { + expect(armed("armed", true, "build")).toBe(true) + }) + test("armed + run mode stays off for plan/review-class agents", () => { + expect(armed("armed", true, "plan")).toBe(false) + expect(armed("armed", true, "review")).toBe(false) + }) + + // The compaction summarizer runs through the same processor under the session's + // OWN id, so without an exemption its single mutation-free step would advance + // the working agent's shared tracker. + test("the compaction summarizer is exempt: no tracker is wired for a summary message", () => { + expect(tracks("annotate", "build", true)).toBe(false) + expect(tracks("armed", "build", true)).toBe(false) + // a normal working step on the same session still tracks + expect(tracks("annotate", "build", false)).toBe(true) + }) + + test("the compaction summarizer never arms, even in armed run mode", () => { + expect(armed("armed", true, "build", true)).toBe(false) + expect(armed("armed", true, "build", false)).toBe(true) + }) + + test("mode 'off' wires no tracker at all", () => { + expect(tracks("off", "build")).toBe(false) + }) +}) + +describe("nudge arbiter (one-directive-per-turn contract)", () => { + test("at most one directive per turn — highest precedence wins, rest dropped", () => { + const sid = "ses_arbiter_1" + NudgeArbiter.clear(sid) + NudgeArbiter.register(sid, { source: "budget_reminder", kind: "budget_60", text: "turn N of M" }) + NudgeArbiter.register(sid, { source: "starvation_breaker", kind: "starvation", text: "starvation directive" }) + NudgeArbiter.register(sid, { source: "termination_challenge", kind: "challenge", text: "confirm DONE" }) + const winner = NudgeArbiter.take(sid) + expect(winner?.source).toBe("termination_challenge") + // everything cleared — the turn gets exactly one directive + expect(NudgeArbiter.take(sid)).toBeUndefined() + }) + + test("precedence: starvation breaker beats budget reminder", () => { + const sid = "ses_arbiter_2" + NudgeArbiter.clear(sid) + NudgeArbiter.register(sid, { source: "budget_reminder", kind: "budget_85", text: "b" }) + NudgeArbiter.register(sid, { source: "starvation_breaker", kind: "repeat_signature", text: "s" }) + expect(NudgeArbiter.take(sid)?.source).toBe("starvation_breaker") + }) + + test("same source+kind replaces rather than stacks", () => { + const sid = "ses_arbiter_3" + NudgeArbiter.clear(sid) + NudgeArbiter.register(sid, { source: "starvation_breaker", kind: "starvation", text: "v1" }) + NudgeArbiter.register(sid, { source: "starvation_breaker", kind: "starvation", text: "v2" }) + expect(NudgeArbiter.pending(sid)).toHaveLength(1) + expect(NudgeArbiter.take(sid)?.text).toBe("v2") + }) + + test("sessions are isolated", () => { + NudgeArbiter.clear("ses_a") + NudgeArbiter.clear("ses_b") + NudgeArbiter.register("ses_a", { source: "starvation_breaker", kind: "starvation", text: "a" }) + expect(NudgeArbiter.take("ses_b")).toBeUndefined() + expect(NudgeArbiter.take("ses_a")?.text).toBe("a") + }) +}) + +describe("session-scoped tracker store", () => { + test("trackers persist across processor instances (per-step create) for the same session", () => { + const resolved = SessionStarvation.resolveConfig({ max_turns_without_mutation: 3 }) + SessionStarvation.clear("ses_store_1") + const first = SessionStarvation.forSession("ses_store_1", resolved) + first.onStepFinish({ mutatedFiles: [] }) + first.onStepFinish({ mutatedFiles: [] }) + // a new processor for the next step must see the accumulated state + const second = SessionStarvation.forSession("ses_store_1", resolved) + const out = second.onStepFinish({ mutatedFiles: [] }) + expect(out.starvation).toBeDefined() + SessionStarvation.clear("ses_store_1") + }) +}) + +describe("forSession LRU eviction", () => { + test("an active session's tracker survives churn from newer sessions", () => { + const prefix = "ses_lru_starve_" + const first = SessionStarvation.forSession(`${prefix}0`, cfg) + for (let i = 1; i < 128; i++) SessionStarvation.forSession(`${prefix}${i}`, cfg) + // Access #0 again: recency refreshed, same tracker returned. + expect(SessionStarvation.forSession(`${prefix}0`, cfg)).toBe(first) + // A new session evicts the least-recently-used (#1), never the active #0. + SessionStarvation.forSession(`${prefix}extra`, cfg) + expect(SessionStarvation.forSession(`${prefix}0`, cfg)).toBe(first) + // Cleanup so this suite leaves no global state behind. + for (let i = 0; i < 128; i++) SessionStarvation.clear(`${prefix}${i}`) + SessionStarvation.clear(`${prefix}extra`) + }) +}) diff --git a/packages/opencode/test/session/task-pin.test.ts b/packages/opencode/test/session/task-pin.test.ts new file mode 100644 index 0000000000..35d5974e7b --- /dev/null +++ b/packages/opencode/test/session/task-pin.test.ts @@ -0,0 +1,403 @@ +// harness plan / item 2 — pin the original task verbatim through compaction. +// Pure-function unit tests: pin-source selection (mode-aware, incl. the +// mid-session-redirect case), verbatim/head+tail+contract-card assembly, +// dynamic budget math with the livelock invariant, and the livelock guard +// that halves the pin after two consecutive failed compactions. +import { beforeEach, describe, expect, test } from "bun:test" +import { SessionPrompt } from "../../src/session/prompt" +import { SessionCompaction } from "../../src/session/compaction" +import { Token } from "@/util/token" +import type { MessageV2 } from "../../src/session/message-v2" +import type { Provider } from "@/provider/provider" + +let seq = 0 +function nextID() { + seq += 1 + return `msg_${String(seq).padStart(6, "0")}` +} + +function userMsg( + text: string, + opts: { synthetic?: boolean; compaction?: boolean } = {}, +): MessageV2.WithParts { + const id = nextID() + const parts: any[] = [] + if (opts.compaction) parts.push({ id: nextID(), messageID: id, sessionID: "ses_test", type: "compaction" }) + if (text) + parts.push({ + id: nextID(), + messageID: id, + sessionID: "ses_test", + type: "text", + text, + ...(opts.synthetic ? { synthetic: true } : {}), + }) + return { info: { id, role: "user", sessionID: "ses_test" }, parts } as unknown as MessageV2.WithParts +} + +function assistantMsg(opts: { summary?: boolean; finish?: string; error?: unknown } = {}): MessageV2.WithParts { + const id = nextID() + return { + info: { + id, + role: "assistant", + sessionID: "ses_test", + summary: opts.summary, + finish: opts.finish ?? "stop", + error: opts.error, + }, + parts: [], + } as unknown as MessageV2.WithParts +} + +function model(input: { context: number; input?: number; output?: number }): Provider.Model { + return { + limit: { context: input.context, input: input.input, output: input.output ?? 4_096 }, + } as unknown as Provider.Model +} + +function cfg(compaction?: Record) { + return { compaction } as any +} + +const TASK_RUN = "Build the orders model in models/marts/orders.sql and verify row counts." +const TASK_REDIRECT = + "Stop working on orders. Instead rename the output file to final_report.csv and do not touch models/marts/orders.sql again." + +function historyWithRedirect() { + const first = userMsg(TASK_RUN) + const a1 = assistantMsg() + const redirect = userMsg(TASK_REDIRECT) + const a2 = assistantMsg() + const marker = userMsg("", { compaction: true }) + const summary = assistantMsg({ summary: true }) + const cont = userMsg("Continue if you have next steps.", { synthetic: true }) + return { history: [first, a1, redirect, a2, marker, summary, cont], first, redirect, summary, cont } +} + +describe("selectPinSource — mode-aware pin selection", () => { + test("run mode pins the FIRST non-synthetic user message", () => { + const { history } = historyWithRedirect() + const source = SessionPrompt.selectPinSource(history, true) + expect(source?.text).toBe(TASK_RUN) + }) + + test("interactive pins the MOST RECENT substantive instruction (mid-session redirect)", () => { + const { history, redirect } = historyWithRedirect() + const source = SessionPrompt.selectPinSource(history, false) + expect(source?.id).toBe(redirect.info.id) + expect(source?.text).toBe(TASK_REDIRECT) + }) + + test("synthetic-only and compaction-marker user messages are never pin sources", () => { + const { history, cont } = historyWithRedirect() + // interactive: latest substantive is the redirect, NOT the synthetic continue msg + const source = SessionPrompt.selectPinSource(history, false) + expect(source?.id).not.toBe(cont.info.id) + // a history of only synthetic/marker messages yields no pin + const empty = SessionPrompt.selectPinSource( + [userMsg("", { compaction: true }), userMsg("continue", { synthetic: true })], + false, + ) + expect(empty).toBeUndefined() + }) + + test("empty history yields no pin", () => { + expect(SessionPrompt.selectPinSource([], true)).toBeUndefined() + }) +}) + +describe("taskPinText — compaction-gated assembly", () => { + test("mid-session redirect: after compaction the interactive pin reflects the LATER instruction", () => { + const { history, summary, cont } = historyWithRedirect() + const visible = [summary, cont] + const pin = SessionPrompt.taskPinText({ + history, + visible, + runMode: false, + capTokens: 4_096, + cardCapTokens: 500, + }) + expect(pin).toBeDefined() + expect(pin!).toContain("Original task — authoritative over any summary") + expect(pin!).toContain(TASK_REDIRECT) + expect(pin!).not.toContain(TASK_RUN) + }) + + test("run mode pins the ORIGINAL first task through the same compaction", () => { + const { history, summary, cont } = historyWithRedirect() + const pin = SessionPrompt.taskPinText({ + history, + visible: [summary, cont], + runMode: true, + capTokens: 4_096, + cardCapTokens: 500, + }) + expect(pin).toBeDefined() + expect(pin!).toContain(TASK_RUN) + }) + + test("skipped while the pin source is still visible verbatim in context", () => { + const { history, redirect, summary, cont } = historyWithRedirect() + const pin = SessionPrompt.taskPinText({ + history, + visible: [redirect, summary, cont], + runMode: false, + capTokens: 4_096, + cardCapTokens: 500, + }) + expect(pin).toBeUndefined() + }) + + test("zero budget yields no pin", () => { + const { history, summary, cont } = historyWithRedirect() + const pin = SessionPrompt.taskPinText({ + history, + visible: [summary, cont], + runMode: true, + capTokens: 0, + cardCapTokens: 500, + }) + expect(pin).toBeUndefined() + }) +}) + +describe("buildPinnedTask — verbatim under cap, head+tail + contract card over cap", () => { + const filler = Array.from({ length: 400 }, (_, i) => `Background paragraph ${i} with routine detail.`).join(" ") + const longTask = [ + "Rebuild the daily channel sales mart.", + `Output MUST go to reports/final_output.csv and the model lives in models/marts/fct_daily_channel_sales.sql.`, + filler, + "Use the column customer_lifetime_value and run `make verify` after every change.", + 'Do not modify the "legacy_billing" schema under any circumstances.', + filler, + "Finish by summarizing results.", + ].join("\n") + + test("under cap: text is pinned verbatim, unmodified", () => { + const out = SessionPrompt.buildPinnedTask({ text: TASK_RUN, capTokens: 4_096, cardCapTokens: 500 }) + expect(out).toBe(TASK_RUN) + }) + + test("over cap: verbatim head + tail, budget respected, contract card preserves mid-task literals", () => { + const cap = 1_000 + expect(Token.estimate(longTask)).toBeGreaterThan(cap) + const out = SessionPrompt.buildPinnedTask({ text: longTask, capTokens: cap, cardCapTokens: 500 }) + expect(out).toBeDefined() + expect(Token.estimate(out!)).toBeLessThanOrEqual(cap) + // verbatim head and tail + expect(out!.startsWith("Rebuild the daily channel sales mart.")).toBe(true) + expect(out!).toContain("truncated") + // mid-task literals that middle-truncation alone would delete survive in the card + expect(out!).toContain("Contract card") + expect(out!).toContain("fct_daily_channel_sales") + expect(out!).toContain("final_output.csv") + }) + + test("contract card entries are verbatim substrings of the task — never paraphrased", () => { + const card = SessionPrompt.extractContractCard(longTask, 500) + expect(card).not.toBe("") + expect(Token.estimate(card)).toBeLessThanOrEqual(500) + for (const line of card.split("\n").slice(1)) { + const body = line.replace(/^- (files\/paths|identifiers|code\/commands|quoted terms): /, "").replace(/^- constraints \(verbatim lines\):$/, "").replace(/^ {2}- /, "") + if (!body) continue + for (const item of body.split(", ")) { + if (!item) continue + expect(longTask).toContain(item) + } + } + }) + + test("contract card is deterministic and extracts all literal families", () => { + const a = SessionPrompt.extractContractCard(longTask, 500) + const b = SessionPrompt.extractContractCard(longTask, 500) + expect(a).toBe(b) + expect(a).toContain("reports/final_output.csv") + expect(a).toContain("customer_lifetime_value") + expect(a).toContain("make verify") + expect(a).toContain("legacy_billing") + expect(a).toContain('Do not modify the "legacy_billing" schema under any circumstances.') + }) + + test("contract card respects a tiny cap by tail-truncating", () => { + const tiny = SessionPrompt.extractContractCard(longTask, 30) + expect(Token.estimate(tiny)).toBeLessThanOrEqual(30) + }) +}) + +describe("pinBudget — dynamic cap min(4k, fraction × usable) with the livelock invariant", () => { + beforeEach(() => SessionCompaction.resetPinState()) + + test("large window: capped at PIN_MAX_TOKENS (4k)", () => { + // context 200k, output 8k → reserved default 20k, threshold 180k; + // fraction cap 31.5k, invariant cap 158k → min is 4096. + const budget = SessionCompaction.pinBudget({ cfg: cfg(), model: model({ context: 200_000, output: 8_192 }) }) + expect(budget).toBe(SessionCompaction.PIN_MAX_TOKENS) + }) + + test("mid window: fraction of the effective threshold wins over 4k", () => { + // context 16k, output 2k, reserved 2k (config) → headroom 2k; effective + // threshold min(14k, max(floor(16k × 0.65) − 2k, 4k)) = 8,400; + // fraction cap floor(8,400 × 0.175) = 1,470; invariant cap 8,400 − 2k − 2k = 4,400. + const threshold = SessionCompaction.overflowThreshold({ base: 16_000, headroom: 2_000, fraction: 0.65 }) + expect(threshold).toBe(8_400) + const budget = SessionCompaction.pinBudget({ + cfg: cfg({ reserved: 2_000 }), + model: model({ context: 16_000, output: 2_000 }), + }) + expect(budget).toBe(Math.floor(threshold * SessionCompaction.PIN_WINDOW_FRACTION)) + expect(budget).toBeLessThan(SessionCompaction.PIN_MAX_TOKENS) + }) + + test("pin capacity comes from the SAME threshold isOverflow uses — 65,536/20,000 boundary case", () => { + // context 65,536, reserved 20,000, output 8,192 → headroom 20,000. + // Estimate-domain boundary: min(45,536, max(floor(65,536 × 0.65) − 20,000, 4,000)) = 22,598. + // The threshold ALREADY excludes the reserved headroom, so the invariant is + // pin + working slack < threshold (subtracting reserved again double-counted + // it and shrank the pin to 598 here): fraction cap floor(22,598 × 0.175) = + // 3,954 binds, below both PIN_MAX_TOKENS and the invariant cap 20,598. + const threshold = SessionCompaction.overflowThreshold({ base: 65_536, headroom: 20_000, fraction: 0.65 }) + expect(threshold).toBe(22_598) + const budget = SessionCompaction.pinBudget({ cfg: cfg(), model: model({ context: 65_536, output: 8_192 }) }) + expect(budget).toBe(Math.floor(threshold * SessionCompaction.PIN_WINDOW_FRACTION)) + // The livelock invariant holds against the ACTUAL trigger. + expect(budget + SessionCompaction.PIN_WORKING_SLACK).toBeLessThanOrEqual(threshold) + }) + + test("small window: the invariant still admits a small pin instead of silently zeroing it", () => { + // context 32k, output 4k → reserved default 20k, estimate-domain threshold + // 4,000 (floor). Invariant cap 4,000 − 2,000 = 2,000; fraction cap + // floor(4,000 × 0.175) = 700 binds. The old double-subtract arithmetic + // (threshold − reserved − slack < 0) forced 0 on every window this size. + const threshold = SessionCompaction.overflowThreshold({ base: 32_000, headroom: 20_000, fraction: 0.65 }) + const budget = SessionCompaction.pinBudget({ cfg: cfg(), model: model({ context: 32_000, output: 4_096 }) }) + expect(budget).toBe(Math.floor(threshold * SessionCompaction.PIN_WINDOW_FRACTION)) + expect(budget + SessionCompaction.PIN_WORKING_SLACK).toBeLessThanOrEqual(threshold) + }) + + test("degenerate window: pin is 0 when even slack exceeds the threshold (skip, never violate)", () => { + // Force a threshold at/below the working slack via a tiny explicit reserved + // buffer and window: base 5,000, reserved 4,000 → threshold min(1,000, …) + // <= PIN_WORKING_SLACK → invariant cap <= 0 → no pin fits. + const budget = SessionCompaction.pinBudget({ + cfg: cfg({ reserved: 4_000 }), + model: model({ context: 5_000, output: 500 }), + }) + expect(budget).toBe(0) + }) + + test("config overrides: pin_task=false disables; pin_max_tokens respected", () => { + const m = model({ context: 200_000, output: 8_192 }) + expect(SessionCompaction.pinBudget({ cfg: cfg({ pin_task: false }), model: m })).toBe(0) + expect(SessionCompaction.pinBudget({ cfg: cfg({ pin_max_tokens: 1_024 }), model: m })).toBe(1_024) + }) + + test("zero-context model yields no pin", () => { + expect(SessionCompaction.pinBudget({ cfg: cfg(), model: model({ context: 0 }) })).toBe(0) + }) +}) + +describe("livelock guard — two consecutive failed compactions halve the pin", () => { + beforeEach(() => SessionCompaction.resetPinState()) + const SID = "ses_livelock" + + function immediateRefire() { + // a completed summary followed by only ONE finished non-summary assistant: + // the previous compaction did not get the session below threshold. + return [assistantMsg({ summary: true }), userMsg("continue", { synthetic: true }), assistantMsg()] + } + + function normalProgress() { + return [ + assistantMsg({ summary: true }), + userMsg("continue", { synthetic: true }), + assistantMsg(), + assistantMsg(), + assistantMsg(), + ] + } + + test("first compaction of a session never counts as a failure", () => { + SessionCompaction.notePinCompaction(SID, [userMsg(TASK_RUN), assistantMsg()] as any) + expect(SessionCompaction.pinScale(SID)).toBe(1) + }) + + test("two consecutive immediate re-fires halve the pin; budget scales down", () => { + SessionCompaction.notePinCompaction(SID, immediateRefire() as any) + expect(SessionCompaction.pinScale(SID)).toBe(1) + SessionCompaction.notePinCompaction(SID, immediateRefire() as any) + expect(SessionCompaction.pinScale(SID)).toBe(0.5) + const budget = SessionCompaction.pinBudget({ + cfg: cfg(), + model: model({ context: 200_000, output: 8_192 }), + sessionID: SID, + }) + expect(budget).toBe(SessionCompaction.PIN_MAX_TOKENS / 2) + }) + + test("a compaction after real progress resets the consecutive-failure count", () => { + SessionCompaction.notePinCompaction(SID, immediateRefire() as any) + SessionCompaction.notePinCompaction(SID, normalProgress() as any) + SessionCompaction.notePinCompaction(SID, immediateRefire() as any) + expect(SessionCompaction.pinScale(SID)).toBe(1) + }) + + test("further failure pairs keep halving; state is per-session", () => { + for (let i = 0; i < 4; i++) SessionCompaction.notePinCompaction(SID, immediateRefire() as any) + expect(SessionCompaction.pinScale(SID)).toBe(0.25) + expect(SessionCompaction.pinScale("ses_other")).toBe(1) + }) +}) + +describe("summary-template addition", () => { + test("is an addition constant, phrased as do-not-restate + pinned-task authority", () => { + expect(SessionCompaction.PIN_SUMMARY_ADDITION).toContain("Do NOT restate") + expect(SessionCompaction.PIN_SUMMARY_ADDITION).toContain("pinned") + expect(SessionCompaction.PIN_SUMMARY_ADDITION).toContain("authoritative") + }) +}) + +describe("resolvePinRunMode — explicit run-mode value wins over the legacy fallback", () => { + test("explicit ALTIMATE_RUN_MODE=0 wins even when ALTIMATE_NON_INTERACTIVE=1", () => { + expect(SessionPrompt.resolvePinRunMode({ ALTIMATE_RUN_MODE: "0", ALTIMATE_NON_INTERACTIVE: "1" })).toBe(false) + }) + + test("explicit ALTIMATE_RUN_MODE=1 wins regardless of the fallback", () => { + expect(SessionPrompt.resolvePinRunMode({ ALTIMATE_RUN_MODE: "1" })).toBe(true) + expect(SessionPrompt.resolvePinRunMode({ ALTIMATE_RUN_MODE: "1", ALTIMATE_NON_INTERACTIVE: "0" })).toBe(true) + }) + + test("legacy fallback applies only when the marker is undefined or blank", () => { + expect(SessionPrompt.resolvePinRunMode({ ALTIMATE_NON_INTERACTIVE: "1" })).toBe(true) + expect(SessionPrompt.resolvePinRunMode({ ALTIMATE_RUN_MODE: " ", ALTIMATE_NON_INTERACTIVE: "1" })).toBe(true) + expect(SessionPrompt.resolvePinRunMode({})).toBe(false) + }) + + // `run --continue` / `--session` / `--fork` resume a session whose first user + // message belongs to an EARLIER invocation. Run-mode selection would pin that + // stale request as authoritative over the summary and the current prompt. + test("a resumed run falls back to interactive selection", () => { + expect(SessionPrompt.resolvePinRunMode({ ALTIMATE_RUN_MODE: "1", ALTIMATE_RUN_RESUMED: "1" })).toBe(false) + expect(SessionPrompt.resolvePinRunMode({ ALTIMATE_NON_INTERACTIVE: "1", ALTIMATE_RUN_RESUMED: "1" })).toBe(false) + }) + + test("a fresh run is unaffected", () => { + expect(SessionPrompt.resolvePinRunMode({ ALTIMATE_RUN_MODE: "1" })).toBe(true) + }) +}) + +describe("selectPinSource — resumed run sessions", () => { + test("interactive selection picks this run's task, not the previous run's", () => { + // A session resumed with `run --continue ""`: the earlier + // invocation's task is still message #1. + const previous = userMsg("Previous run: migrate the staging schema.") + const current = userMsg("This run: add regression tests for the migration.") + const history = [previous, current] + // Run-mode selection would hand back the completed previous task. + expect(SessionPrompt.selectPinSource(history, true)?.id).toBe(previous.info.id) + // The resumed path resolves runMode=false and gets the current request. + const source = SessionPrompt.selectPinSource(history, false) + expect(source?.id).toBe(current.info.id) + expect(source?.text).toContain("regression tests") + }) +}) diff --git a/packages/opencode/test/session/termination.test.ts b/packages/opencode/test/session/termination.test.ts new file mode 100644 index 0000000000..a7cc5c8d84 --- /dev/null +++ b/packages/opencode/test/session/termination.test.ts @@ -0,0 +1,224 @@ +// Harness reliability unit gates — SessionTermination completion-token +// contract and the explicit-DONE stop-path decision. +// +// (a): "finished naturally" requires finishReason "stop" PLUS an explicit +// trailing DONE assertion — never bare "stop". +import { describe, expect, test } from "bun:test" +import { SessionTermination } from "../../src/session/termination" + +describe("SessionTermination.isExplicitDone", () => { + test("accepts a standalone final-line DONE assertion", () => { + expect(SessionTermination.isExplicitDone("DONE")).toBe(true) + expect(SessionTermination.isExplicitDone("All 14 checks green.\nDONE")).toBe(true) + expect(SessionTermination.isExplicitDone("Verified the build.\n\nDONE")).toBe(true) + expect(SessionTermination.isExplicitDone("DONE ")).toBe(true) + expect(SessionTermination.isExplicitDone("DONE\n\n")).toBe(true) + expect(SessionTermination.isExplicitDone(" DONE ")).toBe(true) + }) + + test("rejects ordinary text and mid-sentence mentions", () => { + expect(SessionTermination.isExplicitDone("Let me now read the schema file.")).toBe(false) + expect(SessionTermination.isExplicitDone("marked the TODO as DONE and moving on")).toBe(false) + expect(SessionTermination.isExplicitDone("DONE with step 1, continuing to step 2")).toBe(false) + expect(SessionTermination.isExplicitDone("")).toBe(false) + }) + + test("requires exactly DONE on the final line — no punctuation or markup wrappers", () => { + expect(SessionTermination.isExplicitDone("All checks green. DONE")).toBe(false) + expect(SessionTermination.isExplicitDone("Verified the build.\n\nDONE.")).toBe(false) + expect(SessionTermination.isExplicitDone("DONE!")).toBe(false) + expect(SessionTermination.isExplicitDone("**DONE**")).toBe(false) + expect(SessionTermination.isExplicitDone("`DONE`")).toBe(false) + }) + + test("code-fenced text ending in DONE never terminates", () => { + // Closed fence: the final line is the closing fence, not DONE. + expect(SessionTermination.isExplicitDone("Example output:\n```\nDONE\n```")).toBe(false) + // Unclosed fence: the final DONE line is inside quoted code. + expect(SessionTermination.isExplicitDone("Reply like this:\n```\nDONE")).toBe(false) + expect(SessionTermination.isExplicitDone("~~~\nDONE")).toBe(false) + // A closed fence FOLLOWED by a real plaintext DONE still terminates. + expect(SessionTermination.isExplicitDone("```\nbuild ok\n```\nDONE")).toBe(true) + }) + + test("fence state tracks marker character and length (CommonMark), not parity", () => { + // A ``` line inside an unclosed ````-fence is content, not a closer — + // the trailing DONE is still quoted material. + expect(SessionTermination.isExplicitDone("Here is the doc:\n````\n```\nDONE")).toBe(false) + // A ~~~ line that is content of a closed ``` block is not a fence — + // the final plaintext DONE is a real assertion. + expect(SessionTermination.isExplicitDone("```\n~~~\n```\nDONE")).toBe(true) + // A shorter same-character run inside an open fence does not close it. + expect(SessionTermination.isExplicitDone("`````\n```\n`````\nDONE")).toBe(true) + // A longer same-character run closes a shorter opener. + expect(SessionTermination.isExplicitDone("```\ncode\n`````\nDONE")).toBe(true) + // Tilde fences follow the same rules. + expect(SessionTermination.isExplicitDone("~~~~\n~~~\nDONE")).toBe(false) + expect(SessionTermination.isExplicitDone("~~~\n```\n~~~\nDONE")).toBe(true) + }) + + test("a fence-looking line with trailing info-string text never closes the fence", () => { + // Per CommonMark, only whitespace may follow a closing fence's marker — + // a line like "```not-a-closer" is content (or a nested opener), not a + // closer. The DONE that follows is still inside the open fence. + expect(SessionTermination.isExplicitDone("```\ncode\n```not-a-closer\nDONE")).toBe(false) + expect(SessionTermination.isExplicitDone("~~~\ncode\n~~~lang\nDONE")).toBe(false) + // A real closer with trailing whitespace only still closes normally. + expect(SessionTermination.isExplicitDone("```\ncode\n``` \nDONE")).toBe(true) + }) + + test("quoted and indented-code DONE never terminates", () => { + expect(SessionTermination.isExplicitDone("The instructions said:\n> DONE")).toBe(false) + expect(SessionTermination.isExplicitDone("Example:\n DONE")).toBe(false) + expect(SessionTermination.isExplicitDone("Example:\n\tDONE")).toBe(false) + }) + + test("is case-sensitive: prose 'done' never counts", () => { + expect(SessionTermination.isExplicitDone("I'm done")).toBe(false) + expect(SessionTermination.isExplicitDone("done.")).toBe(false) + expect(SessionTermination.isExplicitDone("Done")).toBe(false) + }) + + test("does not match inside a longer trailing word", () => { + expect(SessionTermination.isExplicitDone("ABANDONED")).toBe(false) + expect(SessionTermination.isExplicitDone("UNDONE")).toBe(false) + }) +}) + +describe("SessionTermination.explicitDoneStop (stop-path decision)", () => { + const textPart = (text: string, synthetic?: boolean) => ({ type: "text", text, synthetic }) + + test("errorless stop + trailing DONE in the final real text part → stop", () => { + expect( + SessionTermination.explicitDoneStop({ + finish: "stop", + hasError: false, + parts: [{ type: "tool" }, textPart("Everything verified.\nDONE")], + }), + ).toBe(true) + }) + + test("bare finishReason stop is NEVER enough", () => { + expect( + SessionTermination.explicitDoneStop({ + finish: "stop", + hasError: false, + parts: [textPart("Let me now read the schema file.")], + }), + ).toBe(false) + }) + + test("non-stop finish reasons never terminate", () => { + for (const finish of ["tool-calls", "length", "error", "content-filter", "unknown", undefined]) { + expect( + SessionTermination.explicitDoneStop({ + finish, + hasError: false, + parts: [textPart("DONE")], + }), + ).toBe(false) + } + }) + + test("an errored turn never terminates via DONE", () => { + expect( + SessionTermination.explicitDoneStop({ + finish: "stop", + hasError: true, + parts: [textPart("DONE")], + }), + ).toBe(false) + }) + + test("synthetic (system-authored) text parts are ignored", () => { + expect( + SessionTermination.explicitDoneStop({ + finish: "stop", + hasError: false, + parts: [textPart("still working"), textPart("reply with DONE and stop.", true)], + }), + ).toBe(false) + }) + + test("the LAST real text part governs (a later non-DONE text clears it)", () => { + expect( + SessionTermination.explicitDoneStop({ + finish: "stop", + hasError: false, + parts: [textPart("DONE"), textPart("actually, one more thing")], + }), + ).toBe(false) + }) + + test("no text parts at all → no stop", () => { + expect( + SessionTermination.explicitDoneStop({ finish: "stop", hasError: false, parts: [{ type: "tool" }] }), + ).toBe(false) + }) +}) + +describe("SessionTermination directive texts (/c/d wording contracts)", () => { + test("the completion nudge offers all three options and instructs the DONE token", () => { + const nudge = SessionTermination.COMPLETION_NUDGE + expect(nudge).toContain("(1)") + expect(nudge).toContain("(2)") + expect(nudge).toContain("(3)") + expect(nudge).toContain("ask for clarification") + expect(nudge).toContain(`reply with ${SessionTermination.DONE_TOKEN}`) + }) + + test("the confirm challenge asks for DONE or what remains — outcome-neutral", () => { + const challenge = SessionTermination.CONFIRM_DONE_CHALLENGE + expect(challenge).toContain(SessionTermination.DONE_TOKEN) + expect(challenge).toContain("state specifically what remains") + }) + + test("the overflow notice is mechanism-accurate: no media-attachment blame", () => { + expect(SessionTermination.OVERFLOW_NOTICE).not.toContain("media") + expect(SessionTermination.OVERFLOW_NOTICE).toContain("context limit") + }) + + test("no vertical/product tokens in any directive text (leak-lens hard requirement)", () => { + for (const text of [ + SessionTermination.COMPLETION_NUDGE, + SessionTermination.CONFIRM_DONE_CHALLENGE, + SessionTermination.OVERFLOW_NOTICE, + ]) { + expect(text.toLowerCase()).not.toContain("dbt") + expect(text.toLowerCase()).not.toContain("warehouse") + expect(text.toLowerCase()).not.toContain("sql") + } + }) +}) + +// Regression coverage for the fence-state tracker's CommonMark conformance. +describe("SessionTermination.isExplicitDone — fence-state conformance", () => { + test("CRLF input still closes fences, so a real DONE is accepted", () => { + // Interior lines keep a trailing \r before normalization, which made the + // closing fence fail its whitespace-only check and left the fence open. + expect(SessionTermination.isExplicitDone("```sh\r\necho hi\r\n```\r\n\r\nDONE\r\n")).toBe(true) + }) + + test("CRLF input inside an UNCLOSED fence is still rejected", () => { + expect(SessionTermination.isExplicitDone("```sh\r\necho hi\r\nDONE\r\n")).toBe(false) + }) + + test("bare-CR input splits into lines rather than collapsing to one", () => { + expect(SessionTermination.isExplicitDone("```sh\recho hi\r```\r\rDONE\r")).toBe(true) + }) + + test("a backtick run whose info string contains a backtick is not an opener", () => { + // CommonMark: a backtick fence's info string may not contain a backtick, so + // this line is paragraph text. Treating it as an opener made the next + // backtick run read as its closer, exposing the block interior. + expect(SessionTermination.isExplicitDone(["```foo`bar", "```", "DONE"].join("\n"))).toBe(false) + }) + + test("a tilde fence's info string may contain backticks", () => { + expect(SessionTermination.isExplicitDone(["~~~foo`bar", "x", "~~~", "DONE"].join("\n"))).toBe(true) + }) + + test("a closing fence may not carry an info string", () => { + expect(SessionTermination.isExplicitDone(["```sh", "x", "```sh", "DONE"].join("\n"))).toBe(false) + }) +}) diff --git a/packages/opencode/test/session/tool-callid-sanitize.test.ts b/packages/opencode/test/session/tool-callid-sanitize.test.ts new file mode 100644 index 0000000000..bd50c436c9 --- /dev/null +++ b/packages/opencode/test/session/tool-callid-sanitize.test.ts @@ -0,0 +1,248 @@ +// — tool-call id sanitation. Malformed (non-string) tool-call ids must be +// coerced/regenerated DETERMINISTICALLY at ingestion (processor.ts) with the +// mapping propagated atomically to the paired tool-result, and the replay path +// (message-v2.ts toModelMessages) must apply the same coercion so both halves of +// a persisted pair render identical toolCallId values. A regenerated call id with +// an un-regenerated result id 400s every subsequent provider request. +import { describe, expect, test } from "bun:test" +import type { SessionV1 } from "@opencode-ai/core/v1/session" +import { MessageV2 } from "../../src/session/message-v2" +import { SessionProcessor } from "../../src/session/processor" +import type { Provider } from "@/provider/provider" +import { SessionID, MessageID, PartID } from "../../src/session/schema" +import { ProviderID, ModelID } from "../../src/provider/schema" + +const sessionID = SessionID.make("session") +const providerID = ProviderID.make("test") +const model: Provider.Model = { + id: ModelID.make("test-model"), + providerID, + api: { + id: "test-model", + url: "https://example.com", + npm: "@ai-sdk/openai", + }, + name: "Test Model", + capabilities: { + temperature: true, + reasoning: false, + attachment: false, + toolcall: true, + input: { text: true, audio: false, image: false, video: false, pdf: false }, + output: { text: true, audio: false, image: false, video: false, pdf: false }, + interleaved: false, + }, + cost: { + input: 0, + output: 0, + cache: { read: 0, write: 0 }, + }, + limit: { context: 128000, output: 4096 }, + status: "active", + options: {}, + headers: {}, + release_date: "2026-01-01", +} as unknown as Provider.Model + +function basePart(messageID: string, id: string) { + return { + id: PartID.make(`prt_${id}`), + sessionID, + messageID: MessageID.make(`msg_${messageID}`), + } +} + +function userMsg(id: string): SessionV1.WithParts { + return { + info: { + id, + sessionID, + role: "user", + time: { created: 0 }, + agent: "user", + model: { providerID, modelID: ModelID.make("test") }, + tools: {}, + mode: "", + }, + parts: [{ ...basePart(id, "u1"), type: "text", text: "run tool" }], + } as unknown as SessionV1.WithParts +} + +function assistantToolMsg(id: string, callID: unknown): SessionV1.WithParts { + return { + info: { + id, + sessionID, + role: "assistant", + time: { created: 0 }, + parentID: "m-user", + modelID: model.api.id, + providerID, + mode: "", + agent: "agent", + path: { cwd: "/", root: "/" }, + cost: 0, + tokens: { input: 0, output: 0, reasoning: 0, cache: { read: 0, write: 0 } }, + }, + parts: [ + { + ...basePart(id, "a1"), + type: "tool", + callID, + tool: "bash", + state: { + status: "completed", + input: { cmd: "ls" }, + output: "ok", + title: "Bash", + metadata: {}, + time: { start: 0, end: 1 }, + }, + }, + ], + } as unknown as SessionV1.WithParts +} + +/** Extract the tool-call and tool-result ids from replayed model messages. */ +function pairIDs(messages: Awaited>) { + const callIDs: unknown[] = [] + const resultIDs: unknown[] = [] + for (const msg of messages) { + if (!Array.isArray(msg.content)) continue + for (const item of msg.content) { + if (item.type === "tool-call") callIDs.push(item.toolCallId) + if (item.type === "tool-result") resultIDs.push(item.toolCallId) + } + } + return { callIDs, resultIDs } +} + +describe("MessageV2.sanitizeToolCallID", () => { + test("passes valid non-empty string ids through untouched", () => { + expect(MessageV2.sanitizeToolCallID("call_abc123")).toBe("call_abc123") + }) + + test("regenerates non-string ids deterministically", () => { + const a = MessageV2.sanitizeToolCallID(12345) + const b = MessageV2.sanitizeToolCallID(12345) + expect(a).toBe(b) + expect(typeof a).toBe("string") + expect(a.length).toBeGreaterThan(0) + }) + + test("distinct malformed ids map to distinct sanitized ids", () => { + expect(MessageV2.sanitizeToolCallID(1)).not.toBe(MessageV2.sanitizeToolCallID(2)) + }) + + test("handles empty string, null, undefined, and objects without throwing", () => { + for (const raw of ["", null, undefined, { id: 1 }, []]) { + const out = MessageV2.sanitizeToolCallID(raw) + expect(typeof out).toBe("string") + expect(out.length).toBeGreaterThan(0) + expect(out).toBe(MessageV2.sanitizeToolCallID(raw)) + } + }) +}) + +describe("SessionProcessor.createToolCallIDCoercer (ingestion half)", () => { + test("coerces a malformed call id and propagates the SAME id to the paired result", () => { + const coerce = SessionProcessor.createToolCallIDCoercer() + const callHalf = coerce(42) // tool-input-start / tool-call event + const resultHalf = coerce(42) // tool-result event + expect(callHalf).toBe(resultHalf) + expect(typeof callHalf).toBe("string") + }) + + test("pairs survive a provider type flip (numeric call id, string result id)", () => { + const coerce = SessionProcessor.createToolCallIDCoercer() + const callHalf = coerce(42) + const resultHalf = coerce("42") // some servers stringify the id on the result event + expect(resultHalf).toBe(callHalf) + }) + + test("valid string ids are untouched so healthy providers see no behavior change", () => { + const coerce = SessionProcessor.createToolCallIDCoercer() + expect(coerce("call_ok")).toBe("call_ok") + }) +}) + +describe("malformed-id round-trip: ingest → persist → replay", () => { + test("ingested-then-persisted id replays with matching call/result ids", async () => { + // Ingestion: the processor's coercer regenerates the malformed id; the + // sanitized value is what gets persisted as the part's callID. + const coerce = SessionProcessor.createToolCallIDCoercer() + const persistedCallID = coerce(9876) + expect(coerce("9876")).toBe(persistedCallID) // paired result resolves to the same part + + // Replay: the persisted transcript renders both halves with the same string id. + const replayed = await MessageV2.toModelMessages( + [userMsg("m-user"), assistantToolMsg("m-assistant", persistedCallID)] as unknown as MessageV2.WithParts[], + model, + ) + const { callIDs, resultIDs } = pairIDs(replayed) + expect(callIDs).toEqual([persistedCallID]) + expect(resultIDs).toEqual([persistedCallID]) + }) + + test("replay defensively coerces a malformed PERSISTED id identically on both halves", async () => { + // Transcripts written before the ingestion fix may carry non-string callIDs. + const replayed = await MessageV2.toModelMessages( + [userMsg("m-user"), assistantToolMsg("m-assistant", 12345)] as unknown as MessageV2.WithParts[], + model, + ) + const { callIDs, resultIDs } = pairIDs(replayed) + expect(callIDs).toHaveLength(1) + expect(resultIDs).toHaveLength(1) + const expected = MessageV2.sanitizeToolCallID(12345) + expect(callIDs[0]).toBe(expected) + expect(resultIDs[0]).toBe(expected) + expect(typeof callIDs[0]).toBe("string") + }) + + test("ingestion and replay halves produce identical output for the same raw id", () => { + // The contract that keeps a pair consistent across the two code paths. + const coerce = SessionProcessor.createToolCallIDCoercer() + for (const raw of [7, "7x", 0, { a: 1 }, ""]) { + expect(coerce(raw)).toBe(MessageV2.sanitizeToolCallID(raw)) + } + }) + + test("prototype-key ids are safe: __proto__/constructor/toString never resolve to inherited members", () => { + const coerce = SessionProcessor.createToolCallIDCoercer() + for (const raw of ["__proto__", "constructor", "toString", "hasOwnProperty", "valueOf"]) { + const first = coerce(raw) + // Valid non-empty strings pass through as themselves — but critically as + // STRINGS, never as inherited Object.prototype members. + expect(typeof first).toBe("string") + expect(first).toBe(raw) + // The alias lookup on repeat must return the same string, not an object. + const second = coerce(raw) + expect(second).toBe(raw) + expect(typeof second).toBe("string") + } + // No prototype pollution occurred. + expect(({} as any).polluted).toBeUndefined() + }) + + test("per-processor salt separates regenerated ids for identical malformed raw values", () => { + const a = SessionProcessor.createToolCallIDCoercer("msg_a") + const b = SessionProcessor.createToolCallIDCoercer("msg_b") + for (const raw of ["", 0, null, { a: 1 }]) { + const idA = a(raw) + const idB = b(raw) + expect(idA).not.toBe(idB) + expect(idA).toMatch(/^call_[0-9a-f]{8}$/) + expect(idB).toMatch(/^call_[0-9a-f]{8}$/) + } + // Within one processor the mapping stays deterministic (pairing contract). + expect(a("")).toBe(a("")) + expect(MessageV2.sanitizeToolCallID("", "msg_a")).toBe(a("")) + }) + + test("salted regeneration still passes persisted valid ids through unchanged on replay", () => { + // Persisted sanitized ids are valid strings; replay must not re-hash them. + const persisted = MessageV2.sanitizeToolCallID(999, "msg_a") + expect(MessageV2.sanitizeToolCallID(persisted)).toBe(persisted) + expect(MessageV2.sanitizeToolCallID(persisted, "different-salt")).toBe(persisted) + }) +}) diff --git a/packages/opencode/test/session/tool-result-cap.test.ts b/packages/opencode/test/session/tool-result-cap.test.ts new file mode 100644 index 0000000000..ae0d416bf6 --- /dev/null +++ b/packages/opencode/test/session/tool-result-cap.test.ts @@ -0,0 +1,148 @@ +import { describe, expect, test } from "bun:test" +import { ToolResultCap } from "../../src/session/tool-result-cap" +import { TruncateCore } from "../../src/tool/truncate-core" +import { Token } from "../../src/util/token" + +// ─── per-tool-result dispatch cap ──────────────────────────────── +// A single tool result must never exceed a bounded token estimate at +// dispatch time. Production incident: one giant duckdb/query dump jumped a +// ~4K-token conversation past a 65K window in one step, bypassing the +// tool-level truncation service. + +const MODEL_65K = { limit: { context: 65_536 } } + +describe("ToolResultCap.resolve", () => { + test("explicit config dispatch_max_tokens wins", () => { + const cap = ToolResultCap.resolve({ + config: { tool_output: { dispatch_max_tokens: 1_234 } }, + model: MODEL_65K, + safetyFraction: 0.65, + }) + expect(cap).toBe(1_234) + }) + + test("default is min(max_bytes-derived estimate, 15% of effective limit)", () => { + // existing cap: ceil(51,200 / 3.0) = 17,067 tokens + // effective limit: floor(65,536 * 0.65) = 42,598 → 15% = 6,389 tokens + const cap = ToolResultCap.resolve({ model: MODEL_65K, safetyFraction: 0.65 }) + expect(cap).toBe(6_389) + expect(cap).toBeLessThan(Math.ceil(TruncateCore.MAX_BYTES / ToolResultCap.MIN_CHARS_PER_TOKEN)) + }) + + test("large-context model: the byte-derived cap is the binding constraint", () => { + // 15% of floor(1M * 0.65) = 97,500 → existing cap 17,067 wins + const cap = ToolResultCap.resolve({ model: { limit: { context: 1_000_000 } }, safetyFraction: 0.65 }) + expect(cap).toBe(Math.ceil(TruncateCore.MAX_BYTES / ToolResultCap.MIN_CHARS_PER_TOKEN)) + }) + + test("configured tool_output.max_bytes feeds the byte-derived cap", () => { + const cap = ToolResultCap.resolve({ + config: { tool_output: { max_bytes: 9_000 } }, + model: { limit: { context: 1_000_000 } }, + safetyFraction: 0.65, + }) + expect(cap).toBe(Math.ceil(9_000 / ToolResultCap.MIN_CHARS_PER_TOKEN)) + }) + + test("unknown model limits fall back to the CONSERVATIVE small-window bound, not ~17K", () => { + // The byte-derived cap (~17K tokens) can overwhelm a small window on its + // own; unknown limits assume the smallest protected window instead. + expect(ToolResultCap.UNKNOWN_MODEL_CAP_TOKENS).toBeLessThan( + Math.ceil(TruncateCore.MAX_BYTES / ToolResultCap.MIN_CHARS_PER_TOKEN), + ) + expect(ToolResultCap.resolve({})).toBe(ToolResultCap.UNKNOWN_MODEL_CAP_TOKENS) + expect(ToolResultCap.resolve({ model: { limit: { context: 0 } } })).toBe(ToolResultCap.UNKNOWN_MODEL_CAP_TOKENS) + // A configured max_bytes below the bound stays binding. + expect(ToolResultCap.resolve({ config: { tool_output: { max_bytes: 3_000 } } })).toBe( + Math.ceil(3_000 / ToolResultCap.MIN_CHARS_PER_TOKEN), + ) + }) + + test("limit.input takes precedence over limit.context", () => { + const cap = ToolResultCap.resolve({ + model: { limit: { context: 1_000_000, input: 65_536 } }, + safetyFraction: 0.65, + }) + expect(cap).toBe(6_389) + }) +}) + +describe("ToolResultCap.apply", () => { + test("output within the cap passes through unchanged", () => { + const output = "select 1;\n".repeat(50) + const result = ToolResultCap.apply(output, 6_389) + expect(result.truncated).toBe(false) + expect(result.content).toBe(output) + }) + + test("cap of 0 disables enforcement", () => { + const giant = "x".repeat(1_000_000) + const result = ToolResultCap.apply(giant, 0) + expect(result.truncated).toBe(false) + expect(result.content).toBe(giant) + }) + + test("giant single-result case: dense multi-line dump is bounded below the cap", () => { + // ~400KB of dense query rows — the shape that overflowed a 65K window. + const rows: string[] = [] + for (let i = 0; i < 8_000; i++) { + rows.push(`{"order_id":${i},"customer":"c-${i}","total":${i * 13.37},"status":"SHIPPED","ts":"2026-08-19"}`) + } + const output = rows.join("\n") + const cap = 6_389 + expect(Token.estimate(output)).toBeGreaterThan(cap) + + const result = ToolResultCap.apply(output, cap) + expect(result.truncated).toBe(true) + // Bounded: kept bytes ≤ cap * 3 chars/token, plus the ~fixed-size notice. + expect(Token.estimate(result.content)).toBeLessThanOrEqual(cap) + // Middle truncation keeps head AND tail, with the standard marker + notice. + expect(result.content.startsWith('{"order_id":0,')).toBe(true) + expect(result.content).toContain('"order_id":7999') + expect(result.content).toMatch(/\.\.\.\d+ (bytes|lines) truncated\.\.\./) + expect(result.content).toContain("output exceeded the per-result context budget and was truncated") + }) + + test("giant SINGLE-LINE dump (minified JSON) still keeps head and tail", () => { + const giant = '{"rows":["' + "abcdef".repeat(20_000) + '"]}' + const cap = 2_000 + expect(giant.includes("\n")).toBe(false) + expect(Token.estimate(giant)).toBeGreaterThan(cap) + + const result = ToolResultCap.apply(giant, cap) + expect(result.truncated).toBe(true) + expect(Token.estimate(result.content)).toBeLessThanOrEqual(cap) + expect(result.content.startsWith('{"rows":[')).toBe(true) + expect(result.content.trimEnd().endsWith("]}")).toBe(true) + }) + + test("framing is reserved INSIDE the cap: final output estimate <= cap", () => { + const rows = Array.from({ length: 4_000 }, (_, i) => `{"id":${i},"value":"row-${i}"}`) + const output = rows.join("\n") + for (const cap of [200, 500, 2_000]) { + const result = ToolResultCap.apply(output, cap) + expect(result.truncated).toBe(true) + expect(Token.estimate(result.content)).toBeLessThanOrEqual(cap) + expect(result.content).toContain("per-result context budget") + } + }) + + test("cap=1 edge: result is still bounded by the cap (framing dropped when it cannot fit)", () => { + const giant = "x".repeat(100_000) + const result = ToolResultCap.apply(giant, 1) + expect(result.truncated).toBe(true) + expect(Token.estimate(result.content)).toBeLessThanOrEqual(1) + expect(result.content.length).toBeGreaterThan(0) + }) + + test("incident replay: 4K conversation + one giant result stays far below a 65K window", () => { + const conversationTokens = 4_000 + const giant = "SELECT * FROM orders; -- " + "0123456789abcdef".repeat(20_000) // ~340KB dense + const cap = ToolResultCap.resolve({ model: MODEL_65K, safetyFraction: 0.65 }) + const result = ToolResultCap.apply(giant, cap) + expect(result.truncated).toBe(true) + const after = conversationTokens + Token.estimate(result.content) + // Even at the worst observed 1.55x estimator error, the real size fits. + expect(Math.ceil(after * 1.55)).toBeLessThan(65_536) + }) +}) diff --git a/packages/opencode/test/session/uncounted-tail.test.ts b/packages/opencode/test/session/uncounted-tail.test.ts new file mode 100644 index 0000000000..739fe05654 --- /dev/null +++ b/packages/opencode/test/session/uncounted-tail.test.ts @@ -0,0 +1,43 @@ +import { describe, expect, test } from "bun:test" + +import { SessionCompaction } from "../../src/session/compaction" +import type { MessageV2 } from "../../src/session/message-v2" +import type { Provider } from "../../src/provider/provider" + +// The proactive overflow path adds Token.estimate of parts appended after the +// last assistant usage reading. These tests pin the fitHead turn-boundary +// behavior that backs it (full prompt-loop testing lives in integration). + +function msg(id: string, role: "user" | "assistant", text: string): MessageV2.WithParts { + return { + info: { id, sessionID: "s", role, time: { created: 1 }, model: { providerID: "p", modelID: "m" } }, + parts: [{ id: `${id}-p`, sessionID: "s", messageID: id, type: "text", text }], + } as unknown as MessageV2.WithParts +} + +function model(context: number): Provider.Model { + return { + id: "m", providerID: "p", + api: { npm: "@ai-sdk/openai-compatible" }, + limit: { context, output: 4096 }, + } as unknown as Provider.Model +} + +describe("fitHead turn boundaries", () => { + test("truncation lands on a user message, never mid-turn", async () => { + const head: MessageV2.WithParts[] = [] + for (let i = 0; i < 24; i++) { + head.push(msg(`u${i}`, "user", "q".repeat(8_000))) + head.push(msg(`a${i}`, "assistant", "r".repeat(8_000))) + } + const result = await SessionCompaction.fitHead({ head, model: model(16384) }) + expect(result.dropped).toBeGreaterThan(0) + expect(result.head[0]!.info.role).toBe("user") + }) + + test("no truncation when the head already fits", async () => { + const head = [msg("u", "user", "small"), msg("a", "assistant", "tiny")] + const result = await SessionCompaction.fitHead({ head, model: model(131072) }) + expect(result.dropped).toBe(0) + }) +}) diff --git a/packages/opencode/test/tool/truncate-core.test.ts b/packages/opencode/test/tool/truncate-core.test.ts new file mode 100644 index 0000000000..f24d10cd0a --- /dev/null +++ b/packages/opencode/test/tool/truncate-core.test.ts @@ -0,0 +1,160 @@ +import { describe, test, expect } from "bun:test" +import { TruncateCore } from "@/tool/truncate-core" + +// Pure algorithm tests for the module shared by tool/truncate.ts (the Effect +// Service wired into every Tool.define() output, including bash) and +// tool/truncation.ts (the plain-async twin). Exercising the shared function +// directly is what guarantees a change here cannot silently apply to only +// one of the two call paths. + +function assembleDefault(text: string, opts: Partial = {}) { + const resolved: TruncateCore.ResolvedOptions = { + maxLines: TruncateCore.MAX_LINES, + maxBytes: TruncateCore.MAX_BYTES, + direction: TruncateCore.DEFAULT_DIRECTION, + headRatio: TruncateCore.DEFAULT_HEAD_RATIO, + ...opts, + } + const lines = text.split("\n") + const totalBytes = Buffer.byteLength(text, "utf-8") + if (TruncateCore.fits(lines, totalBytes, resolved.maxLines, resolved.maxBytes)) { + return { truncated: false as const, content: text } + } + const p = TruncateCore.preview(lines, totalBytes, resolved) + return { truncated: true as const, content: TruncateCore.assemble(p, "[hint]", resolved.direction), preview: p } +} + +describe("TruncateCore", () => { + test("defaults to middle direction", () => { + expect(TruncateCore.DEFAULT_DIRECTION).toBe("middle") + }) + + test("defaults to a 1/3 head : 2/3 tail split", () => { + expect(TruncateCore.DEFAULT_HEAD_RATIO).toBeCloseTo(1 / 3) + }) + + test("fits() reports false only when a limit is exceeded", () => { + expect(TruncateCore.fits(["a", "b"], 2, 10, 10)).toBe(true) + expect(TruncateCore.fits(["a", "b", "c"], 2, 2, 10)).toBe(false) + expect(TruncateCore.fits(["a", "b"], 100, 10, 10)).toBe(false) + }) + + test("head direction keeps only the leading lines", () => { + const text = Array.from({ length: 10 }, (_, i) => `line${i}`).join("\n") + const result = assembleDefault(text, { maxLines: 3, direction: "head" }) + expect(result.truncated).toBe(true) + expect(result.content).toContain("line0") + expect(result.content).toContain("line2") + expect(result.content).not.toContain("line9") + }) + + test("tail direction keeps only the trailing lines", () => { + const text = Array.from({ length: 10 }, (_, i) => `line${i}`).join("\n") + const result = assembleDefault(text, { maxLines: 3, direction: "tail" }) + expect(result.truncated).toBe(true) + expect(result.content).toContain("line9") + expect(result.content).toContain("line7") + expect(result.content).not.toContain("line0") + }) + + test("middle direction keeps head and tail, never the same line twice", () => { + const text = Array.from({ length: 12 }, (_, i) => `line${i}`).join("\n") + // maxLines 6 -> 1/3 head = 2 lines, 2/3 tail = 4 lines. + const result = assembleDefault(text, { maxLines: 6, direction: "middle" }) + expect(result.truncated).toBe(true) + expect(result.content).toContain("line0") + expect(result.content).toContain("line1") + expect(result.content).toContain("line8") + expect(result.content).toContain("line9") + expect(result.content).toContain("line10") + expect(result.content).toContain("line11") + expect(result.content).not.toContain("line5") + }) + + test("middle direction respects a custom head ratio", () => { + const text = Array.from({ length: 30 }, (_, i) => `line${i}`).join("\n") + // headRatio 0.5 with maxLines 10 -> 5 head lines, 5 tail lines. + const result = assembleDefault(text, { maxLines: 10, direction: "middle", headRatio: 0.5 }) + expect(result.truncated).toBe(true) + for (let i = 0; i < 5; i++) expect(result.content).toContain(`line${i}`) + for (let i = 25; i < 30; i++) expect(result.content).toContain(`line${i}`) + expect(result.content).not.toContain("line15") + }) + + test(">50KB log: a trailing success line survives default middle truncation", () => { + const noise = Array.from({ length: 3000 }, (_, i) => `build step ${i}: compiling module_${i}.ts`) + const successLine = "Done. PASS=42 FAIL=0" + const text = [...noise, successLine].join("\n") + expect(Buffer.byteLength(text, "utf-8")).toBeGreaterThan(50 * 1024) + + const result = assembleDefault(text) + expect(result.truncated).toBe(true) + expect(result.content).toContain(successLine) + }) + + test(">50KB log: the first error line survives default middle truncation", () => { + const firstError = "ERROR: schema.sql:1: syntax error near CREAT" + const noise = Array.from({ length: 3000 }, (_, i) => `build step ${i}: compiling module_${i}.ts`) + const text = [firstError, ...noise].join("\n") + expect(Buffer.byteLength(text, "utf-8")).toBeGreaterThan(50 * 1024) + + const result = assembleDefault(text) + expect(result.truncated).toBe(true) + expect(result.content).toContain(firstError) + }) + + test(">50KB log with both a leading error and a trailing success line: both survive", () => { + const firstError = "ERROR: schema.sql:1: syntax error near CREAT" + const successLine = "Done. PASS=42 FAIL=0" + const noise = Array.from({ length: 3000 }, (_, i) => `build step ${i}: compiling module_${i}.ts`) + const text = [firstError, ...noise, successLine].join("\n") + expect(Buffer.byteLength(text, "utf-8")).toBeGreaterThan(50 * 1024) + + const result = assembleDefault(text) + expect(result.truncated).toBe(true) + expect(result.content).toContain(firstError) + expect(result.content).toContain(successLine) + // and the elided middle noise is gone + expect(result.content).not.toContain("build step 1500") + }) + + test("reports unit as bytes when the byte budget (not the line budget) is the binding constraint", () => { + const text = "a".repeat(2000) + const result = assembleDefault(text, { maxLines: 1_000_000, maxBytes: 100, direction: "middle" }) + expect(result.truncated).toBe(true) + expect(result.content).toContain("bytes truncated") + }) + + test("assemble() places the elision marker and hint between head and tail for middle direction", () => { + const p: TruncateCore.Preview = { head: "HEAD", tail: "TAIL", removed: 5, unit: "lines" } + const content = TruncateCore.assemble(p, "HINT", "middle") + const headIdx = content.indexOf("HEAD") + const markerIdx = content.indexOf("...5 lines truncated...") + const hintIdx = content.indexOf("HINT") + const tailIdx = content.lastIndexOf("TAIL") + expect(headIdx).toBeGreaterThanOrEqual(0) + expect(markerIdx).toBeGreaterThan(headIdx) + expect(hintIdx).toBeGreaterThan(markerIdx) + expect(tailIdx).toBeGreaterThan(hintIdx) + }) +}) + +describe("TruncateCore maxLines=1 edge", () => { + test("middle direction with a 1-line budget keeps exactly one line (the tail), never two", () => { + const text = ["first error line", "noise", "noise", "final verdict line"].join("\n") + const lines = text.split("\n") + const totalBytes = Buffer.byteLength(text, "utf-8") + const p = TruncateCore.preview(lines, totalBytes, { + maxLines: 1, + maxBytes: TruncateCore.MAX_BYTES, + direction: "middle", + headRatio: TruncateCore.DEFAULT_HEAD_RATIO, + }) + const kept = [p.head, p.tail].filter((part) => part.length > 0).join("\n").split("\n") + expect(kept).toHaveLength(1) + // Tail-weighted design: the surviving line is the last one. + expect(p.tail).toBe("final verdict line") + expect(p.head).toBe("") + expect(p.removed).toBe(3) + }) +}) diff --git a/packages/opencode/test/tool/truncation.test.ts b/packages/opencode/test/tool/truncation.test.ts index 450559c7d2..497f5b3bb5 100644 --- a/packages/opencode/test/tool/truncation.test.ts +++ b/packages/opencode/test/tool/truncation.test.ts @@ -74,12 +74,31 @@ describe("Truncate", () => { }), ) - it.live("truncates from head by default", () => + // altimate_change start — default direction is "middle" (head+tail, + // tail-weighted), not pure head. Pure head truncation is still available + // via an explicit `direction: "head"` override, covered below. + it.live("truncates from the middle by default (head+tail, tail-weighted)", () => Effect.gen(function* () { const svc = yield* Truncate.Service const lines = Array.from({ length: 10 }, (_, i) => `line${i}`).join("\n") const result = yield* svc.output(lines, { maxLines: 3 }) + // 1/3 head : 2/3 tail split of a 3-line budget = 1 head line + 2 tail lines. + expect(result.truncated).toBe(true) + expect(result.content).toContain("line0") + expect(result.content).toContain("line8") + expect(result.content).toContain("line9") + expect(result.content).not.toContain("line1") + expect(result.content).not.toContain("line5") + }), + ) + + it.live("explicit direction 'head' still truncates from the head only", () => + Effect.gen(function* () { + const svc = yield* Truncate.Service + const lines = Array.from({ length: 10 }, (_, i) => `line${i}`).join("\n") + const result = yield* svc.output(lines, { maxLines: 3, direction: "head" }) + expect(result.truncated).toBe(true) expect(result.content).toContain("line0") expect(result.content).toContain("line1") @@ -88,6 +107,39 @@ describe("Truncate", () => { }), ) + it.live("default middle truncation preserves a trailing success line in a >50KB log", () => + Effect.gen(function* () { + const svc = yield* Truncate.Service + const noise = Array.from({ length: 3000 }, (_, i) => `build step ${i}: compiling module_${i}.ts`) + const successLine = "Done. PASS=42 FAIL=0" + const text = [...noise, successLine].join("\n") + expect(text.split("\n").length).toBeGreaterThan(Truncate.MAX_LINES) + expect(Buffer.byteLength(text, "utf-8")).toBeGreaterThan(Truncate.MAX_BYTES) + + const result = yield* svc.output(text) + + expect(result.truncated).toBe(true) + expect(result.content).toContain(successLine) + }), + ) + + it.live("default middle truncation preserves the first error line in a >50KB log", () => + Effect.gen(function* () { + const svc = yield* Truncate.Service + const firstError = "ERROR: schema.sql:1: syntax error near CREAT" + const noise = Array.from({ length: 3000 }, (_, i) => `build step ${i}: compiling module_${i}.ts`) + const text = [firstError, ...noise].join("\n") + expect(text.split("\n").length).toBeGreaterThan(Truncate.MAX_LINES) + expect(Buffer.byteLength(text, "utf-8")).toBeGreaterThan(Truncate.MAX_BYTES) + + const result = yield* svc.output(text) + + expect(result.truncated).toBe(true) + expect(result.content).toContain(firstError) + }), + ) + // altimate_change end + it.live("truncates from tail when direction is tail", () => Effect.gen(function* () { const svc = yield* Truncate.Service