fix(orchestrator): Preserve claude/codex post-interrupt recovery state

[mwolson]

Summary

• Preserve the first Claude user turn after Stop when a stale background task
  notification arrives during session resume.
• Terminalize Codex command projections when interrupted or failed turns omit
  the command completion event.
• Interrupt active Codex descendant turns and terminate tracked background
  terminals against each owning native thread during root Stop.
• Delay interrupted or failed Codex root terminal events until descendant
  provider turns and threads have reached durable terminal state.
• Suppress redundant empty or identical Codex final-answer items in the same
  native turn without hiding a sole empty response or distinct later answer.
• Terminalize run-owned subagent projections, linked child nodes, and linked
  child timeline items when a stopped root turn leaves them open.
• Add deterministic regressions for the observed Claude and Codex frame orders.

Stack

This draft targets t3code/codex-turn-mapping and is stacked on #4193, the same
post-merge fixture-fix base used by #4218. This branch contains one unique
commit above that stack base.

Problem and Fix

Problem and Why it Happened	Fix
Claude can emit a background task_notification result between the first post-Stop prompt and its real response. The adapter treated that result as the active root result, completing the recovery run with no assistant output.	Detect task-notification-origin results and ignore them for normal user turns before fallback text or terminalization. Provider continuation turns still consume the same result shape while draining buffered wakes.
Codex app-server can omit item/completed for an in-flight command, accept turn/interrupt without later sending turn/completed, and keep descendant native turns running after the root stops. Its interrupt response also does not contain provider-owned background terminals. The projected run could look interrupted while a child shell survived Stop and later emitted a final notification. A root terminal arriving before its child provider terminal events also stopped run ingestion and stranded the durable child provider turn and thread as running.	Track command snapshots and process IDs per native turn. Interrupt the active root lineage, terminate each tracked terminal against its owning native thread immediately and after settlement, bound the native completion wait, and terminalize projections before turn.terminal. Defer non-completed root terminal events until active descendant turns have terminalized, including timeout and interrupt-request failure paths. Verify ambiguous false results against the paginated terminal list.
Codex app-server can emit multiple final-answer items in one native turn, including a trailing empty item or a second streamed item with text identical to the first. The UI selected the empty item as terminal or rendered the same recovery response twice.	Track completed final texts per native turn, buffer later final streams, and discard later empty or identical completions. Preserve sole empty answers, distinct later finals, commentary, unknown-phase compatibility, buffered non-empty text, and subagent results.
Claude and Codex can stop the root run while its subagent entity, node, turn item, or linked child timeline remains running.	Cascade interrupted, failed, or cancelled status to open run-owned subagent records, linked child nodes, and projected child turn items before root finalization, while preserving child linkage and partial result text.

Validation

• vp check: pass with pre-existing warnings only
• vp run typecheck: pass
• Codex adapter tests: 40 passed, including descendant-turn interruption,
  owning-thread terminal containment, bounded interrupt fallback, late native
  completion rejection, redundant empty and identical-final suppression,
  overlapping streams, sole-empty preservation, unknown phases, zero-length
  deltas, subagent result safety, and child provider terminal ordering after
  native completion, timeout, interrupt-request failure, and a descendant that
  starts after the initial Stop snapshot
• Targeted Codex turn_interrupt_mid_tool replay and fixture-loading checks:
  pass
• Parent interrupt-safety fixtures: pass, including simulated Darwin cleanup,
  queued scheduler, projection, process ownership, and driver timeout gates
• claude-interrupt-direct-stop: pass with one interrupted run and two distinct
  completed recovery runs
• Isolated paired browser verification: both Claude and Codex recovery messages
  rendered as distinct assistant responses with no stuck Working state
• Packaged AppImage Claude race: an empty task-notification result arrived
  during recovery 1; both recovery runs completed with their expected markers
• Packaged AppImage foreground-subagent probes: reproduced the stale parent row
  in Claude and Codex and the stale Claude child root node
• Packaged AppImage Codex recovery probe: confirmed containment and recovery,
  then reproduced two identical streamed final answers in one native turn
• Manual packaged Codex child-Stop probe: reproduced the native child command
  surviving root Stop and emitting a late final notification; the focused
  descendant-turn replay now covers the exact native thread and process boundary
• Final packaged Codex descendant-Stop probe: pass. Root Stop interrupted the
  child native turn, terminated its tracked foreground process, kept the child
  command and parent subagent interrupted, and completed one same-thread
  recovery response. No forbidden child output appeared through the original
  command deadline or the additional late-event polling window.
• Follow-up manual Codex Stop probe: process containment and visible recovery
  passed, but the root terminal reached the run stream before the child
  provider terminal events. The durable child provider turn remained running
  and its provider thread remained active, which the new ordering regressions
  reproduce.
• Shared run-execution tests: 18 passed, including interrupt cascade,
  failed/cancelled mapping, supersede protection, linked-child cleanup,
  linked-child timeline cleanup, unreferenced-child exclusion, suppressed
  streaming exclusion, partial result preservation, and same-attempt and
  cross-attempt late-event rejection
• Strict codex-interrupt-direct-stop on Codex 0.144.6: pass with run statuses
  [interrupted, completed, completed], no running command projection, and the
  foreground workload PID absent after Stop

Defensive Fixes

If Codex returns terminated: false, query the paginated background-terminal
list before deciding whether containment succeeded. Treat an absent process as
already contained, fail if it remains listed, attempt every tracked process ID,
and always clear interrupt bookkeeping.

If Codex never emits turn/completed after accepting an interrupt, locally
terminalize after the same 10-second ceiling used by sibling adapters. A shared
finalization permit prevents the timeout and a racing native completion from
publishing duplicate terminal events; late native events cannot reopen the
settled turn.

When root Stop has active native descendant turns, install completion waiters
for the full active lineage before sending interrupts. Treat a target that
settled during waiter setup as already complete. Run the first terminal sweep
concurrently with the single completion deadline, then sweep again after
settlement so commands observed during the interrupt race are also contained. If
a marked native target reports completed during that race, preserve the
user-requested interrupted projection.

For non-completed Codex roots, retain the root terminal event while any native
descendant turn remains active. Descendant finalization publishes its provider
turn and provider-thread terminal updates first, then releases the root event.
Interrupt RPC failures locally terminalize remaining targets before release,
and the existing finalization permit serializes native completion, timeout, and
failure cleanup.

Before Stop returns, rescan the interrupted native lineage and interrupt plus
terminalize descendants that started after the initial snapshot. Native child
starts that arrive after an ancestor has terminalized non-completed are
interrupted without creating a new running provider projection.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: e857dabf-8715-48df-893e-e3657ab12aaf

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit ae99f40. Configure here.}

[cursor]

Interrupt awaits completion indefinitely

Medium Severity

interruptTurn now blocks on Deferred.await(turnCompleted) with no timeout after turn/interrupt. If Codex never emits turn/completed, the provider-turn.interrupt outbox effect never finishes, leaving interrupt bookkeeping and the worker fiber stuck even though the user already saw an interrupted projection.

 

^{Reviewed by Cursor Bugbot for commit ae99f40. Configure here.}

[macroscopeapp]

🟡 Medium turn_interrupt_mid_tool/codex_output.ts:58

assert.isAbove(terminateIndex, completedIndex) passes even when completedIndex is -1, so this fixture can accept a thread/backgroundTerminals/terminate request that was sent without a corresponding turn/completed frame. That defeats the regression check — termination should be verified to occur only after completion is confirmed. Add an explicit nonnegative assertion for completedIndex before the comparison.

  const completedIndex = transcript.entries.findIndex(
    (entry) => entry.type === "emit_inbound" && protocolMethod(entry.frame) === "turn/completed",
  );
+ assert.isAtLeast(
+   completedIndex,
+   0,
+   "Codex interrupt fixture must record turn completion before termination",
+ );
  const terminateIndex = transcript.entries.findIndex(
    (entry) =>
      entry.type === "expect_outbound" &&
      protocolMethod(entry.frame) === "thread/backgroundTerminals/terminate",
  );

🤖 Copy this AI Prompt to have your agent fix this:

In file @apps/server/src/orchestration-v2/testkit/fixtures/turn_interrupt_mid_tool/codex_output.ts around lines 58-66:

`assert.isAbove(terminateIndex, completedIndex)` passes even when `completedIndex` is `-1`, so this fixture can accept a `thread/backgroundTerminals/terminate` request that was sent without a corresponding `turn/completed` frame. That defeats the regression check — termination should be verified to occur only after completion is confirmed. Add an explicit nonnegative assertion for `completedIndex` before the comparison.