fix(background-agent): force-deliver parent wakes held past a stale tool-call ceiling - #7117
Open
andrei-kiparuk wants to merge 1 commit into
Conversation
…ool-call ceiling The stale tool-call deferral holds reply-required parent wakes while the latest assistant turn looks mid-flight, admitting them as noReply deposits and waiting for the idle/consumption machinery. When the blocking tool part can never resolve (e.g. a bash tool left at status "running" after its stream died), the hold never releases: observed in the wild as a wake held for 13+ minutes with a retry log entry every second while every background task had already completed, leaving the parent session idle indefinitely. Cap the hold at PARENT_WAKE_STALE_TOOL_BLOCK_MAX_HOLD_MS (60s). After the ceiling the wake is force-delivered; toolCallDeferralStartedAt stays set so the pre-dispatch confirmation re-check converges on the same decision. Without a configured ceiling the historical unbounded hold is preserved. Also hide admit-only/noReply wake deposits from the TUI by marking their text part synthetic, so deferred notifications no longer render as raw system-reminder user messages during the hold window.
Contributor
|
All contributors have signed the CLA. Thank you! ✅ |
Author
|
I have read the CLA Document and I hereby sign the CLA |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Background-task completion notifications can be starved forever by the stale tool-call deferral, leaving a fully idle parent session that never resumes work.
When the latest assistant turn in the parent session ends with an unresolved tool part,
getParentWakeSessionHistoryDeferralDecisionholds the wake (admitting it as a noReply deposit at most) and waits for "the idle/consumption machinery" to resume. That is correct while the turn is genuinely mid-flight, but there is no upper bound on the hold. If the blocking tool part can never resolve — e.g. abashtool left atstate.status: "running"after its stream died mid-tool — the state is permanent:latestAssistantTurnBlocksInternalPrompttrue forever,deferReplyWakeWhileUnsafeand reschedules,Observed in production
From a single real session (
ses_fda080e47…):209 hold events across two sessions on one day. The session history confirmed the blocker: the latest assistant message had
finish: null, 0 tokens, and exactly one part — abashtool stuck at"running"(agh run watch) from hours earlier. Every background task had long since completed; the parent just sat idle.Fix
Cap the hold with
PARENT_WAKE_STALE_TOOL_BLOCK_MAX_HOLD_MS(60s), wired through as an optionalstaleToolBlockMaxHoldMsoption:ses_14a3ab27bffefix.defer: false). Sixty seconds of zero activity across message time, part time, and tool-state timestamps overwhelmingly indicates a dead turn rather than a quiet one, and unbounded starvation is worse than the residual fork risk of dispatching into an idle-status session.toolCallDeferralStartedAtis deliberately not cleared on the escape path so the pre-dispatch confirmation re-check reaches the same decision instead of re-arming the deferral.undefined) preserves the historical unbounded hold for existing callers.Also marks the admit-only/noReply wake text part as
synthetic: true, so deferred notifications no longer render as raw<system-reminder>user messages in the TUI while they sit deposited during the hold window.Test updates
Two tests in
task-completion-cleanup.test.tspinned the unbounded-hold behavior at a 60s-stale zombie tool block ("admitted as noReply with reply liveness retained"). That exact scenario is the starvation case this PR fixes, so they now assert the post-ceiling contract: force-delivered reply dispatch, pending wake consumed. Sub-ceiling admission behavior remains covered by the untouched neighboring tests plus three new cases inparent-wake-history-deferral.test.ts(held under ceiling → still held; past ceiling → delivered, including confirmation re-check convergence; no ceiling configured → historical hold preserved).Testing
bun test packages/omo-opencode/src/features/background-agent/→ 746 pass, 0 failtsgo --noEmit -p packages/omo-opencode/tsconfig.json→ cleanSummary by cubic
Stops indefinite starvation of background-task completion notifications by capping the stale tool-call deferral and force-delivering the parent wake after 60s. Previously we held reply-required wakes indefinitely when the last assistant turn had a zombie tool; now we hold under the ceiling and dispatch after it. Deferred noReply deposits are marked synthetic to avoid TUI noise.
ParentWakeNotifierandParentWakeSessionInspector;BackgroundManagersets it to 60_000 ms. Omitting it preserves the legacy unbounded hold for other callers.Written for commit c92f51a. Summary will update on new commits.