Skip to content

fix(runtime): acknowledge terminal commands before cleanup - #431

Merged
Yevanchen merged 1 commit into
mainfrom
fix/runtime-terminal-ack-latency
Jul 29, 2026
Merged

fix(runtime): acknowledge terminal commands before cleanup#431
Yevanchen merged 1 commit into
mainfrom
fix/runtime-terminal-ack-latency

Conversation

@Yevanchen

Copy link
Copy Markdown
Collaborator

Summary

  • Return the Driver's terminal command acknowledgement immediately after the command transition is durably stored.
  • Keep linked run/lease cleanup alive with Durable Object waitUntil and log cleanup failures independently.
  • Add a regression test that blocks the terminal session_run lookup and proves the acknowledgement is no longer hostage to cleanup latency.

Root cause

Production runs 01KYNP1C4XG9KC16SX8RP2843K and 01KYNP5RKZW1NSYJ9G3WDBVWD9 completed successfully, but each Driver failed 3–5 seconds later with terminal status could not be delivered. Production D1 takes roughly 41–62 ms per round trip. The command-update handler persisted the terminal command, then synchronously ran the linked run/lease cleanup before returning. That cleanup pushed the response past the Driver's 250 ms per-attempt deadline. The Driver retried, then self-failed, so the next message created/resumed another runtime and erased the warm-retention benefit from #400.

The Chrome reproduction on the same production Worker confirmed the mechanism: 11.96 s then 10.90 s to reply, but the two runs used different Driver IDs and both Drivers subsequently failed with the same delivery error.

Verification

  • Red/green regression: just test-file apps/api/tests/driver-command-terminal-ack.test.ts
  • API typecheck: vp run --filter @mosoo/api tc
  • Related runtime/terminal suite: 33 passed
  • Full API suite: 1,063 passed
  • just commit-check
  • just check: format, docs, lint, and workspace typecheck passed; tests reached 1,078 passed / 42 skipped / 1 unrelated existing failure in the pinned Driver submodule (acp-file-system.test.ts compares macOS /var/... with canonical /private/var/...). The exact Driver test fails the same way when rerun alone.

Generated and persistence surfaces

  • Generated files: none
  • GraphQL codegen: N/A
  • DB schema or migrations: none
  • Lockfile changes: none

Refs #387.

@Yevanchen
Yevanchen merged commit 6ee1b70 into main Jul 29, 2026
9 checks passed
@Yevanchen
Yevanchen deleted the fix/runtime-terminal-ack-latency branch July 29, 2026 07:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant