Skip to content

feat: auto-resume on provider usage limit reset + /code-review command - #46

Open
gr3enarr0w wants to merge 4 commits into
QuintinShaw:mainfrom
gr3enarr0w:feat/auto-resume-and-code-review
Open

feat: auto-resume on provider usage limit reset + /code-review command#46
gr3enarr0w wants to merge 4 commits into
QuintinShaw:mainfrom
gr3enarr0w:feat/auto-resume-and-code-review

Conversation

@gr3enarr0w

Copy link
Copy Markdown
Contributor

Summary

Closes #27
Closes #32


feat(workflow-manager): auto-resume after usage-limit pause (#27)

When a workflow hits a provider quota/usage limit, it's now checkpointed as paused (already in place via #26). This PR adds the automatic resume layer on top.

How it works

  • ExecOptions.autoResume (default true) — opt out per-execution by passing autoResume: false
  • When executeRun catches a PROVIDER_USAGE_LIMIT error, it calls scheduleAutoResume(runId, resetHint)
  • parseResetHintMs() extracts hours/minutes from the verbatim resetHint string (e.g. "~3h", "45 min"), falls back to 60 minutes
  • Exponential backoff: each re-pause doubles the delay; MAX_AUTO_RESUME_ATTEMPTS = 5 then gives up
  • recoverStaleRuns() now re-arms timers on cold start for any persisted paused + pauseReason === "usage_limit" runs
  • stop() and deleteRun() cancel pending timers so stopped runs don't ghost-resume
  • Emits "autoResuming" event before each attempt for observability

feat(code-review): /code-review built-in command (#32)

New src/code-review.ts implementing generateCodeReviewWorkflow(), registered as /code-review.

Phases

Phase 1 — Find (7 parallel agents)

Agent Tier Focus
A — line-by-line scan medium Inverted conditions, off-by-one, nil deref, wrong variable, swallowed error
B — removed-behavior audit medium Every deleted line: invariant named, where re-established
C — cross-file tracer medium Changed functions → grep callers → call site breakage
D — reuse small New code duplicating existing helpers
E — simplification small Redundant state, copy-paste variation, dead code
F — efficiency small Redundant I/O, sequential work that could parallel
G — altitude big Fix at wrong abstraction level

Phase 2 — Verify — parallel verifier per candidate, drops REFUTED

Phase 3 — Report — big-tier synthesis, ranked A/B/C > D/E/F > G, capped at 10

Input modes

/code-review                  → git diff HEAD
/code-review HEAD~3..HEAD     → git range
/code-review src/foo.ts       → specific file
/code-review 42               → gh pr diff 42

@QuintinShaw

Copy link
Copy Markdown
Owner

Thanks — the auto-resume work is the valuable part here, but this PR is doing three separate things and I can't take it as one:

Could you rebase this down to just auto-resume + /code-review, drop the isolation files (they'll come from #43), and add coverage for parseResetHintMs, the backoff/cap in scheduleAutoResume, and the recoverStaleRuns re-arm? Happy to merge once it's scoped to its own change.

Clark Everson and others added 2 commits June 27, 2026 10:55
feat(workflow-manager): schedule automatic resume after usage-limit pause (QuintinShaw#27)
- ExecOptions.autoResume (default true) controls per-run opt-out
- Parses resetHint for h/min patterns; falls back to 60 min
- Exponential backoff per runId, capped at 5 attempts
- Re-arms timer on cold start in recoverStaleRuns() for persisted paused runs
- cancelAutoResume() called in stop() and deleteRun() to prevent ghost resumes

feat(code-review): add /code-review built-in workflow command (QuintinShaw#32)
- New src/code-review.ts: generateCodeReviewWorkflow()
- Phase 1: 7 parallel finder agents (A-G) with tier routing:
    A/B/C (correctness) → medium, D/E/F (cleanup) → small, G (altitude) → big
- Phase 2: parallel verify pass per candidate, drops REFUTED findings
- Phase 3: big-tier synthesis, ranked by angle priority, capped at 10
- Registered in builtin-commands.ts with 4 input modes:
    /code-review           → git diff HEAD
    /code-review HEAD~3..  → git range
    /code-review src/foo   → specific file
    /code-review 42        → gh pr diff 42
- Exported from index.ts as generateCodeReviewWorkflow

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- workflow-manager.test.ts: unit-test parseResetHintMs (2h / 30min /
  1h30min / garbage fallback), scheduleAutoResume exponential backoff
  + MAX_AUTO_RESUME_ATTEMPTS (5) cap, and recoverStaleRuns re-arming
  timers for usage-limit paused runs
- builtin-commands.test.ts: update count 4→5 and sorted names to
  include "code-review"; fix idempotent and missing-commands assertions
- builtin-workflows.test.ts: add generateCodeReviewWorkflow test
  asserting meta.name, Find/Verify/Report phases, parallel + candidateSchema
- src/index.ts: move code-review.js export to correct alphabetical
  position (after builtin-commands.js) to satisfy biome sort check

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@gr3enarr0w
gr3enarr0w force-pushed the feat/auto-resume-and-code-review branch from 1aa292b to 7fb49fc Compare June 27, 2026 15:04
@QuintinShaw

Copy link
Copy Markdown
Owner

Thanks for the quick turnaround — the split is clean (no more #43 overlap, rebases onto main with zero conflicts) and the new tests are real. Two things before merge, both in the new code:

  1. stop() can't cancel an auto-resume timer that recoverStaleRuns() arms on cold start. The if (!managed) return false guard returns before cancelAutoResume() runs, and recovered runs aren't in this.runs — so a run the user explicitly stops can still silently auto-resume and re-spend quota. deleteRun() already cancels unconditionally; stop() should mirror it (move the two cancel lines above the guard).

  2. The /code-review diff-source runs through exec() (/bin/sh -c) with the arg interpolated raw in the .. and filename branches: git diff ${input} / git diff HEAD -- ${input}. That's a shell-injection footgun. worktree.ts already uses promisify(execFile) + array args — please follow that here.

A regression test asserting both stop() and deleteRun() call cancelAutoResume would pin #1 (that gap is exactly why it slipped through). Optional: clamp the backoff with Math.max(delay, 60_000) so a hint that parses to ~0 doesn't fire five instant retries.

…with execFile

Move cancelAutoResume() and autoResumeAttempts.delete() above the !managed guard
in stop() so timers armed by recoverStaleRuns() for persistence-only runs are
always cancelled, even when the run has no in-memory ManagedRun.

Replace promisify(exec) with promisify(execFile) + array args in the /code-review
diff-source invocation to eliminate the shell-injection vector from user-supplied
branch/range/path inputs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@gr3enarr0w

gr3enarr0w commented Jun 28, 2026

Copy link
Copy Markdown
Contributor Author

PR Reviewer Guide 🔍

(Review updated until commit 2347f1a)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

27 - Partially compliant

Compliant requirements:

  • ExecOptions.autoResume defaults to true with per-execution opt-out
  • executeRun calls scheduleAutoResume on PROVIDER_USAGE_LIMIT
  • parseResetHintMs extracts hours/minutes and falls back to 60 minutes
  • Exponential backoff doubles the delay each re-pause
  • MAX_AUTO_RESUME_ATTEMPTS = 5
  • recoverStaleRuns re-arms timers on cold start
  • stop() and deleteRun() cancel pending timers
  • "autoResuming" event emitted before each attempt

Non-compliant requirements:

  • The backoff attempt counter is held only in memory, so the cap resets on process restart and a run can exceed the intended maximum attempts across restarts

Requires further human verification:

  • None

32 - Partially compliant

Compliant requirements:

  • src/code-review.ts created with generateCodeReviewWorkflow()
  • /code-review registered alongside existing commands
  • Input modes for no args, git range, file path, and PR number are implemented
  • Model tier routing matches the spec
  • 7 parallel finder agents with structured candidate output
  • Candidates are deduplicated before verify
  • Ranking correctness (A/B/C) before cleanup (D/E/F) before altitude (G) with cap at 10
  • Synthesis produces JSON findings and a markdown report

Non-compliant requirements:

  • README.md documentation for /code-review is not updated
  • Verify pass does not use the verify() stdlib with reviewers: 1, threshold: 0.5; it uses manual agent() calls
  • Verifier agents are not given the relevant file(s) context beyond the diff

Requires further human verification:

  • Confirm whether the verify() stdlib exists in this codebase; if it does, the manual agent implementation should be replaced
⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Non-persistent attempt cap

autoResumeAttempts is stored only in a private Map, so the exponential-backoff attempt counter resets to 0 on process restart. A run that has already been auto-resumed several times and re-paused can receive another full set of attempts after a cold start, breaking the MAX_AUTO_RESUME_ATTEMPTS cap and causing repeated resume attempts against an exhausted budget.

private autoResumeAttempts = new Map<string, number>();

autoResumeAttempts was in-memory only. On cold start, recoverStaleRuns()
re-armed timers with the count reset to 0, so a run that hit the 5-attempt
cap could receive another 5 attempts after restart.

Fix: add autoResumeAttempts to PersistedRunState (optional field, zero-safe
for existing records). recoverStaleRuns() restores the count before
scheduling. persistRun() writes the current count on every usage-limit pause.
The timer callback persists the incremented count before calling resume() so
a crash mid-attempt doesn't reset the cap.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@gr3enarr0w

Copy link
Copy Markdown
Contributor Author

Persistent review updated to latest commit 2347f1a

@QuintinShaw

Copy link
Copy Markdown
Owner

This bundles two unrelated features with very different risk profiles, and they're blocking each other: /code-review is close to mergeable, while auto-resume needs another iteration. Please split it — /code-review can land quickly.

Auto-resume (#27), the things that need rework:

  1. When the timer fires, resume() returning false (lease held, script missing) is silently swallowed: the attempt is consumed, autoResuming was already emitted, no timer is rescheduled — the run stays paused forever. Reschedule on failure or don't consume the attempt.
  2. Cold-start re-arm computes the full delay from "now" instead of the remaining time from persisted updatedAt. Timers are unref()'d, so anyone who restarts pi more often than the hint interval never auto-resumes — Auto-resume a workflow once a provider usage limit resets #27 explicitly asked for a timer that fires when the reset window elapses.
  3. autoResume: false isn't persisted: recoverStaleRuns() re-arms unconditionally and a re-pause after manual resume() gets the default true. Same for stop() — the persisted pauseReason survives, so the next restart re-arms the ghost timer.
  4. Please add a floor on the delay (a hint parsing to ~0 can fire before persistRun writes the paused state — the callback then sees a non-paused run and drops tracking), and a fake-timer test that actually fires the callback end-to-end; the current backoff test patches setTimeout so the fire path is never executed.

/code-review (#32), minor: raise execFileAsync's maxBuffer (default 1 MB — gh pr diff on a big PR throws) and cap/truncate the diff fed into the 7 finder prompts; the issue's file list includes a README update; and either use the existing verify() stdlib for the verify phase per the issue spec or note why the hand-rolled verifier is preferable.

@QuintinShaw

Copy link
Copy Markdown
Owner

Hey @gr3enarr0w — no rush at all, just checking in since it's been a couple weeks. Are you planning to pick up the review notes here? The /code-review half is close to mergeable; the auto-resume half is where the rework is (the reschedule/persistence points above). If you're swamped, I'm happy to take it forward from here and keep you credited — just say which you'd prefer.

QuintinShaw added a commit that referenced this pull request Jul 11, 2026
Extracts the /code-review half of #46 onto current main, leaving the
auto-resume half (run-persistence.ts / workflow-manager.ts) for a later PR.

- src/code-review.ts: generateCodeReviewWorkflow() — 7 parallel finder
  agents (correctness x3, cleanup x3, altitude x1) -> per-candidate verify
  pass -> ranked markdown report.
- src/builtin-commands.ts: register /code-review alongside the other
  bundled commands; gather the diff via execFile (no shell) so
  branch/range/path input can't break out into a shell command.
- src/index.ts: export generateCodeReviewWorkflow + MAX_DIFF_CHARS.
- tests/builtin-commands.test.ts, tests/builtin-workflows.test.ts:
  /code-review coverage.
- README.md: document /code-review's input modes and behavior.

Addresses the maintainer's earlier review nits on this half:
- maxBuffer on the diff-gathering exec raised from Node's 1MB default to
  64MB, with a clear notify (not a raw ERR_CHILD_PROCESS_STDOUT_MAXBUFFER)
  when even that's exceeded.
- Diffs over 200k chars (MAX_DIFF_CHARS) are truncated with a visible
  notice at both the command layer and, defensively, inside the generated
  workflow script itself (which also stamps diffTruncated on the result).
- Verify phase deliberately keeps its hand-rolled 3-way CONFIRMED /
  PLAUSIBLE / REFUTED agent() call instead of the verify() stdlib, which
  only returns a boolean and would collapse that signal for no behavioral
  gain (only REFUTED is filtered) — documented inline in code-review.ts.

Co-authored-by: Clark Everson <clark@everson.dev>
@QuintinShaw

Copy link
Copy Markdown
Owner

Heads up — I've split the /code-review half of this PR out and landed it separately in #62 (merged), crediting you as co-author. It's the same command with a few maintainer follow-ups on top: diff-capture maxBuffer raised to 64MB with a clear over-limit message, a 200k-char truncation guard, and README docs. Closed #32.

This PR stays open for the auto-resume half, which still needs the rework from my earlier review (the resume()→false swallow, cold-start re-arm using the full delay vs remaining-from-updatedAt, the autoResume:false+stop() ghost re-arm, a delay floor, and the untested timer-fire path). No rush — happy to take that forward too if you'd rather hand it off, otherwise it's all yours. Thanks again for both features.

@QuintinShaw

Copy link
Copy Markdown
Owner

Hi @gr3enarr0w — an update on this one. The /code-review half shipped separately a while back as #62. For the auto-resume half: since it had been open a few weeks and auto-resume-on-usage-limit is something I needed for my own long runs, I implemented it in #78 (now merged, closing #27).

I want to be straight that your PR directly informed that work — reviewing it is what surfaced the exact edge cases the rewrite had to get right (the swallowed resume()→false, cold-start using remaining vs full delay, the un-persisted opt-out, the missing delay floor, the untested timer path). So thank you — it wasn't wasted even though the final code took a different route (a standalone scheduler, to stay decoupled from manager internals).

I think this can be closed now that both halves are covered (#62 + #78), but I'll leave that to you in case you'd rather repurpose it. Really appreciate the contributions — hope to see more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: add /code-review built-in workflow command Auto-resume a workflow once a provider usage limit resets

2 participants