Skip to content

fix(docs): never queue the GPU pass for a busy runner (#174) - #175

Merged
ocots merged 1 commit into
mainfrom
fix/174-docs-gpu-skip-when-busy
Nov 14, 2026
Merged

ocots merged 1 commit into
mainfrom
fix/174-docs-gpu-skip-when-busy

Conversation

@ocots

@ocots ocots commented Nov 14, 2026 •

Copy link
Copy Markdown
Member

Written by Claude (Claude Code) at the request of ocots.

Closes #174.

Why

A documentation caller with a GPU pass (OptimalControl.jl, CTSolvers.jl) has a concurrency group per ref that queues runs on main (ordering of the deployments, no cancellation of a Julia process on a self-hosted runner). The group is held by the whole run: a build-gpu queued for a busy runner made the next runs wait, CPU build included, with no time limit of its own. On 2026-11-14, OptimalControl.jl's dev was not updated for three merges until the stuck run was force-cancelled (details in #174).

What

  • select-gpu-runner.yml: new input when_busy:

    • wait (default): today's behaviour, unchanged for ci-gpu.yml: only offline counts, a busy runner is waited for;
    • skip: a runner counts only if it is online and idle; kkt, else occidata, else the new output skip is 'true' and runner is empty.

    New input ctactions_ref (default main), as in guard.yml, for the script checkout. An unknown state (no CT_RUNNERS_* secrets, API error) still gives the fallback, never a skip (principle 7).

  • The decision moves to a script (principles 2 and 3): scripts/select-gpu/pick.sh, tested by scripts/select-gpu/test.sh (19 cases: both modes, several runners per label, another GPU runner ignored, unknown state, a bad WHEN_BUSY), run by the new select-gpu-tests.yml. select-gpu-runner.yml leaves the "only linted" exceptions.

  • documentation.yml: select-gpu with when_busy: skip; build-gpu runs only if skip != 'true'. When skipped, the site published by build stays, as when the pass fails.

  • Callers: no change. Their concurrency group stays; it no longer waits on a pass queued for a busy runner.

  • Docs: workflows/select-gpu-runner.md (inputs, outputs, rules), workflows/documentation.md (choosing the runner, ordering, how to force-cancel a stuck queued run), design/documentation.md (the choice, the options set aside, two known limits, timeline), guides/documentation.md, principles.md, README.md.

Options set aside

  • cancel-in-progress: true on main in the callers: cancels a running GPU pass, the orphan-process case.
  • Separate groups for build and build-gpu with a "still the head of main?" check before the GPU deployment: the deployment happens inside each package's docs/make.jl, in the same step as the build, so CTActions cannot place the check.

Known limits

  • A runner idle when select-gpu reads it can be taken before build-gpu starts; the pass then waits for that job, bounded by its timeout. The manual way out is documented.
  • With both runners busy for long periods, the GPU pass is skipped as often.

Checked

  • bash scripts/select-gpu/test.sh: 19/19; shellcheck scripts/select-gpu/*.sh: clean; actionlint on every workflow: clean.
  • End to end on the live state of the organization's runners (same gh api … | jq -sc . as the workflow): kkt offline, occidata online and busy →
    • wait: runner=occidata, skip=false (unchanged for the tests);
    • skip: runner=, skip=true (no pass queued, today's case);
    • unknown state: runner=occidata, skip=false.
  • In GitHub Actions: select-gpu-tests.yml and lint-workflows.yml run on this PR. The reusable workflows call each other at @main (uses: takes no expression), so documentation.yml with when_busy: skip is first exercised by the next docs run of OptimalControl.jl or CTSolvers.jl on main after the merge.

🤖 Generated with Claude Code

The concurrency group of a documentation caller is held by the whole run.
A build-gpu queued for a busy runner made the runs of the next pushes to
main wait, CPU build included (OptimalControl.jl, 2026-11-14: three merges
not published until the run was force-cancelled).

- select-gpu-runner.yml: input when_busy, 'wait' (default, unchanged for
  ci-gpu.yml) or 'skip' (only an online and idle runner counts, else the
  new output skip is 'true'); input ctactions_ref.
- The decision moves to scripts/select-gpu/pick.sh, tested by
  scripts/select-gpu/test.sh (19 cases) in select-gpu-tests.yml; an
  unknown state still means the fallback, never a skip.
- documentation.yml: select-gpu with when_busy 'skip'; build-gpu runs
  only if skip is not 'true', so the site of the CPU build stays.
- Docs: select-gpu-runner, documentation (choosing the runner, ordering,
  manual force-cancel), design (choice, set-aside options, known limits,
  timeline), guide, principles, README.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ocots
ocots marked this pull request as ready for review November 14, 2026 18:46
@ocots
ocots merged commit f1d6b68 into main Nov 14, 2026
3 checks passed
@ocots
ocots deleted the fix/174-docs-gpu-skip-when-busy branch November 14, 2026 18:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

documentation.yml: a queued GPU pass blocks the publication of the next commits on main

1 participant