Skip to content

perf(ci): use the prebuilt release binary for interface collection#76

Merged
guibeira merged 1 commit intomainfrom
perf/use-release-binary-for-publish
May 5, 2026
Merged

perf(ci): use the prebuilt release binary for interface collection#76
guibeira merged 1 commit intomainfrom
perf/use-release-binary-for-publish

Conversation

@guibeira
Copy link
Copy Markdown
Collaborator

@guibeira guibeira commented May 5, 2026

Summary

Collect worker interface was paying a full cold cargo run (~50-90s for skills/mcp because of 200+ deps) every publish, even though the binary-build job upstream already produced a working x86_64-unknown-linux-gnu artifact on the GitHub Release that publish depends on. The compile is the dominant cost of the publish step today — see skills/v0.1.4 (run) where Starting local workerHTTP 200 was ~2 minutes wall-clock.

Fix

Add a release-binary mode to Start local worker for interface collection for any deploy: binary worker:

  1. Download $BIN-x86_64-unknown-linux-gnu.tar.gz from $REPO_URL/releases/download/$TAG/ (same URL pattern that resolve_binary_artifacts.py:37-49 uses).
  2. Extract to /tmp/, chmod +x.
  3. Run from the worker directory (so ./config.yaml resolves the way cargo run did).

Mode-detection priority becomes: binary deploy → release-binaryiii-add (Node/Python with scripts.start) → cargo-run (Rust without binary deploy, if anyone ever adds one) → unsupported.

Why it's safe

  • _rust-binary.yml is a needs: of publish in release.yml, so when publish runs the asset is guaranteed to be on the release.
  • Dry runs skip publish entirely (release.yml:190), so we never hit release-binary without a release.
  • cargo-run stays as a defensive fallback; nothing in main currently exercises it but it costs nothing.

Expected impact

step before after
Start local worker ~5s setup ~5s setup
Worker boots 50-90s (cargo compile) ~5s (curl + extract)
Collect worker interface poll until compile finishes poll until binary registers

Net: ~50-80s saved per Rust worker publish.

Test plan

  • After merge, dispatch create-tag.yml for one fast worker (image-resize) and one heavy worker (skills or mcp); both should publish in well under a minute end-to-end.
  • Start local worker step logs show Fetching prebuilt worker: ...releases/download/... rather than cargo run.
  • Worker connects to engine, Collect worker interface finishes within ~10s of the worker download.
  • POST /publish returns 200 with the same payload shape (functions, triggers, etc.) as before.

Summary by CodeRabbit

  • Chores
    • Release workflow now includes binary deployment mode, enabling deployment using prebuilt worker packages downloaded from release assets.

The publish job's `Start local worker for interface collection` step was running `cargo run --bin $WORKER --`, paying a full cold cargo compile (200+ deps for skills/mcp) before the worker could even register with the engine. That added 50-90s to every publish despite the `binary-build` job having already produced a working linux-gnu binary on the GitHub Release that publish depends on.

Add a `release-binary` mode that, for any `deploy: binary` worker, downloads `$BIN-x86_64-unknown-linux-gnu.tar.gz` from `releases/download/$TAG/`, extracts it, and runs the binary directly. Cargo compile drops out of the critical path; the Collect step now waits ~5s of curl + a fresh process boot instead of ~60s of compile.

The mode-detection now prefers `release-binary` whenever `deploy == binary` (the only path that reaches publish — dry runs short-circuit upstream, and `_rust-binary.yml` is a `needs:` of `publish` so the asset is guaranteed to exist). `cargo-run` is kept as a fallback for any future Rust worker that ships via `image` deploy or otherwise has no release asset; `iii-add` (Node/Python via `scripts.start`) is unaffected.
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 5, 2026

📝 Walkthrough

Walkthrough

The pull request extends the GitHub Actions publish workflow to support deploying and running prebuilt worker binaries during interface collection. When deploy is set to "binary", the workflow downloads a prebuilt tarball from GitHub releases, extracts it, and starts the binary instead of building from source.

Changes

Binary Deployment Path in Publish Workflow

Layer / File(s) Summary
Environment Wiring
.github/workflows/_publish-registry.yml (lines 97–110)
Extended the interface-collection step's environment to pass DEPLOY, TAG, and REPO_URL into the Python mode-selection logic.
Mode Selection Logic
.github/workflows/_publish-registry.yml (lines 119–123)
Updated the Python branching to read the DEPLOY input and map deploy == "binary" to a new release-binary run mode.
Binary Startup Handler
.github/workflows/_publish-registry.yml (lines 134–156)
Added a new release-binary case that downloads the prebuilt worker tarball from GitHub release assets, extracts it to /tmp, starts the binary, and monitors for unexpected early termination.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

  • iii-hq/workers#68: Modifies the same "Start local worker for interface collection" step to fix Rust cargo invocation.
  • iii-hq/workers#75: Adjusts the same step's cargo-run startup arguments by removing --no-stdio.
  • iii-hq/workers#67: Adds binary artifact resolution and workflow support for running downloaded binaries during publish.

Suggested reviewers

  • sergiofilhowz
  • andersonleal

Poem

🐰 A rabbit hops with binary cheer,
No builds required, the preset's here!
From GitHub's vault, the tarball flies,
Extract and start—no compromise!
Deploy with speed, deployment thrives! 🎉

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: using prebuilt release binaries instead of cargo builds for interface collection in the CI workflow, which is the core objective of this PR.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch perf/use-release-binary-for-publish

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@guibeira guibeira marked this pull request as ready for review May 5, 2026 11:14
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
.github/workflows/_publish-registry.yml (1)

139-140: 💤 Low value

Tar extraction assumes a flat tarball root — add a post-extract existence check

tar -xzf /tmp/worker-bin.tar.gz -C /tmp/ extracts into /tmp/, then the code immediately assumes the binary will be at /tmp/${bin_name}. If the binary-build job wraps the binary in a directory (e.g., tar czf ... bin/myworker), bin_path won't exist and chmod +x will fail with a confusing "No such file or directory" rather than an actionable error.

🛠️ Proposed fix: validate extraction result
              tar -xzf /tmp/worker-bin.tar.gz -C /tmp/
              bin_path="/tmp/${bin_name}"
+             if [[ ! -f "$bin_path" ]]; then
+               echo "::error::Expected binary not found at $bin_path after extracting tarball"
+               echo "Contents of /tmp after extraction:"
+               ls /tmp/
+               exit 1
+             fi
              chmod +x "$bin_path"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/_publish-registry.yml around lines 139 - 140, After
extracting /tmp/worker-bin.tar.gz, verify that the expected binary exists at
bin_path="/tmp/${bin_name}" and handle cases where the tarball has a nested
root; update the extraction step to check for the file (or search under /tmp for
a matching ${bin_name} if not found), set bin_path accordingly, and emit a clear
error and exit if the binary cannot be located before calling chmod +x. Ensure
the logic references the existing bin_path and bin_name variables so the
workflow fails with an actionable message instead of a generic "No such file or
directory".
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In @.github/workflows/_publish-registry.yml:
- Around line 139-140: After extracting /tmp/worker-bin.tar.gz, verify that the
expected binary exists at bin_path="/tmp/${bin_name}" and handle cases where the
tarball has a nested root; update the extraction step to check for the file (or
search under /tmp for a matching ${bin_name} if not found), set bin_path
accordingly, and emit a clear error and exit if the binary cannot be located
before calling chmod +x. Ensure the logic references the existing bin_path and
bin_name variables so the workflow fails with an actionable message instead of a
generic "No such file or directory".

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 5074b426-2115-4c4f-ba15-614dfff2d8dc

📥 Commits

Reviewing files that changed from the base of the PR and between 2544ccf and abc7902.

📒 Files selected for processing (1)
  • .github/workflows/_publish-registry.yml

@guibeira guibeira merged commit 6541845 into main May 5, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant