feat: collect worker interface (request/response format) on publish#67
Merged
feat: collect worker interface (request/response format) on publish#67
Conversation
📝 WalkthroughWalkthroughThis PR introduces a refactored publish workflow for registry workers. Three new Python scripts are added: ChangesUnified Registry Publish Workflow
Sequence DiagramsequenceDiagram
participant GHA as GitHub Actions
participant CLI as III CLI
participant Engine as III Engine
participant Worker as Local Worker
participant Scripts as Collection Scripts
participant Pub as Publish Registry
GHA->>GHA: Install III CLI
GHA->>Engine: Start III Engine (background)
Engine-->>GHA: Ready (readiness probe)
GHA->>Worker: Start Worker (via iii worker add/cargo run)
Worker->>Engine: Register
GHA->>Scripts: Run collect_worker_interface.py
Scripts->>CLI: iii trigger engine::workers::list
CLI->>Engine: Query workers
Engine-->>CLI: Workers JSON
CLI-->>Scripts: Worker data
Scripts->>CLI: iii trigger engine::functions::list
CLI->>Engine: Query functions
Engine-->>CLI: Functions JSON
CLI-->>Scripts: Functions data
Scripts->>CLI: iii trigger engine::triggers::list (optional)
CLI->>Engine: Query triggers
Engine-->>CLI: Triggers JSON
CLI-->>Scripts: Triggers data
Scripts->>Scripts: Normalize interface (workers_json, functions, triggers)
Scripts-->>GHA: worker-interface.json
GHA->>Scripts: Run resolve_binary_artifacts.py
Scripts->>Scripts: Fetch .sha256 files from GitHub releases
Scripts-->>GHA: binaries.json
GHA->>Scripts: Run build_publish_payload.py
Scripts->>Scripts: Load worker YAML, README, config
Scripts->>Scripts: Build payload with interface + binaries
Scripts-->>GHA: publish-payload.json
GHA->>Pub: POST payload
Pub-->>GHA: Published
GHA->>GHA: Stop worker & engine (cleanup)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Review rate limit: 0/1 reviews remaining, refill in 60 minutes.Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/scripts/build_publish_payload.py:
- Around line 10-17: The normalize_dependencies function returns raw lists
unchanged, which can leave string entries like "foo" instead of dicts; update
the list branch in normalize_dependencies to map each item to a dict of the form
{"name": ..., "version": ...}: if an item is a str, convert to {"name": item,
"version": ""} (or None if you prefer), if it's already a dict keep it as-is (or
validate it has "name"/"version"), and if an item is another type raise a
ValueError; this ensures normalize_dependencies always returns list[dict[str,
Any]].
- Around line 141-153: The loop that builds the functions list should fail fast
when a worker_function_ids entry has no corresponding details in functions_by_id
instead of appending empty placeholders; update the code around the for loop
that iterates worker_function_ids (the logic that reads details =
functions_by_id.get(function_id, {}), derive_registry_function_name, and
functions.append) to detect missing details (e.g., if function_id not in
functions_by_id) and raise an exception or exit with a clear error including the
missing function_id so the publish stops rather than emitting blanks.
In @.github/scripts/collect_worker_interface.py:
- Around line 24-55: Add a timeout to the subprocess.run() call in run_iii()
(e.g., timeout=some_seconds) so iii calls cannot hang indefinitely, and update
collect_triggers() to also catch subprocess.TimeoutExpired (in addition to
subprocess.CalledProcessError and json.JSONDecodeError) and treat it as a
soft-fail (returning None or the same behavior used for other caught
exceptions). Locate run_iii and collect_triggers in this file to add the timeout
argument to subprocess.run(...) and include subprocess.TimeoutExpired in the
except clause in collect_triggers().
In @.github/workflows/_publish-registry.yml:
- Around line 55-59: The workflow step named "Install iii CLI" currently
downloads the floating installer at install.iii.dev/iii/main/install.sh; update
it to pin to a specific release and invoke the installer with the version flag
(e.g., pass "-s -- v0.11.3") instead of tracking main. Concretely, modify the
curl target and the subsequent sh invocation in that step so you fetch the
release-specific installer URL or pass the version to the installer script and
run it with the secure flag, ensuring the installer verifies SHA256 checksums as
intended.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 32a2cc6a-62fd-4ee2-bae6-c44516ada299
📒 Files selected for processing (4)
.github/scripts/build_publish_payload.py.github/scripts/collect_worker_interface.py.github/scripts/resolve_binary_artifacts.py.github/workflows/_publish-registry.yml
Comment on lines
+55
to
+59
| - name: Install iii CLI | ||
| run: | | ||
| set -euo pipefail | ||
| curl -fsSL https://install.iii.dev/iii/main/install.sh -o /tmp/install-iii.sh | ||
| sh /tmp/install-iii.sh |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
find . -name "_publish-registry.yml" -o -name "*publish*registry*"Repository: iii-hq/workers
Length of output: 100
🏁 Script executed:
cat .github/workflows/_publish-registry.ymlRepository: iii-hq/workers
Length of output: 8462
🌐 Web query:
III installer versioning checksums signatures install.iii.dev documentation
💡 Result:
The install.iii.dev scripts for iii engine, console, and related tools download platform-specific binaries from GitHub releases of iii-hq/iii and verify them using SHA256 checksums published alongside each release (e.g., iii-x86_64-unknown-linux-musl.tar.gz.sha256 files containing the hash). The install.sh scripts explicitly state they verify the SHA256 checksum before installation. No cryptographic signatures (e.g., GPG, cosign .sig/.pem) on checksums or binaries are mentioned in documentation or releases. Binaries are unsigned (macOS requires xattr -d com.apple.quarantine). Versioning uses semantic tags like iii/v0.11.3; installers support pinning via -s -- v0.11.3 or default to latest stable (non-prerelease, iii/v* prefix). Official docs at https://iii.dev/docs/install recommend the curl | sh method without manual verification steps.
Citations:
- 1: https://iii.dev/docs/install
- 2: https://github.com/iii-hq/iii/blob/115674a1cc7887ec7ad1d850e09834fc061bc1ef/console/README.md
- 3: https://github.com/iii-hq/iii-cli
- 4: https://github.com/iii-hq/iii/tree/859b4497e42f7478b197cc2c1e7b2630d1e88ca6/console
- 5: https://github.com/iii-hq/iii/blob/06a6580d46649804e7635bb3160d8f9709247e74/engine/README.md
- 6: https://github.com/iii-hq/iii/blob/main/engine/README.md
- 7: https://github.com/iii-hq/iii/releases
- 8: https://github.com/iii-hq/iii/releases/tag/iii%2Fv0.9.0
- 9: https://github.com/iii-hq/iii/releases/tag/iii%2Fv0.11.3
- 10: https://github.com/iii-hq/iii/releases/tag/iii%2Fv0.11.0
Pin the III installer to a specific version instead of tracking main.
This step downloads and executes a floating remote shell script from install.iii.dev/iii/main/install.sh. Because the publish secret is used later in the same job, any unexpected change or compromise in the main branch installer can own the release path. Switch to a version-pinned install (e.g., install.sh -s -- v0.11.3). The installer automatically verifies SHA256 checksums, so no additional manual verification is needed.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/workflows/_publish-registry.yml around lines 55 - 59, The workflow
step named "Install iii CLI" currently downloads the floating installer at
install.iii.dev/iii/main/install.sh; update it to pin to a specific release and
invoke the installer with the version flag (e.g., pass "-s -- v0.11.3") instead
of tracking main. Concretely, modify the curl target and the subsequent sh
invocation in that step so you fetch the release-specific installer URL or pass
the version to the installer script and run it with the secure flag, ensuring
the installer verifies SHA256 checksums as intended.
- normalize_dependencies now converts list-form string entries to {name, version} dicts and validates each item
- normalize_worker_interface fails fast when worker_function_ids reference missing function details, instead of publishing blank schemas
- run_iii has a 30s subprocess timeout, and collect_triggers handles TimeoutExpired alongside the existing soft-fail paths
- pin the iii CLI installer to v0.11.5 via the VERSION env var so the publish job no longer tracks main
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/scripts/build_publish_payload.py:
- Around line 141-143: The code currently coerces falsy non-list values to [] by
using matches[0].get("functions") or [], which masks malformed payloads; change
the assignment to capture the raw value (e.g., worker_function_ids =
matches[0].get("functions")) and keep the existing type check (if not
isinstance(worker_function_ids, list): raise ValueError(...)) so that non-list
falsy values like {} or "" fail fast instead of being converted to an empty
list; ensure any subsequent logic handles a None case explicitly if you expect
the key to be optional.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 4a8dc27d-d914-4e58-a164-267819016a4c
📒 Files selected for processing (3)
.github/scripts/build_publish_payload.py.github/scripts/collect_worker_interface.py.github/workflows/_publish-registry.yml
🚧 Files skipped from review as they are similar to previous changes (2)
- .github/scripts/collect_worker_interface.py
- .github/workflows/_publish-registry.yml
Comment on lines
+141
to
+143
| worker_function_ids = matches[0].get("functions") or [] | ||
| if not isinstance(worker_function_ids, list): | ||
| raise ValueError("worker `functions` must be an array") |
There was a problem hiding this comment.
Don't coerce invalid functions payloads to [].
matches[0].get("functions") or [] turns falsy non-lists like {} or "" into a valid empty list. That weakens the fail-fast path below and can still let a malformed engine response publish with no functions.
Proposed fix
- worker_function_ids = matches[0].get("functions") or []
- if not isinstance(worker_function_ids, list):
+ raw_worker_function_ids = matches[0].get("functions")
+ if raw_worker_function_ids is None:
+ worker_function_ids = []
+ elif not isinstance(raw_worker_function_ids, list):
raise ValueError("worker `functions` must be an array")
+ else:
+ worker_function_ids = raw_worker_function_ids📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| worker_function_ids = matches[0].get("functions") or [] | |
| if not isinstance(worker_function_ids, list): | |
| raise ValueError("worker `functions` must be an array") | |
| raw_worker_function_ids = matches[0].get("functions") | |
| if raw_worker_function_ids is None: | |
| worker_function_ids = [] | |
| elif not isinstance(raw_worker_function_ids, list): | |
| raise ValueError("worker `functions` must be an array") | |
| else: | |
| worker_function_ids = raw_worker_function_ids |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/scripts/build_publish_payload.py around lines 141 - 143, The code
currently coerces falsy non-list values to [] by using
matches[0].get("functions") or [], which masks malformed payloads; change the
assignment to capture the raw value (e.g., worker_function_ids =
matches[0].get("functions")) and keep the existing type check (if not
isinstance(worker_function_ids, list): raise ValueError(...)) so that non-list
falsy values like {} or "" fail fast instead of being converted to an empty
list; ensure any subsequent logic handles a None case explicitly if you expect
the key to be optional.
sergiofilhowz
approved these changes
May 4, 2026
This was referenced May 4, 2026
sergiofilhowz
pushed a commit
that referenced
this pull request
May 4, 2026
…#69) * fix(ci): bump iii to v0.11.6-next.1 and refetch annotated release tag - _publish-registry.yml: pin iii install to v0.11.6-next.1 with --next. The previous v0.11.5 pin (added in #67) didn't surface external workers correctly in `engine::workers::list`, so `collect_worker_interface.py` raised `ValueError: expected exactly one worker matching 'image-resize', found 0` even when the worker was alive and connected. iii-hq/iii#1598 traced the same symptom on engine workers to PR iii-hq/iii#1585 landing in 0.11.6. - release.yml: actions/checkout@v4 fetches all tags first, then re-fetches the target SHA with `--no-tags +SHA:refs/tags/<tag>`, which silently overwrites the annotated tag with a lightweight one and drops the message. Add an explicit `git fetch origin +refs/tags/<tag>:refs/tags/<tag>` so `git tag -l --format=%(contents)` can still read the `registry-tag:` line and route the publish to next/latest as intended. * fix(ci): fall back to namespace match when worker has no name The Rust SDK's register_worker(url, opts) doesn't take a worker name, so external Rust workers (image-resize, mcp, proof, iii-lsp) register with id=<UUID> and name=null. Matching by `name == worker_name` always failed on these, even with the worker alive and connected. Add a fallback that looks for a worker whose registered functions are namespaced under the worker dir's snake_case (image-resize → image_resize::*). If exactly one worker matches, use it. On failure, include the workers list summary in the error so future regressions are debuggable from the log. Direct name/id match still wins so the iii-add path (Node/Python workers with explicit names) is unaffected.
4 tasks
3 tasks
sergiofilhowz
pushed a commit
that referenced
this pull request
May 5, 2026
The mcp/v0.5.1 publish failed because `_publish-registry.yml` started the worker with `cargo run --bin mcp -- --no-stdio`, but mcp's CLI no longer accepts that flag (mcp/src/main.rs only declares `--config`, `--url`, `--manifest`). cargo exited with `error: unexpected argument '--no-stdio' found` before the worker could connect, the 2s `kill -0` post-spawn check passed because cargo itself was still draining, and `collect_worker_interface` then waited 120s for a worker that never registered before raising `ValueError: could not match worker 'mcp'`. The flag was added in #67 specifically to keep mcp from binding stdio in the publish job; that concern is moot now (default mcp run is fine for interface collection). Drop the special case so mcp goes through the same `cargo run --bin mcp --` path as every other rust worker.
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
_publish-registry.ymlinto three testable scripts under.github/scripts/:build_publish_payload.py,collect_worker_interface.py,resolve_binary_artifacts.py.iiiengine and starts the worker locally during publish so we can collect the live function/trigger interface (descriptions,request_format,response_format, metadata) and ship it to the registry.request_format/response_formatfields to the registry'srequest_schema/response_schemapayload shape, and normalizes triggers under their owning worker.scripts.start(Rust binaries) can still serve their interface viacargo run.Test plan
_publish-registry.ymlend-to-end against the staging registry and verify/publishacceptsrequest_schema/response_schema.image-resize) and oneiii-addworker to confirm both runtime modes collect the interface.mcpworker still publishes correctly (uses the special-cased--no-stdioflag).functions[]is non-empty for every worker that previously published.Summary by CodeRabbit
Release Notes