feat(landmine): port prod hotpatch landmines into main - #53
Conversation
Queue on empty 1-GPU Blackwell inventory instead of terminal-failing capacity misses so training plane can wait for stock. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Prefix-owned orphan cleanup guards against leaked paid pods after interrupted training runs. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Expose training-plane env knobs and prism_dispatch_variant so Lium wiring and orchestration can fail closed without an API key. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Build capacity/orphan/client helpers from LiumTrainingSettings with fail-closed behavior when credentials are missing. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Add for_prism_training surface and training GPU lock helpers used by the host landmine training plane. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Register training-plane settings and factories on the CLI entry path alongside existing constation wiring. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Port host allowlist repository deltas required by Lium training constation checks. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Hook capacity, orphan cleanup, and training client factories into the master orchestration loop. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Pure validators for SHA/URL/env forbid lists used before Lium pod boot. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Add PRISM_CHECKPOINT_UPLOAD_ENABLED gate, public top-prism repo default, CheckpointPublishError→502 mapping, and last_status/last_error intake observability from the host landmine ports. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
When worker-plane is on, process_next becomes a no-op and container processing refuses so Lium owns GPU execution. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Port host base-master-entrypoint defaults for worker-plane and plagiarism LLM env into docker/master-entrypoint.sh. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
📝 WalkthroughWalkthroughChangesThe PR adds checkpoint upload gating and publish observability, secure Prism pod-boot planning, worker-plane ownership enforcement, Lium training capacity management, and Constation identity pin integration into master orchestration. Checkpoint publishing
Pod boot and worker-plane controls
Lium training capacity
Constation identity orchestration
Estimated code review effort: 5 (Critical) | ~120 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Satisfy CI ruff/format/mypy/prism-checks: format touched modules, sort queue imports, narrow orphan terminate arg, cast orchestration request in tests, wrap long entrypoint assertion lines. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (6)
packages/challenges/prism/src/prism_challenge/queue.py (1)
198-207: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winExtract the worker-plane ownership check into one helper to prevent drift.
The exact condition
self.settings.worker_plane.enabled and not self.settings.worker_plane.cpu_reexec_test_modeis duplicated inprocess_nextand_process_container. Since this guards the critical VAL-PRISM-037 invariant (master must never execute GPU/container eval when the worker plane owns it), a future edit to one site without the other would silently reopen the hole.♻️ Proposed helper extraction
+ def _worker_plane_owns_execution(self) -> bool: + """True when the worker plane must own GPU/container eval, not this process.""" + return ( + self.settings.worker_plane.enabled + and not self.settings.worker_plane.cpu_reexec_test_mode + ) + async def process_next(self) -> str | None: ... - if ( - self.settings.worker_plane.enabled - and not self.settings.worker_plane.cpu_reexec_test_mode - ): + if self._worker_plane_owns_execution(): return None- if ( - self.settings.worker_plane.enabled - and not self.settings.worker_plane.cpu_reexec_test_mode - ): + if self._worker_plane_owns_execution(): raise RuntimeError( "worker_plane_enabled: master container eval disabled; " "Lium/miners own GPU execution (process_next no-op)" )Also applies to: 437-445
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/challenges/prism/src/prism_challenge/queue.py` around lines 198 - 207, Extract the duplicated worker-plane ownership condition into a private helper on the challenge queue class, then replace the checks in both process_next and _process_container with that helper. Preserve the exact enabled-and-not-cpu_reexec_test_mode behavior so both execution paths consistently enforce the VAL-PRISM-037 invariant.packages/challenges/prism/tests/test_pod_boot.py (1)
262-278: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winMissing dedicated negative tests for
_validate_master_checkpoint_url.Unlike
validate_repo_url, which gets its owntest_validate_repo_url_rejects_dangerous_shapesparametrize block,_validate_master_checkpoint_url's reject branches (shell metachars, embedded credentials,javascript:/file:scheme, missing host/path) are only indirectly reached through twobuild_boot_envhappy/negative cases. Consider a dedicated parametrized test mirroring the repo-URL coverage for this new security-relevant validator.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/challenges/prism/tests/test_pod_boot.py` around lines 262 - 278, The test suite lacks direct coverage for _validate_master_checkpoint_url rejection paths. Add a dedicated parametrized negative test mirroring validate_repo_url’s dangerous-shape coverage, including shell metacharacters, embedded credentials, javascript:/file: schemes, and missing host or path, and assert each case raises PodBootError directly through the validator.docker/master-entrypoint.sh (1)
211-224: 🩺 Stability & Availability | 🔵 TrivialLoopback placeholder defaults for worker-plane/constation URLs need an explicit override in real deployments.
PRISM_WORKER_PLANE__MASTER_BASE_URLandPRISM_CONSTATION_BASE_URLdefault tohttp://127.0.0.1:8081, unreachable from an actual miner-funded Lium pod onceworker_plane.enabled=true. Since these are separate variables fromPRISM_MASTER_BASE_URL, an operator enabling the worker plane could easily forget to override them, silently breaking pod→master coordination with no startup-time failure. Consider a startup check (or prominent deploy-doc callout) that fails fast whenworker_plane.enabledis true but this URL is still the loopback default.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docker/master-entrypoint.sh` around lines 211 - 224, The startup environment assembly around PRISM_WORKER_PLANE__MASTER_BASE_URL and PRISM_CONSTATION_BASE_URL must fail fast when worker_plane.enabled is true and either URL remains its loopback default. Add validation before launching the service, while preserving explicitly configured non-loopback URLs and existing defaults when the worker plane is disabled.src/base/master/orchestration.py (1)
210-240: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winCache the active pin once per bridge pass instead of per work-unit.
_stamp_constation_identityis called once per prism pending-work item in this loop, and each call re-runsget_active_pin(allowlist_repository.py), which issues up to 3 full-table queries in a fresh session. Sinceself._prism_dispatch_variantis fixed for the whole pass, the resolved pin is the same for every item in this batch — resolving it once perbridge_pending_work()call and reusing it for all items would cut a potential 3N queries per pass down to a constant few, and also makes stamping consistent across the whole batch.♻️ Proposed refactor sketch
async def bridge_pending_work(self) -> dict[str, list[str]]: works = await self._work_source.fetch_pending_work() bridged: dict[str, list[str]] = {} + constation_pin = await self._resolve_constation_pin() for work in works: ... else: payload = dict(work.payload) if work.job_id is not None: payload[PAYLOAD_JOB_ID_KEY] = work.job_id - payload = await self._stamp_constation_identity(payload) + payload = self._apply_constation_identity(payload, constation_pin)Split
_stamp_constation_identityinto a pin-resolution helper (called once) and a pure payload-merge helper (called per item).Also applies to: 267-293
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/base/master/orchestration.py` around lines 210 - 240, Resolve the active pin once at the start of the bridge pass in bridge_pending_work, using a dedicated pin-resolution helper, then reuse that value for every prism work item. Refactor _stamp_constation_identity into separate pin-resolution and pure payload-merge helpers so per-item processing does not call get_active_pin, while preserving the existing payload stamping behavior.packages/challenges/prism/src/prism_challenge/evaluator/checkpoint_publisher.py (1)
150-165: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueDuplicate
PRISM_CHECKPOINT_UPLOAD_ENABLEDtruthy-parsing logic.The same env-var parse-and-check block appears in
publisher_from_env()andHuggingFaceCheckpointPublisher.publish(). Extract a shared helper to avoid drift.♻️ Proposed refactor
+def _checkpoint_upload_enabled() -> bool: + import os + + value = (os.environ.get("PRISM_CHECKPOINT_UPLOAD_ENABLED") or "false").strip().lower() + return value in ("1", "true", "yes", "on") + + def publisher_from_env( *, repo_id: str | None = None, token: str | None = None, ) -> CheckpointPublisher: """Return HF publisher only when upload is explicitly enabled; else disabled no-op.""" - import os - - enabled = (os.environ.get("PRISM_CHECKPOINT_UPLOAD_ENABLED") or "false").strip().lower() - if enabled in ("1", "true", "yes", "on"): + if _checkpoint_upload_enabled(): return HuggingFaceCheckpointPublisher( repo_id=repo_id or DEFAULT_CHECKPOINT_REPO_ID, token=token, ) return DisabledCheckpointPublisher(repo_id=repo_id or DEFAULT_CHECKPOINT_REPO_ID)def publish(self, upload: CheckpointUpload) -> PublishedCheckpoint: - import os - enabled = (os.environ.get("PRISM_CHECKPOINT_UPLOAD_ENABLED") or "false").strip().lower() - if enabled not in ("1", "true", "yes", "on"): + if not _checkpoint_upload_enabled(): raise RuntimeError(Also applies to: 195-204
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/challenges/prism/src/prism_challenge/evaluator/checkpoint_publisher.py` around lines 150 - 165, Extract the duplicated PRISM_CHECKPOINT_UPLOAD_ENABLED parsing and truthy-value check into a shared helper, then reuse it in publisher_from_env() and HuggingFaceCheckpointPublisher.publish(). Preserve the existing accepted values and behavior, including returning DisabledCheckpointPublisher when uploads are not enabled.packages/challenges/prism/tests/test_prism_hf_checkpoint_publish.py (1)
409-426: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winTighten status-code assertion to the specific 502 contract.
response.status_code >= 400doesn't verify the actual mapping this PR introduces (CheckpointPublishError→ HTTP 502 inapp.py). Asserting the exact code better guards the new contract against regressions.✅ Proposed tightening
- assert response.status_code >= 400, response.text + assert response.status_code == 502, response.text🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/challenges/prism/tests/test_prism_hf_checkpoint_publish.py` around lines 409 - 426, Update test_publish_failure_records_no_checkpoint_ref_and_exposes_last_error to assert that response.status_code equals HTTP 502, replacing the broad >= 400 check. Keep the existing response text and checkpoint-ref assertions unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@packages/challenges/prism/src/prism_challenge/evaluator/checkpoint_intake.py`:
- Around line 220-231: Update _safe_error_message to accept the configured
secret value(s), redact exact secret matches before returning the message, and
retain the existing marker-based redaction for unidentified secrets. Update its
callers to pass the publisher token, and add coverage that constructs an
exception containing a marker and verifies the returned message contains no
secret value.
In
`@packages/challenges/prism/src/prism_challenge/evaluator/checkpoint_publisher.py`:
- Around line 120-147: Update DisabledCheckpointPublisher.publish() so disabled
uploads produce no resumable checkpoint reference, using the existing empty/None
sentinel expected by PrismRepository.latest_checkpoint_ref() and
_stage_resume_checkpoint(). Preserve recording the upload and published result,
but ensure the returned PublishedCheckpoint cannot be resolved or passed to
download() as a valid resume reference.
In `@packages/challenges/prism/src/prism_challenge/evaluator/pod_boot.py`:
- Around line 84-92: Expand _SECRET_SUFFIXES to include the generic "_TOKEN" and
"_KEY" suffixes so _is_obvious_secret_name rejects names such as GITEA_TOKEN,
SLACK_TOKEN, and STRIPE_KEY when build_boot_env constructs pod environment
variables.
In `@src/base/compute/lium_capacity.py`:
- Around line 201-228: Update tick() so an _admit_one() result of None does not
break FIFO admission or consume the remaining admission opportunity; mark the
failed lease’s capacity reason as needed, then continue attempting later queued
leases with remaining offers and free capacity. Preserve the existing stop
conditions for exhausted offers or capacity, while ensuring persistently failing
leases cannot block subsequent queued leases indefinitely.
In `@src/base/compute/lium_training_wiring.py`:
- Around line 92-140: Update build_lium_capacity_scheduler to fail closed when
enabled training configuration lacks lium_training.ssh_public_key_file, rather
than allowing the scheduler’s placeholder key default. Load the configured file,
split its contents into non-empty lines, and pass the resulting keys through
ssh_public_keys; reject missing or empty key-file content.
---
Nitpick comments:
In `@docker/master-entrypoint.sh`:
- Around line 211-224: The startup environment assembly around
PRISM_WORKER_PLANE__MASTER_BASE_URL and PRISM_CONSTATION_BASE_URL must fail fast
when worker_plane.enabled is true and either URL remains its loopback default.
Add validation before launching the service, while preserving explicitly
configured non-loopback URLs and existing defaults when the worker plane is
disabled.
In
`@packages/challenges/prism/src/prism_challenge/evaluator/checkpoint_publisher.py`:
- Around line 150-165: Extract the duplicated PRISM_CHECKPOINT_UPLOAD_ENABLED
parsing and truthy-value check into a shared helper, then reuse it in
publisher_from_env() and HuggingFaceCheckpointPublisher.publish(). Preserve the
existing accepted values and behavior, including returning
DisabledCheckpointPublisher when uploads are not enabled.
In `@packages/challenges/prism/src/prism_challenge/queue.py`:
- Around line 198-207: Extract the duplicated worker-plane ownership condition
into a private helper on the challenge queue class, then replace the checks in
both process_next and _process_container with that helper. Preserve the exact
enabled-and-not-cpu_reexec_test_mode behavior so both execution paths
consistently enforce the VAL-PRISM-037 invariant.
In `@packages/challenges/prism/tests/test_pod_boot.py`:
- Around line 262-278: The test suite lacks direct coverage for
_validate_master_checkpoint_url rejection paths. Add a dedicated parametrized
negative test mirroring validate_repo_url’s dangerous-shape coverage, including
shell metacharacters, embedded credentials, javascript:/file: schemes, and
missing host or path, and assert each case raises PodBootError directly through
the validator.
In `@packages/challenges/prism/tests/test_prism_hf_checkpoint_publish.py`:
- Around line 409-426: Update
test_publish_failure_records_no_checkpoint_ref_and_exposes_last_error to assert
that response.status_code equals HTTP 502, replacing the broad >= 400 check.
Keep the existing response text and checkpoint-ref assertions unchanged.
In `@src/base/master/orchestration.py`:
- Around line 210-240: Resolve the active pin once at the start of the bridge
pass in bridge_pending_work, using a dedicated pin-resolution helper, then reuse
that value for every prism work item. Refactor _stamp_constation_identity into
separate pin-resolution and pure payload-merge helpers so per-item processing
does not call get_active_pin, while preserving the existing payload stamping
behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: de8d7e78-974c-4f31-88df-e7f8203ba681
📒 Files selected for processing (32)
docker/master-entrypoint.shpackages/challenges/prism/src/prism_challenge/app.pypackages/challenges/prism/src/prism_challenge/config.pypackages/challenges/prism/src/prism_challenge/evaluator/checkpoint_intake.pypackages/challenges/prism/src/prism_challenge/evaluator/checkpoint_publisher.pypackages/challenges/prism/src/prism_challenge/evaluator/pod_boot.pypackages/challenges/prism/src/prism_challenge/queue.pypackages/challenges/prism/tests/test_checkpoint_upload_gate.pypackages/challenges/prism/tests/test_config.pypackages/challenges/prism/tests/test_pod_boot.pypackages/challenges/prism/tests/test_prism_checkpoint_publisher.pypackages/challenges/prism/tests/test_prism_hf_checkpoint_publish.pypackages/challenges/prism/tests/test_queue_worker_plane_noop.pysrc/base/cli_app/main.pysrc/base/compute/lium.pysrc/base/compute/lium_capacity.pysrc/base/compute/lium_orphan.pysrc/base/compute/lium_training_wiring.pysrc/base/config/settings.pysrc/base/master/constation/allowlist_repository.pysrc/base/master/orchestration.pytests/unit/test_compute_lium_client.pytests/unit/test_constation_main_wiring.pytests/unit/test_constation_settings.pytests/unit/test_digest_allowlist_repository.pytests/unit/test_lium_capacity_scheduler.pytests/unit/test_lium_orphan.pytests/unit/test_lium_training_settings.pytests/unit/test_lium_training_wiring.pytests/unit/test_master_entrypoint_prism_policy.pytests/unit/test_orchestration.pytests/unit/test_orchestration_lium.py
| def _safe_error_message(exc: BaseException) -> str: | ||
| """Human-readable error without secret-shaped values (tokens never logged).""" | ||
| name = type(exc).__name__ | ||
| text = str(exc).strip() or name | ||
| # Hard-cap length; strip common secret-bearing substrings if a caller ever embeds them. | ||
| lowered = text.lower() | ||
| for marker in ("hf_token", "authorization:", "bearer ", "api_key=", "token="): | ||
| if marker in lowered: | ||
| return f"{name}: <redacted>" | ||
| if len(text) > 500: | ||
| text = text[:500] + "…" | ||
| return f"{name}: {text}" if name not in text else text |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
_safe_error_message redaction is heuristic-only and untested for the redaction branch itself.
Redaction only fires when the raw exception text contains one of a handful of literal markers (hf_token, authorization:, bearer , api_key=, token=). A bare secret value embedded without one of these markers (e.g., a raw HF token or key surfaced by a library error) passes straight through into the logged/HTTP-502 error message. Since this function has no access to the actual configured token value, it can't scrub an exact match either. Per coding guidelines, secrets/full token values must never be logged.
Consider passing the known secret value(s) (e.g., the publisher's token) into this function so it can scrub exact matches, in addition to (or instead of) the marker heuristic. Also, none of the added tests actually construct an exception whose message contains one of these markers to verify the redaction path triggers.
As per coding guidelines, "Never log, document, or include evidence containing private keys, wallet mnemonics, API tokens, or full secret values; only names, digests, and SHAs are permitted."
🛡️ Proposed direction
-def _safe_error_message(exc: BaseException) -> str:
+def _safe_error_message(exc: BaseException, *, secrets: tuple[str, ...] = ()) -> str:
"""Human-readable error without secret-shaped values (tokens never logged)."""
name = type(exc).__name__
text = str(exc).strip() or name
+ for secret in secrets:
+ if secret and secret in text:
+ return f"{name}: <redacted>"
# Hard-cap length; strip common secret-bearing substrings if a caller ever embeds them.
lowered = text.lower()
for marker in ("hf_token", "authorization:", "bearer ", "api_key=", "token="):📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def _safe_error_message(exc: BaseException) -> str: | |
| """Human-readable error without secret-shaped values (tokens never logged).""" | |
| name = type(exc).__name__ | |
| text = str(exc).strip() or name | |
| # Hard-cap length; strip common secret-bearing substrings if a caller ever embeds them. | |
| lowered = text.lower() | |
| for marker in ("hf_token", "authorization:", "bearer ", "api_key=", "token="): | |
| if marker in lowered: | |
| return f"{name}: <redacted>" | |
| if len(text) > 500: | |
| text = text[:500] + "…" | |
| return f"{name}: {text}" if name not in text else text | |
| def _safe_error_message(exc: BaseException, *, secrets: tuple[str, ...] = ()) -> str: | |
| """Human-readable error without secret-shaped values (tokens never logged).""" | |
| name = type(exc).__name__ | |
| text = str(exc).strip() or name | |
| for secret in secrets: | |
| if secret and secret in text: | |
| return f"{name}: <redacted>" | |
| # Hard-cap length; strip common secret-bearing substrings if a caller ever embeds them. | |
| lowered = text.lower() | |
| for marker in ("hf_token", "authorization:", "bearer ", "api_key=", "token="): | |
| if marker in lowered: | |
| return f"{name}: <redacted>" | |
| if len(text) > 500: | |
| text = text[:500] + "…" | |
| return f"{name}: {text}" if name not in text else text |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/challenges/prism/src/prism_challenge/evaluator/checkpoint_intake.py`
around lines 220 - 231, Update _safe_error_message to accept the configured
secret value(s), redact exact secret matches before returning the message, and
retain the existing marker-based redaction for unidentified secrets. Update its
callers to pass the publisher token, and add coverage that constructs an
exception containing a marker and verifies the returned message contains no
secret value.
Source: Coding guidelines
| @dataclass | ||
| class DisabledCheckpointPublisher: | ||
| """No-op publisher: Prism HF checkpoint upload is intentionally OFF on this host. | ||
|
|
||
| publish() records the call but never contacts HuggingFace. download() raises so | ||
| resume-from-public-checkpoint is unavailable while upload is disabled. | ||
| """ | ||
|
|
||
| repo_id: str = DEFAULT_CHECKPOINT_REPO_ID | ||
| uploads: list = field(default_factory=list) | ||
| published: list = field(default_factory=list) | ||
|
|
||
| def publish(self, upload: CheckpointUpload) -> PublishedCheckpoint: | ||
| self.uploads.append(upload) | ||
| result = PublishedCheckpoint( | ||
| checkpoint_ref=checkpoint_ref_for(self.repo_id, upload.revision), | ||
| repo_id=self.repo_id, | ||
| revision=upload.revision, | ||
| files=tuple(upload.files), | ||
| ) | ||
| self.published.append(result) | ||
| return result | ||
|
|
||
| def download(self, checkpoint_ref: str, dest_dir: Path) -> Path: | ||
| raise RuntimeError( | ||
| "prism_checkpoint_upload_disabled: download unavailable while " | ||
| "PRISM_CHECKPOINT_UPLOAD_ENABLED is false" | ||
| ) |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Locate resume-on-reassignment call sites that use checkpoint_ref/download()
rg -n -C3 'resume_checkpoint_ref|checkpoint_ref' --type=py -g '!**/tests/**' packages/challenges/prism/src/prism_challengeRepository: BaseIntelligence/base
Length of output: 50377
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== checkpoint_publisher relevant sections =="
sed -n '1,240p' packages/challenges/prism/src/prism_challenge/evaluator/checkpoint_publisher.py
echo
echo "== container resume checkpoint code =="
sed -n '320,365p' packages/challenges/prism/src/prism_challenge/evaluator/container.py
sed -n '640,685p' packages/challenges/prism/src/prism_challenge/queue.py
sed -n '365,390p' packages/challenges/prism/src/prism_challenge/queue.py
echo
echo "== call sites that resolve checkpoint publisher =="
rg -n -C4 '_resolve_checkpoint_publisher|PrismCheckpoint|PrismCheckpointPublisher|HF_CHECKPOINT|CHECKPOINT_UPLOAD' packages/challenges/prism/src/prism_challenge -g '*.py'Repository: BaseIntelligence/base
Length of output: 21704
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== checkpoint_intake publish handling =="
sed -n '100,195p' packages/challenges/prism/src/prism_challenge/evaluator/checkpoint_intake.py
echo
echo "== app/v2 checkpoint intake endpoint handling =="
sed -n '260,320p' packages/challenges/prism/src/prism_challenge/app.py
echo
echo "== repository record/latest checkpoint reference error handling =="
sed -n '330,370p' packages/challenges/prism/src/prism_challenge/repository.py
sed -n '1040,1100p' packages/challenges/prism/src/prism_challenge/repository.py
echo
echo "== all publish calls with exception handling =="
rg -n -C4 '\.publish\(|publisher_from_env|CheckpointPublishError|last_checkpoint_ref|record.*checkpoint' packages/challenges/prism/src/prism_challenge packages/challenges/prism -g '*.py' | head -n 220Repository: BaseIntelligence/base
Length of output: 33927
Make the disabled checkpoint ref unusable for resume instead of resolvable.
DisabledCheckpointPublisher.publish() returns a valid checkpoint_ref, and PrismRepository.latest_checkpoint_ref() stores it in the work-unit payload, but DisabledCheckpointPublisher.download() then raises RuntimeError. A reassigned validator will call _stage_resume_checkpoint(resume_checkpoint_ref=...), which passes that ref directly to download() without a fallback, so the resume path fails instead of treating the submission as having no resumable checkpoint.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@packages/challenges/prism/src/prism_challenge/evaluator/checkpoint_publisher.py`
around lines 120 - 147, Update DisabledCheckpointPublisher.publish() so disabled
uploads produce no resumable checkpoint reference, using the existing empty/None
sentinel expected by PrismRepository.latest_checkpoint_ref() and
_stage_resume_checkpoint(). Preserve recording the upload and published result,
but ensure the returned PublishedCheckpoint cannot be resolved or passed to
download() as a valid resume reference.
| _SECRET_SUFFIXES: Final[tuple[str, ...]] = ( | ||
| "_PASSWORD", | ||
| "_PASSWD", | ||
| "_SECRET", | ||
| "_SECRET_KEY", | ||
| "_API_KEY", | ||
| "_ACCESS_TOKEN", | ||
| "_PRIVATE_KEY", | ||
| ) |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
Secret-suffix heuristic misses generic _TOKEN/_KEY names.
_SECRET_SUFFIXES only special-cases _ACCESS_TOKEN, _API_KEY, _PRIVATE_KEY, _SECRET_KEY — a kwarg like GITEA_TOKEN, SLACK_TOKEN, or STRIPE_KEY passed to build_boot_env would not match any exact FORBIDDEN_ENV_KEYS entry, the HF_/LIUM_ prefix rules, or a suffix here, so _is_obvious_secret_name returns False and the value is happily injected into the pod env — contradicting the module's stated "Zero secrets on the pod" / "other obvious secret names" guarantee.
🔒 Broaden the suffix heuristic
_SECRET_SUFFIXES: Final[tuple[str, ...]] = (
"_PASSWORD",
"_PASSWD",
"_SECRET",
"_SECRET_KEY",
"_API_KEY",
"_ACCESS_TOKEN",
"_PRIVATE_KEY",
+ "_TOKEN",
+ "_KEY",
)Also applies to: 200-222
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/challenges/prism/src/prism_challenge/evaluator/pod_boot.py` around
lines 84 - 92, Expand _SECRET_SUFFIXES to include the generic "_TOKEN" and
"_KEY" suffixes so _is_obvious_secret_name rejects names such as GITEA_TOKEN,
SLACK_TOKEN, and STRIPE_KEY when build_boot_env constructs pod environment
variables.
| async def tick(self) -> list[LiumLease]: | ||
| """Admit FIFO queued leases up to free slots. Never raises for capacity.""" | ||
| client = self._client_factory() | ||
| if self._spend_gate is not None and not self._spend_gate(): | ||
| self._mark_queued_reason(REASON_SPEND_CEILING) | ||
| return [] | ||
|
|
||
| offers = list( | ||
| await client.list_offers(max_price_per_hour=self._max_price_per_hour) | ||
| ) | ||
| free = self._free_slots(len(offers)) | ||
| if free <= 0: | ||
| self._mark_queued_reason(REASON_CAPACITY_WAIT) | ||
| return [] | ||
|
|
||
| admitted: list[LiumLease] = [] | ||
| for lease in self._queued_fifo(): | ||
| if len(admitted) >= free or not offers: | ||
| if not offers: | ||
| self._set_reason(lease.submission_id, REASON_CAPACITY_WAIT) | ||
| break | ||
| offer = offers.pop(0) | ||
| result = await self._admit_one(client, lease, offer) | ||
| if result is None: | ||
| self._set_reason(lease.submission_id, REASON_CAPACITY_WAIT) | ||
| break | ||
| admitted.append(result) | ||
| return admitted |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Files:"
git ls-files | rg '(^|/)lium_capacity\.py$|lium_capacity_scheduler|leas|scheduler' || true
echo
echo "Target file outline:"
ast-grep outline src/base/compute/lium_capacity.py || true
echo
echo "Relevant source:"
cat -n src/base/compute/lium_capacity.py | sed -n '1,280p'
echo
echo "Search queued_fifo/admit_one/queued reason:"
rg -n "_queued_fifo|_admit_one|_queued|QUEUED|set_reason|reason" src/base/compute/lium_capacity.pyRepository: BaseIntelligence/base
Length of output: 18932
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Remaining lium_capacity source:"
cat -n src/base/compute/lium_capacity.py | sed -n '282,370p'
echo
echo "Scheduler tests outline/search:"
ast-grep outline tests/unit/test_lium_capacity_scheduler.py || true
rg -n "tick|admit|fail|reason|capacity|queued|head|fifo|offer|None" tests/unit/test_lium_capacity_scheduler.py src/base/compute/lium_capacity.py
echo
echo "Behavioral probe from source text: parse tick/_admit_one loop-break and requeue fields"
python3 - <<'PY'
from pathlib import Path
src = Path("src/base/compute/lium_capacity.py").read_text()
print("tick break after None:", "if result is None:" in src and "break" in src[src.index("if result is None:"):src.index("admitted.append(result)")])
print("_admit_one reenqueue excerpt:")
for line in src[src.index("async def _admit_one"):src.index("def pod_name_for")].splitlines():
if "state=LeaseState.QUEUED" in line or "enqueued_at=current.enqueued_at" in line or "accept" in line.lower():
print(line.strip())
print("offer popped before admit:", "offer = offers.pop(0)" in src[src.index("offers = list("):src.index("result = await self._admit_one")])
PYRepository: BaseIntelligence/base
Length of output: 19336
Avoid head-of-line blocking in tick() when provision() fails.
_admit_one() catches exceptions, requeues the lease as QUEUED without updating enqueued_at, and returns None; tick() then breaks before trying remaining queued leases and remaining offers. A single persistently failing queued lease can therefore block all later queued leases indefinitely. Consider continuing with the remaining capacity for the next queued lease, or quarantining/backoff-failing leases so they don’t block the FIFO queue.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/base/compute/lium_capacity.py` around lines 201 - 228, Update tick() so
an _admit_one() result of None does not break FIFO admission or consume the
remaining admission opportunity; mark the failed lease’s capacity reason as
needed, then continue attempting later queued leases with remaining offers and
free capacity. Preserve the existing stop conditions for exhausted offers or
capacity, while ensuring persistently failing leases cannot block subsequent
queued leases indefinitely.
| def build_lium_capacity_scheduler( | ||
| settings: _HasLiumTraining, | ||
| *, | ||
| image: str | None = None, | ||
| client_factory: Callable[[], LiumCapacityClient] | None = None, | ||
| store: object | None = None, | ||
| spend_gate: Callable[[], bool] | None = None, | ||
| ) -> LiumCapacityScheduler: | ||
| """Build :class:`LiumCapacityScheduler` from settings + training-locked client. | ||
|
|
||
| Requires ``lium_training.enabled=True`` and a resolvable API key (fail-closed). | ||
| Image defaults to :data:`DEFAULT_LIUM_TRAINING_IMAGE` (prism-evaluator, no digest). | ||
| """ | ||
| lt = settings.lium_training | ||
| if not lt.enabled: | ||
| raise ValueError( | ||
| "lium_training.enabled is False; refuse to build LiumCapacityScheduler" | ||
| ) | ||
|
|
||
| factory = client_factory | ||
| if factory is None: | ||
| # Capture key once so factory does not re-read settings on every call | ||
| # in a way that could race; still fail-closed if key missing. | ||
| api_key = resolve_lium_training_api_key(lt) | ||
| if api_key is None: | ||
| raise ValueError( | ||
| "lium_training.enabled is True but API key is missing " | ||
| "(set lium_training.api_key or lium_training.api_key_file)" | ||
| ) | ||
|
|
||
| def factory() -> LiumCapacityClient: | ||
| return LiumClient.for_prism_training(api_key) | ||
|
|
||
| ssh_keys = _load_ssh_public_keys(lt) | ||
| kwargs: dict[str, object] = { | ||
| "concurrency_cap": int(lt.concurrency_cap), | ||
| "pod_name_prefix": str(lt.pod_name_prefix), | ||
| "max_price_per_hour": float(lt.max_price_per_hour), | ||
| "max_lifetime_hours": float(lt.max_lifetime_hours), | ||
| "image": image if image is not None else DEFAULT_LIUM_TRAINING_IMAGE, | ||
| } | ||
| if store is not None: | ||
| kwargs["store"] = store | ||
| if spend_gate is not None: | ||
| kwargs["spend_gate"] = spend_gate | ||
| if ssh_keys is not None: | ||
| kwargs["ssh_public_keys"] = ssh_keys | ||
|
|
||
| return LiumCapacityScheduler(factory, **kwargs) # type: ignore[arg-type] |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== locate files =="
fd -a 'lium_training_wiring\.py|lium_capacity\.py$' . || true
echo "== relevant outlines =="
for f in $(fd 'lium_training_wiring\.py|lium_capacity\.py$' .); do
echo "--- $f ($(wc -l < "$f") lines) ---"
ast-grep outline "$f" --view expanded || true
done
echo "== relevant content =="
for f in $(fd 'lium_training_wiring\.py|lium_capacity\.py$' .); do
echo "--- $f (lines 1-260) ---"
sed -n '1,260p' "$f" | nl -ba
done
echo "== search ssh API usages =="
rg -n "ssh-public|ssh_public|/ssh-keys|ensure_ssh_key|ensure_ssh" . -S || trueRepository: BaseIntelligence/base
Length of output: 4598
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== build_lium_training_wiring excerpt =="
sed -n '42,160p' src/base/compute/lium_training_wiring.py
echo
echo "== lium_capacity default/provision excerpts =="
sed -n '115,145p' src/base/compute/lium_capacity.py
sed -n '296,340p' src/base/compute/lium_capacity.py
echo
echo "== tests excerpts =="
sed -n '75,125p' tests/unit/test_lium_training_wiring.py
echo
echo "== deterministic load/fallback probe =="
python3 - <<'PY'
from pathlib import Path
import ast, inspect, textwrap
src = Path("src/base/compute/lium_training_wiring.py").read_text()
tree = ast.parse(src)
def get_func(name):
return next(n for n in tree.body if isinstance(n, ast.FunctionDef) and n.name == name)
load = get_func("_load_ssh_public_keys")
build = get_func("build_lium_capacity_scheduler")
load_src = inspect.getsource(load)
build_src = inspect.getsource(build)
print("_load_ssh_public_keys_source:")
print(load_src)
print("calls read_text().strip():", "read_text(encoding='utf-8').strip()" in load_src)
print("assigns ssh_public_keys kwarg only if not None:", 'if ssh_keys is not None:' in build_src)
print("build overwrites without require check:", "ssh_keys = _load_ssh_public_keys(lt)" in build_src and "raise ValueError" not in build_src[build_src.find("ssh_keys = _load_ssh_public_keys(lt)"):])
PY
echo
echo "== search all Liium API references =="
rg -n "LiumClient|for_prism_training|/ssh-keys|ssh-keys|ssh_public|api_key|ensure_ssh" src tests -S || trueRepository: BaseIntelligence/base
Length of output: 10564
🌐 Web query:
Lium /ssh-keys endpoint validate public key format source
💡 Result:
The Lium platform manages SSH keys via its API and CLI, but it does not provide a specific, public-facing documentation page detailing the internal regex or validation logic used to parse SSH public keys at the /ssh-keys endpoint [1][2][3]. The platform exposes an OpenAPI 3.1 schema that defines the structure for SSH key management, which can be accessed for the most accurate and up-to-date specifications of the API payloads: Raw JSON OpenAPI schema: https://lium.io/api/openapi.json [3] Interactive Swagger UI: https://lium.io/documents [3] According to the Lium SDK documentation, the SSHKey model expects a string for the public_key field [1][2]. When using the Lium CLI, the tool facilitates the registration of existing public keys or generates new key pairs [4]. If you are implementing a client or performing manual validation, standard industry practices for SSH public key validation typically involve verifying that the string begins with a valid format header (e.g., ssh-rsa, ssh-ed25519) and that the key data is correctly encoded in base64 format [5]. For definitive validation requirements, it is recommended to inspect the OpenAPI schema directly or test against the Lium API's response behavior using your generated API key [3].
Citations:
- 1: https://docs.lium.io/developers/sdk/reference/models/ssh-key
- 2: https://docs.lium.io/developers/sdk/reference
- 3: https://docs.lium.io/developers/openapi
- 4: https://docs.lium.io/developers/cli/installation
- 5: https://stackoverflow.com/questions/2494450/ssh-rsa-public-key-validation-using-a-regular-expression
🌐 Web query:
Lium /ssh-keys OpenAPI ssh_public_keys required validation
💡 Result:
The Lium platform API requires valid SSH public key information for various operations, particularly when creating pods or managing user identity [1][2][3]. Validation of SSH public keys on the Lium platform is handled via the OpenAPI schema defined by the Lium backend (a FastAPI application) [4]. The specific validation requirements for SSH keys include: 1. API Schema Requirements: When registering or using SSH public keys, the platform expects data to adhere to the SSHKey model or specific request schemas [1][5]. For instance, endpoints involved in creating or registering keys typically require a public_key field (often a string containing the OpenSSH-formatted key) and sometimes an associated name [1][6][3][5]. 2. Endpoint-Specific Validation: - When renting executors or creating pods, the API expects a list of SSH key identifiers or raw public keys depending on the endpoint [1][2][3]. - The OpenAPI definition (available at https://lium.io/api/openapi.json) is the definitive source for required fields and validation rules [4]. 3. Implementation Context: Recent platform updates have shifted away from passing SSH public keys as build arguments in container images (which previously required manual validation) toward managing these keys through the SDK and API via authenticated channels [7][8]. This ensures that keys are handled securely and consistently across the infrastructure [8]. For developers, it is recommended to reference the live OpenAPI documentation at https://lium.io/documents or download the raw JSON specification to inspect the most current required fields and validation constraints for ssh_public_keys or related fields [4][2].
Citations:
- 1: https://context7.com/websites/celiumcompute_ai_api_openapi_json/llms.txt
- 2: https://docs.lium.io/developers/quickstart
- 3: https://docs.lium.io/developers/sdk/reference/client/lium
- 4: https://docs.lium.io/developers/openapi
- 5: https://docs.lium.io/developers/sdk/reference/models/ssh-key
- 6: https://otoyinc.mintlify.app/api-reference/ssh-public-keys/create-ssh-public-key
- 7: Datura-ai/lium-io@b840348
- 8: DAH-2205: Move rental Docker hot path to SDK over SSH Datura-ai/lium-io#1079
Require ssh_public_key_file when building Lium capacity schedulers.
build_lium_capacity_scheduler() fail-closes on a missing training API key, but a missing ssh_public_key_file only falls back to LiumCapacityScheduler’s class default ("ssh-ed25519 AAAA capacity-scheduler",), which is a non-functional placeholder key. Require lium_training.ssh_public_key_file whenever lium_training.enabled=True; also split the file into non-empty lines so multi-key files are not sent as one malformed blob.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/base/compute/lium_training_wiring.py` around lines 92 - 140, Update
build_lium_capacity_scheduler to fail closed when enabled training configuration
lacks lium_training.ssh_public_key_file, rather than allowing the scheduler’s
placeholder key default. Load the configured file, split its contents into
non-empty lines, and pass the resulting keys through ssh_public_keys; reject
missing or empty key-file content.
There was a problem hiding this comment.
🧹 Nitpick comments (1)
packages/challenges/prism/src/prism_challenge/config.py (1)
401-403: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd a regression test for the checkpoint repository default.
The existing config test covers model construction but does not assert that
checkpoint_repo_idusesDEFAULT_CHECKPOINT_REPO_IDor that either validation alias still overrides it. Add both assertions to protect this configuration contract.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/challenges/prism/src/prism_challenge/config.py` around lines 401 - 403, Add regression coverage in the existing configuration test for the checkpoint repository field: assert that constructing the config without either environment alias uses DEFAULT_CHECKPOINT_REPO_ID, and assert that both PRISM_CHECKPOINT_REPO_ID and PRISM_HF_CHECKPOINT_REPO_ID override the default. Keep the tests focused on the checkpoint_repo_id field and its AliasChoices configuration.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@packages/challenges/prism/src/prism_challenge/config.py`:
- Around line 401-403: Add regression coverage in the existing configuration
test for the checkpoint repository field: assert that constructing the config
without either environment alias uses DEFAULT_CHECKPOINT_REPO_ID, and assert
that both PRISM_CHECKPOINT_REPO_ID and PRISM_HF_CHECKPOINT_REPO_ID override the
default. Keep the tests focused on the checkpoint_repo_id field and its
AliasChoices configuration.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: e27b0254-9104-4b21-8941-5e70444251a8
📒 Files selected for processing (16)
packages/challenges/prism/src/prism_challenge/config.pypackages/challenges/prism/src/prism_challenge/evaluator/checkpoint_publisher.pypackages/challenges/prism/src/prism_challenge/evaluator/pod_boot.pypackages/challenges/prism/src/prism_challenge/queue.pypackages/challenges/prism/tests/test_checkpoint_upload_gate.pypackages/challenges/prism/tests/test_config.pypackages/challenges/prism/tests/test_pod_boot.pypackages/challenges/prism/tests/test_prism_checkpoint_publisher.pysrc/base/compute/lium.pysrc/base/compute/lium_capacity.pytests/unit/test_compute_lium_client.pytests/unit/test_lium_capacity_scheduler.pytests/unit/test_lium_training_wiring.pytests/unit/test_master_entrypoint_prism_policy.pytests/unit/test_orchestration.pytests/unit/test_orchestration_lium.py
💤 Files with no reviewable changes (2)
- tests/unit/test_lium_training_wiring.py
- packages/challenges/prism/tests/test_prism_checkpoint_publisher.py
🚧 Files skipped from review as they are similar to previous changes (12)
- tests/unit/test_master_entrypoint_prism_policy.py
- packages/challenges/prism/tests/test_checkpoint_upload_gate.py
- packages/challenges/prism/tests/test_pod_boot.py
- packages/challenges/prism/src/prism_challenge/evaluator/checkpoint_publisher.py
- packages/challenges/prism/tests/test_config.py
- tests/unit/test_lium_capacity_scheduler.py
- tests/unit/test_compute_lium_client.py
- src/base/compute/lium.py
- packages/challenges/prism/src/prism_challenge/evaluator/pod_boot.py
- tests/unit/test_orchestration.py
- src/base/compute/lium_capacity.py
- tests/unit/test_orchestration_lium.py
2 participants
Summary
mainplagiarism_adjudicatoron main (main-newer); companionapp.pymapsCheckpointPublishError→ 502Changes
lium_capacity,lium_orphan,lium_training_wiring, training GPU lock onlium.pyLiumTrainingSettings, prism dispatch variant, CLI wiringpod_boot, checkpoint upload gate + intake observability, worker-plane queue no-opmaster-entrypoint.shworker-plane / plagiarism LLM defaultsEvidence (T2 ports + T3 PR)
.omo/evidence/attested-isolated-platform/T2-landmine-ports/SUMMARY.mdbase-master-prod-base-master-validator-1(image0c9275e2…)feat/landmine-portsTest plan
pytestlandmine suite (lium_*, orchestration_lium, prism pod_boot/queue/checkpoint, entrypoint policy) — 275 passeddocker inspectshows only intentional mountsCI
Notes
0c9275e2…would break prod.feat/agent-challenge-no-phaladirty tree (worktree isolation)./app/packages/...andsite-packages/...— both must drop together.alpha1122x/base(org direct push denied for alpha1122x; merge via maintainer).Summary by CodeRabbit