Repository navigation
eval: Add offline readiness checks and opt-in arm bundles - #87
cgwalters-bot wants to merge 1 commit into
Conversation
## Executive summary Add an offline allocation/enforcement evidence checker and complete opt-in evaluation bundles through the shared tier resolver, leaving normal routing unchanged. This makes the first readiness slice usable without inventing capacity or launching measurements. Part of cgwalters-forge#52 The existing [report](eval/report.md#current-runner-contract-and-external-prerequisites) records the pinned runner contract and a concrete external reservation/enforcement follow-up. The measured comparison remains blocked at 0/54 cells: authenticated capacity/reservations, hard child-inclusive limits, frozen tasks, actual role/model execution and cache evidence remain unproved. No inference or extra jobs were launched. Verification: `just check` passes all 66 tests; the archived-summary test matches `summarize-eval` output. Initial diagnostic/number-spelling test failures and whitespace findings were fixed. Review: one adversarial pass, no blockers; follow-ups are stricter artifact URL syntax and independent assertions of the full built-in agent set, plus a no-inference configuration-load test when available. ```checkpoint {"reason":"incomplete","done":"Offline readiness checker, shared-resolver arm bundles, tests and pinned contract report are implemented and reviewed.","left":"Authenticated capacity and reservation evidence, enforceable aggregate/per-cell limits, execution/cache proof, frozen tasks and 54 measured cells.","next":"Resolve the external accounting/enforcement follow-up documented in eval/report.md; independently validate evidence before any inference, then continue the protocol."} ``` Assisted-by: https://github.com/cgwalters/cgwalters#llms
|
Continuation round 1 of 2 for Generated-by: https://github.com/cgwalters/#llms |
⏸️ [dispatch](https://github.com/cgwalters-forge/tracker/actions/runs/43758959261) is blocked: it handed back missing_data and no change. Its reason:
|
|
Fresh continuation round 3, after 2, for Generated-by: https://github.com/cgwalters/#llms |
|
The poll stopped following this pull request: continuation round 3 is blocked: its run https://github.com/cgwalters-forge/tracker/actions/runs/43528708021 ended (success) and pushed no new head for 3b28d0c (its checkpoint says incomplete). It is labelled Generated-by: https://github.com/cgwalters/#llms |
## Executive summary Move Check's tool provisioning into ordinary pinned, checksum-verified, bounded job-local setup and remove the Rust installer and its CLI/Justfile interfaces. Keep offline eval and configuration behavior intact. Part of #76 The refreshed caller audit is in README.md: no active external installer dependency was found in current tracker, agentic-job or agent-board default branches; deployed and other-branch callers remain unverified. No cross-repository changes or dispatch installation were made. Eval relocation and final Cargo/tracker package removal remain later slices; PR #87 and its human-approval hold are untouched. The audit is durable in README.md; this run cannot update the issue body's Findings link. Verified: just check, reference/dependency audit, and controlled cold-cache network provisioning with all version/navigation smoke checks and checksum-mismatch rejection before extraction. GitHub-hosted CI itself has not run here; no paid evals or hosted role probes ran. Review: one adversarial pass, no blockers; its download-size cap, test-only tempfile and stale constraint-comment findings were addressed afterward and rechecked; no follow-ups remain. Assisted-by: https://github.com/cgwalters/cgwalters#llms
|
Continuation round 1 of 2 for Generated-by: https://github.com/cgwalters/#llms |
|
Fresh continuation round 3, after 2, for Generated-by: https://github.com/cgwalters/#llms |
|
The poll stopped following this pull request: continuation round 3 is blocked: its run https://github.com/cgwalters-forge/tracker/actions/runs/43758959261 ended (success) and pushed no new head for 3b28d0c (its checkpoint says incomplete). It is labelled Generated-by: https://github.com/cgwalters/#llms |
Executive summary
Add an offline allocation/enforcement evidence checker and complete opt-in evaluation bundles through the shared tier resolver, leaving normal routing unchanged. This makes the first readiness slice usable without inventing capacity or launching measurements.
Part of #52
The existing report records the pinned runner contract and a concrete external reservation/enforcement follow-up. The measured comparison remains blocked at 0/54 cells: authenticated capacity/reservations, hard child-inclusive limits, frozen tasks, actual role/model execution and cache evidence remain unproved. No inference or extra jobs were launched.
Verification:
just checkpasses all 66 tests; the archived-summary test matchessummarize-evaloutput. Initial diagnostic/number-spelling test failures and whitespace findings were fixed.Review: one adversarial pass, no blockers; follow-ups are stricter artifact URL syntax and independent assertions of the full built-in agent set, plus a no-inference configuration-load test when available.