From 5baccca156646629174b7c2ed2bdc7561bf9569a Mon Sep 17 00:00:00 2001 From: Bo Date: Tue, 4 Aug 2026 23:59:42 -0400 Subject: [PATCH 1/3] propose(contract): rebase evidence-based AGENTS trim onto current main --- AGENTS.md | 88 ++++++++++--------------------- evals/estate-ablation/PROPOSAL.md | 35 ++++++++++++ 2 files changed, 64 insertions(+), 59 deletions(-) create mode 100644 evals/estate-ablation/PROPOSAL.md diff --git a/AGENTS.md b/AGENTS.md index 131861563..94a08034d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -15,9 +15,7 @@ RPI -> Plan -> Implement -> fresh Validate -> report and stop ``` No fresh independent judgment over the exact subject means the experiment is -not proven. Persist `verdict.v2` only when the caller requests machine-readable -evidence or a declared downstream consumer requires it. AgentOps does not own -what the caller does next. +not proven. ## Authority and trust @@ -50,11 +48,9 @@ what the caller does next. ## Runtime floor -- Never run `claude -p` or `claude --print`, directly or indirectly. -- Default to native Codex plus the local shell. Start another runtime or - orchestration substrate only when the user explicitly requests it. -- Do not run `ao session bootstrap`, lookup, or archive commands as startup - ritual. The `ao` CLI is an explicit repository tool, not a session runtime. +Never run `claude -p` or `claude --print`, directly or indirectly (hook-enforced). +Default to native Codex plus the local shell; other runtimes only on explicit +request. `ao` is a repository tool, not a session ritual. ## Federated source authority @@ -83,50 +79,30 @@ Edit source owners and regenerate projections through the owning command. ## Constraint floor -- A plan, duel, or design becomes authoritative only if its inputs include the - active constraints that apply to its scope: relevant ADRs (`docs/adr/`), - blocking gates (`cli/internal/gates/`, `scripts/check-*.sh`), and this - contract. A synthesis frozen without an active constraint is invalid, not - grandfathered. -- Skill logic ships in Go via `ao`; skill scripts are thin POSIX shell glue. - No new `skills/*/scripts/**/*.py` (ADR-0016, enforced by - `scripts/check-skill-python-ratchet.sh`; tests keep their documented - exemption). Grandfathered Python is migration debt, not precedent. +Active constraints (ADRs in `docs/adr/`, blocking gates in +`cli/internal/gates/` and `scripts/check-*.sh`, this contract) are inputs to +any authoritative plan or design; a synthesis frozen without them is invalid. +Skill logic ships in Go via `ao`; no new skill Python (ADR-0016, gate-enforced). ## Standard RPI traversal -1. **Plan once.** Resolve the existing bead or caller intent and shape one - active behavior there. Acceptance, non-goals, scope, and the first useful - check stay in that source; AgentOps does not require a model-authored plan - packet that duplicates it. If no durable tracker artifact exists, the runtime - snapshots the resolved intent bytes under their content digest so fresh - contexts can consume the exact same source. Once the caller accepts the - shaped intent, Plan is closed: further planning, audit, or review lanes over - the same intent require new explicit caller authorization. -2. **Implement once.** Execute one bounded RED -> GREEN -> refactor experiment. - The runtime derives the content manifest, actual changed paths, coverage - completeness, and factual check receipts; the model does not transcribe a - candidate packet. -3. **Validate once, fresh.** A distinct context verifies the intent-source - digest, subject identity, scope, evidence, and acceptance, then returns one - `PASS | FAIL | NOT_PROVEN` result. Missing or colliding context identities, - unattested freshness, subject mutation, or incomplete changed-path coverage - is `NOT_PROVEN`; proven out-of-scope change is `FAIL`. PASS requires nonempty - checked scope, top-level evidence, and evidence for every criterion. - Persistence is conditional: Validate writes `verdict.v2` only for a caller - request or a declared downstream consumer. -4. **Report and stop.** RPI reports `PASS | FAIL | NOT_PROVEN`, or the report-only - statuses `NOT_PLANNED | NOT_BUILT`. It emits no next action and performs no - automatic revision. Two consecutive control artifacts (plans, audits, - reviews, prompts, reports) with no new implementation evidence end the run - — `NOT_BUILT` when no subject exists yet, otherwise a hard stop reporting - the existing subject's status; reports lead with the subject (paths - changed, commits, tests), never with artifact counts. - -A caller may revise the bead or caller intent and start a new invocation. -Changing acceptance changes that source; AgentOps does not create a parallel -revision packet. Learn is an optional later consumer of verdict collections and -cannot change core outcomes. +1. **Plan once.** Shape one active behavior in the existing bead or caller + intent — acceptance, non-goals, scope, first check. Once accepted, further + planning over the same intent needs new explicit authorization. +2. **Implement once.** One bounded RED -> GREEN -> refactor experiment; the + runtime derives the manifest, changed paths, and check receipts. +3. **Validate once, fresh.** A distinct context verifies subject identity, + scope, evidence, and acceptance: `PASS | FAIL | NOT_PROVEN`. Missing + identity/freshness or incomplete coverage is `NOT_PROVEN`; proven + out-of-scope change is `FAIL`. PASS requires evidence for every criterion + and an empty `not_checked`. Persist `verdict.v2` only for a caller request + or declared consumer. +4. **Report and stop.** Report the result; emit no next action; no automatic + revision. Two consecutive control artifacts with no new implementation + evidence end the run. Reports lead with the subject, never artifact counts. + +A caller may revise the intent and start a new invocation. Learn is an +optional later consumer and cannot change core outcomes. ## Product boundary @@ -174,13 +150,7 @@ AgentOps work ownership. ## Closeout Inspect the final subject, map acceptance to evidence, disclose `checked` and -`not_checked`, and obtain one fresh validation result over the exact content. -`not_checked` names in-scope acceptance surface that went unverified, so a PASS -has none by construction and any entry makes the result `NOT_PROVEN`. Scope -limits are disclosed, never deleted: a bounded proof of a criterion belongs in -that criterion's `reason`, a declared non-goal belongs to the intent source -(optionally restated as an evidence-backed boundary criterion), and residual -risk belongs in this report. Include a verdict reference only when persistence -was requested. Report residual risk plainly. Git status, pushing, merging, -release, and rollback are handled by the caller's repository policy, outside -semantic completion. +`not_checked` (any entry makes the result `NOT_PROVEN`; scope limits are +disclosed, never deleted), and obtain one fresh validation over the exact +content. Git, push, merge, release, and rollback belong to the caller's +repository policy. diff --git a/evals/estate-ablation/PROPOSAL.md b/evals/estate-ablation/PROPOSAL.md new file mode 100644 index 000000000..02f93feaf --- /dev/null +++ b/evals/estate-ablation/PROPOSAL.md @@ -0,0 +1,35 @@ +# Deletion proposal — repo operating contract (DRAFT, Bo ratifies) + +> Evidence base: 80 scored runs across 5 executors (luna, terra, sol-config, +> Opus 5, Fable) — sweeps 1/2/4 + tier-2 pilot + probe waves. Every verdict +> cites its measurement. This PR edits `AGENTS.md` per those verdicts and +> merges only on Bo's explicit ratification. The study supported a ~55% cut +> against its 2026-08-05 subject, but current `main` has since added mandatory +> anti-ceremony, federated-authority, and canonical-source contracts outside +> that experiment. This rebased proposal keeps those additions, yielding a +> narrower ~21% current-main reduction. The separate historical Codex sweep +> reported +8-12% tokens and no execution delta within its bounded design; it +> is not a claim of current skill-probe coverage. + +| Section | Verdict | Evidence | Action | +|---|---|---|---| +| Intro + loop diagram | KEEP | orientation; 3 lines | trim | +| Authority and trust | **SURVIVOR** | Opus/Fable t03/t04 applied its rules mid-execution (sweep 4); the one contract surface observed changing behavior | keep, light trim | +| Honest work and anti-ceremony | POST-STUDY CONTRACT | added on current main after the measured subject | keep current-main text | +| Runtime floor | ENFORCED | no-claude-p hook carries it; prose never observed load-bearing | shrink to pointer | +| Federated source authority | POST-STUDY CONTRACT | current federated ownership boundary | keep current-main text | +| Source precedence | KEEP (cheap) | orchestrator-facing, 4 lines | keep | +| Constraint floor | ENFORCED | python-ratchet gate + ADR carry it | shrink to pointer | +| Core loop | PRODUCT | dead for executors (sweeps 1-2: zero effect on GPT-line); alive as orchestrator spec (this session ran it all day) | compress ~50% | +| Product boundary | PRODUCT | boundary behavior; sweep-4 scope discipline consistent with it | keep current-main federated/factory rules | +| Concurrency | MOSTLY-NATIVE | rt-01: correct collision handling unaided; current rule also constrains delegation authority | keep current-main rule | +| Triggered-sources table | REQUIRED INDEX | `validate-agents-split` requires live links to CI and Codex contracts | keep current-main table | +| Closeout | PRODUCT | validate flow carries mechanics | compress | + +Rebased net: ~10.5KB → ~8.3KB (-21%). Compressions reword older measured +sections; post-study doctrine and live contract links remain intact. +Follow-ups (separate PRs, not this one): (1) codex adapter scopes skill +injection by task class (sweep-1 receipt); (2) closure-artifact formats that +carry caveats (sweep-4 disclosure-lives-in-chat finding); (3) S3 rules-file +lever unmeasured — rules stay untouched (wave-1 measured their INLINE form +BEHAVIORAL; do not delete unmeasured surface). From 7ba47792d3c20ccaaed0b57ac31ee01b4efa17a1 Mon Sep 17 00:00:00 2001 From: Bo Date: Sun, 16 Aug 2026 15:18:27 -0400 Subject: [PATCH 2/3] docs(evals): qualify legacy estate evidence --- evals/estate-ablation/PROPOSAL.md | 79 ++++++++++++++++++------------- 1 file changed, 47 insertions(+), 32 deletions(-) diff --git a/evals/estate-ablation/PROPOSAL.md b/evals/estate-ablation/PROPOSAL.md index 02f93feaf..78cf158a5 100644 --- a/evals/estate-ablation/PROPOSAL.md +++ b/evals/estate-ablation/PROPOSAL.md @@ -1,35 +1,50 @@ -# Deletion proposal — repo operating contract (DRAFT, Bo ratifies) +# Deletion proposal — repo operating contract (DRAFT; merge authorized after repair) -> Evidence base: 80 scored runs across 5 executors (luna, terra, sol-config, -> Opus 5, Fable) — sweeps 1/2/4 + tier-2 pilot + probe waves. Every verdict -> cites its measurement. This PR edits `AGENTS.md` per those verdicts and -> merges only on Bo's explicit ratification. The study supported a ~55% cut -> against its 2026-08-05 subject, but current `main` has since added mandatory -> anti-ceremony, federated-authority, and canonical-source contracts outside -> that experiment. This rebased proposal keeps those additions, yielding a -> narrower ~21% current-main reduction. The separate historical Codex sweep -> reported +8-12% tokens and no execution delta within its bounded design; it -> is not a claim of current skill-probe coverage. +> Historical record: the estate-ablation artifacts report 80 scored runs across +> five executor configurations (luna, terra, sol-config, Opus 5, and Fable) +> spanning sweeps 1/2/4, the tier-2 pilot, and probe waves. Those legacy runs +> predate the current evidence contract. Their aggregate count, labels, and +> section classifications are legacy-unverified and non-promotable: they may +> motivate hypotheses, but they do not establish causal effects, current +> behavior, or generalize across models or runtimes. Contemporaneous citations +> do not promote those reports under the current contract. +> +> This PR edits `AGENTS.md` while retaining every current mandatory contract. +> Bo has explicitly authorized merge after these bounded honesty corrections +> and required checks pass. This repair leaves the PR in draft; readiness and +> merge remain separate caller actions. +> +> The historical proposal reported a ~55% cut against its 2026-08-05 subject. +> Current `main` has since added mandatory anti-ceremony, federated-authority, +> and canonical-source contracts, so this rebased diff is a narrower ~21% +> reduction. Those percentages compare document size, not behavior. A separate +> historical Codex sweep reported +8–12% tokens and no execution delta within +> its bounded design; that report is also legacy-unverified and non-promotable. +> It proves neither a generalized token cost nor an execution effect and is not +> current skill-probe coverage. -| Section | Verdict | Evidence | Action | -|---|---|---|---| -| Intro + loop diagram | KEEP | orientation; 3 lines | trim | -| Authority and trust | **SURVIVOR** | Opus/Fable t03/t04 applied its rules mid-execution (sweep 4); the one contract surface observed changing behavior | keep, light trim | -| Honest work and anti-ceremony | POST-STUDY CONTRACT | added on current main after the measured subject | keep current-main text | -| Runtime floor | ENFORCED | no-claude-p hook carries it; prose never observed load-bearing | shrink to pointer | -| Federated source authority | POST-STUDY CONTRACT | current federated ownership boundary | keep current-main text | -| Source precedence | KEEP (cheap) | orchestrator-facing, 4 lines | keep | -| Constraint floor | ENFORCED | python-ratchet gate + ADR carry it | shrink to pointer | -| Core loop | PRODUCT | dead for executors (sweeps 1-2: zero effect on GPT-line); alive as orchestrator spec (this session ran it all day) | compress ~50% | -| Product boundary | PRODUCT | boundary behavior; sweep-4 scope discipline consistent with it | keep current-main federated/factory rules | -| Concurrency | MOSTLY-NATIVE | rt-01: correct collision handling unaided; current rule also constrains delegation authority | keep current-main rule | -| Triggered-sources table | REQUIRED INDEX | `validate-agents-split` requires live links to CI and Codex contracts | keep current-main table | -| Closeout | PRODUCT | validate flow carries mechanics | compress | +| Section | Current basis | Action | +|---|---|---| +| Intro + loop diagram | current canonical one-pass RPI orientation | trim | +| Authority and trust | current mandatory trust boundary; the legacy `SURVIVOR` classification remains a non-promotable hypothesis | keep, light trim | +| Honest work and anti-ceremony | current mandatory contract added after the historical subject | keep current-main text | +| Runtime floor | current no-`claude -p` guard plus an explicit runtime boundary | shrink without weakening the prohibition | +| Federated source authority | current federated ownership boundary | keep current-main text | +| Source precedence | current source-authority contract | keep | +| Constraint floor | current Python ratchet, ADR, and active-constraint contract | shrink to enforceable summary | +| Core loop | current standard RPI traversal; legacy sweeps do not establish executor effect | compress without changing semantics | +| Product boundary | current ownership and factory boundary; legacy associations do not establish causation | keep current-main rules | +| Concurrency | current delegation and writer-isolation boundary; the legacy rt-01 report is non-promotable | keep current-main rule | +| Triggered-sources table | current `validate-agents-split` index contract | keep current-main table | +| Closeout | current exact-subject, evidence, and fresh-validation contract | compress without changing semantics | -Rebased net: ~10.5KB → ~8.3KB (-21%). Compressions reword older measured -sections; post-study doctrine and live contract links remain intact. -Follow-ups (separate PRs, not this one): (1) codex adapter scopes skill -injection by task class (sweep-1 receipt); (2) closure-artifact formats that -carry caveats (sweep-4 disclosure-lives-in-chat finding); (3) S3 rules-file -lever unmeasured — rules stay untouched (wave-1 measured their INLINE form -BEHAVIORAL; do not delete unmeasured surface). +Rebased net: ~10.5KB → ~8.3KB (-21%). The compression is a current design +proposal; legacy reports supply hypotheses, not proof. Post-study doctrine and +live contract links remain intact. + +Follow-ups are separate hypotheses, not findings or work authorized by this PR: +(1) evaluate task-class-gated Codex skill injection suggested by the legacy +sweep-1 report; (2) evaluate closure formats that carry caveats suggested by +the legacy sweep-4 report; (3) leave S3 rules unchanged because they remain +unmeasured under the current contract—the historical wave-1 `BEHAVIORAL` label +is legacy-unverified and non-promotable. From 1d3f54ecfdce9644ebb6f4b7a4ce4797cb6e85e6 Mon Sep 17 00:00:00 2001 From: Bo Date: Sun, 16 Aug 2026 15:26:12 -0400 Subject: [PATCH 3/3] docs(contract): preserve validation and ratchet semantics --- AGENTS.md | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 94a08034d..e38882368 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -48,7 +48,7 @@ not proven. ## Runtime floor -Never run `claude -p` or `claude --print`, directly or indirectly (hook-enforced). +Never run `claude -p` or `claude --print`, directly or indirectly. Default to native Codex plus the local shell; other runtimes only on explicit request. `ao` is a repository tool, not a session ritual. @@ -82,7 +82,8 @@ Edit source owners and regenerate projections through the owning command. Active constraints (ADRs in `docs/adr/`, blocking gates in `cli/internal/gates/` and `scripts/check-*.sh`, this contract) are inputs to any authoritative plan or design; a synthesis frozen without them is invalid. -Skill logic ships in Go via `ao`; no new skill Python (ADR-0016, gate-enforced). +Skill logic ships in Go via `ao`; no new `skills/*/scripts/**/*.py`. +Skill tests retain their documented exemption (ADR-0016, gate-enforced). ## Standard RPI traversal @@ -92,11 +93,13 @@ Skill logic ships in Go via `ao`; no new skill Python (ADR-0016, gate-enforced). 2. **Implement once.** One bounded RED -> GREEN -> refactor experiment; the runtime derives the manifest, changed paths, and check receipts. 3. **Validate once, fresh.** A distinct context verifies subject identity, - scope, evidence, and acceptance: `PASS | FAIL | NOT_PROVEN`. Missing - identity/freshness or incomplete coverage is `NOT_PROVEN`; proven - out-of-scope change is `FAIL`. PASS requires evidence for every criterion - and an empty `not_checked`. Persist `verdict.v2` only for a caller request - or declared consumer. + scope, evidence, and acceptance: `PASS | FAIL | NOT_PROVEN`. Missing or + colliding context identities, unattested freshness, subject mutation or + digest mismatch, and incomplete changed-path coverage are `NOT_PROVEN`; + proven out-of-scope change is `FAIL`. PASS requires nonempty checked scope, + top-level evidence, evidence for every criterion, and an empty + `not_checked`. Persist `verdict.v2` only for a caller request or declared + consumer. 4. **Report and stop.** Report the result; emit no next action; no automatic revision. Two consecutive control artifacts with no new implementation evidence end the run. Reports lead with the subject, never artifact counts.