Tell extraction agents to namespace the files they write - #2729
Tell extraction agents to namespace the files they write#2729duncancrawford wants to merge 1 commit into
Conversation
The spec guarantees chunk outputs cannot collide, because CHUNK_PATH carries the chunk number. It says nothing about the helper scripts agents write to produce them, so two agents running concurrently independently chose the same path and one overwrote the other. The interesting part is what happens next: the agent that lost then executed the other agent's script, and wrote that agent's chunk numbers instead of its own. Both writes succeed, so nothing errors - the output is simply attributed to the wrong chunk. It was caught only because the affected agents noticed two of their chunks were missing afterwards and re-ran them. Applied to all 14 skill variants, in each one's own register - the fuller wording where the spec is discursive, a one-line RULE where it is terse.
![graphify-labs[bot]](https://avatars.githubusercontent.com/in/4356270?s=60&v=4){kind=small}
There was a problem hiding this comment.
Graphify reviewed this change.
Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).
Graphify review — findings
This PR appends a new instruction paragraph to the extraction-spec.md reference files across all agent skill variants (agents, amp, claude, claw, codex, copilot, droid, kilo, kiro, and others). The added text instructs extraction subagents to include their chunk number in the names of any scratch/helper files they write (e.g. scratchpad/gen_CHUNK_NUM.py), with a rationale about concurrent agents sharing a filesystem. Two phrasings are used depending on the file: a longer version for the full-spec variants and a compact version for the compact variants. The change is documentation/prompt-text only across many parallel copies.
No blocking issues surfaced.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 28 functions depend on the 28 functions this change touches.
Health — grade A; no new coupling hotspots.
Verification — 28 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 28 function(s) in the blast radius were not formally verified this run
1 participant
The extraction spec guarantees chunk outputs cannot collide, because
CHUNK_PATHcarries the chunk number. It says nothing about the helper files agents write on the way there — so two agents running concurrently independently chosescratchpad/gen.py, and one overwrote the other.Why this is worth two lines rather than tolerating
The overwrite is not the interesting part. The agent that lost then executed the other agent's script, and wrote that agent's chunk numbers instead of its own.
Both writes succeed. Nothing errors. The output is simply attributed to the wrong chunk — and on a large fan-out that is indistinguishable from correct output until someone counts. It was caught only because the affected agents noticed two of their own chunks were missing afterwards and re-ran them.
Encouragingly, two agents hit this independently, both diagnosed it correctly and both worked around it unprompted — one moved to
gen_b032.py, the other to a per-batch directory. That they converged on the same fix without being told is the argument that the guidance belongs in the spec: the agents were not confused, the spec was silent.The change
One paragraph, immediately after the
CHUNK_PATHinstruction it belongs beside:Applied to all 14 skill variants, each in its own register — the fuller wording where the spec is discursive, a one-line
Scratch file RULEin the four terser variants, matching their existingsource_file RULEstyle.Documentation only. No behaviour change, no code touched.
Context
Found while running the semantic extraction fan-out on a large corpus. Happy to reword if the phrasing does not suit the house style — the substance is just that a shared filesystem needs a per-agent namespace for anything an agent writes.