Working state for claude/spec-kit-verification-review-1p4fnn. Written so another session can pick
this up without replaying the conversation. Kept current as work lands.
Deliberately at the repository root, not under docs/: it is operational, not a site page. Files
under docs/ are bound by the house style and the prose ratchets, and a handoff should not have to
satisfy them.
Last updated 2026-08-15, after the third revision of the ADR brief.
cd tests && python -m unittest discover -s . -q
Run from inside tests/. From the repository root the imports fail and it looks like a broken
suite. One session lost four commits to that misreading.
Expected result: 173 tests, 5 failures. All 5 are test_word_copy_tracks_the_markdown failing
because pandoc is not installed in this container. They predate all work on this branch. Any
sixth failure is yours.
The three prose ratchets in tests/test_prose_rules_hold.py are at their baselines exactly:
| Metric | Now / baseline | Headroom |
|---|---|---|
| Sentences over 30 words | 180 / 181 | 1 |
| Table cells over 40 words | 19 / 19 | 0 |
| Paragraphs over 300 chars | 88 / 88 | 0 |
So any new page must add zero fat paragraphs and zero fat cells, or the suite goes red. Fix
by rewriting shorter, never by splitting a paragraph in two, and never by deleting a measurement:
PD-1 through PD-7 outrank the length rules.
A scratch measuring script for a single file was useful and is gone with the container. It read
FRAMEWORK_SECTIONS, the BANNED patterns and the baselines out of the test module and reported
per-file counts. Rebuilding it takes about twenty lines.
Eight commits, all pushed. git log --grep='Claude-Session: .*01VwhjHSMA753Yv54BPoNHmb' lists them.
| Landed | State |
|---|---|
docs/FRAMEWORK-spec-kit.md |
Live in nav under "Other frameworks" |
docs/FRAMEWORK-bmad.md |
Live in nav under "Other frameworks" |
docs/CASE-STUDY-refuted-but-true.md |
Live in nav under "In practice" |
HS-21 in docs/HOUSE-STYLE.md |
Issued; numbering table moved to HS-22 |
tests/test_a_series_answers_one_set_of_questions.py |
Gates HS-21. 9 tests |
docs/SEQUENCE-ALLOC.md |
Two additions, see section 3 |
tests/_ccxtest.py FRAMEWORK_SECTIONS is the only definition of the section order. A new
framework page must carry all nine ## sections in that order. Extra sections are allowed; a
missing one is refused, because an absent section reads as "this framework has no such property"
rather than "nobody established it". Write "not established" in the section instead.
docs/_data/nav.yml carries the same list as a note for whoever writes the next page, and the test
asserts the note still matches the constant. It caught its own first drift on its first run.
Do not re-derive these. Each was measured, not read about.
Spec Kit resolves feature state per worktree, until the worktree has no .specify/.
Find-SpecifyRoot walks parent directories looking for a .specify/ directory and consults git
nowhere. A worktree whose branch predates specify init therefore binds to the first ancestor
holding one. Two such worktrees nested under a primary both bound to the primary's feature.json;
the second overwrote the first, git status stayed clean in both, and nothing reported it.
BMAD's work-item state has no lock. _atomic_write in sprint_status.py is crash-atomic and
not concurrency-safe: _load_document reads at line 455, _atomic_write writes at line 561, and no
Python file in the install contains flock, fcntl, lockf, O_EXCL or a lockfile. The file is
committed rather than gitignored, so cross-worktree divergence surfaces as a merge conflict. The
residual window is two writers in one worktree.
BMAD's project-root resolution is fail-closed, which is the one place it is plainly better than
Spec Kit. load_central_config requires _bmad/config.toml and does not walk parents, so the case
that silently mis-binds Spec Kit produces a named error.
/speckit-converge measures artifact drift, not implementation completeness. Reported from one
v0.16.4 build on 2026-08-15: 22 requirements, 65 tasks, 85 tests through the full flow, then converge
three times. 24 findings, 2 changed application code, and none was a feature that did not work. The
loop terminates when the artifacts stop moving, not when the code becomes right. This corrected live
text on the Spec Kit page, which had repeated the command's own description.
Spec Kit's feature numbering is still unlocked at 0.16.4. Get-HighestNumberFromSpecs takes the
highest prefix and adds one. This re-verifies the claim docs/SEQUENCE-ALLOC.md already carried
from a 2026-08-12 reading.
Two projects reached the same ADR layout independently. examples/sequence-adr/ numbers records
at docs/adr/NNNN-slug.md; the panaversity/spec-kit-plus fork uses history/adr/NNNN-slug.md.
The fork settles scope the same way and leaves concurrency open.
The two ADR extensions, read from extensions/catalog.community.json on 2026-08-15. adrkit
requires speckit_version >=0.13.0,<0.16.0, so it is not certified for 0.16.x, and needs a separate
adr CLI. arch-governance requires >=0.1.0 and needs Python 3.11+ and uv. Both are
verified: false, 0 downloads, 0 stars.
What the originating team decided, reported 2026-08-15. Six decision records written by hand at
docs/adr/NNNN-title.md, statuses Accepted and Superseded only, body frozen above
## Amendments, each carrying a line naming what it is evidence for. About an hour, no
dependencies.
adrkit ruled out. arch-governance recommended, then declined on duplication rather than
quality: the team already ran a control scorecard whose cells carry a rationale, a last-verified
date, a commit sha, the review method, and anchors pinning the exact source text a citation must
still match, and every one of the extension's six checks had a counterpart in it.
That reversal is the most transferable thing in the brief and is now on the Spec Kit page. A Spec-Kit-shaped hole is not always a hole in your organisation, and a second convention beside a mature one is this repository's own subject at organisational scale.
One design property worth carrying separately. adrkit is effect: read-write and drafts
records from the plan artifact the agent just wrote. Where the record is the deliverable, generating
them at volume yields a trail that looks complete and attests to nothing. A gap is honest; a hollow
record is a claim. The same line separates a convention from evidence: "the body above
## Amendments is frozen" is a statement about behaviour, and a check that the body is unchanged
since first commit is proof of it.
a. BMAD's remaining two gaps. The page is now in the nav and its rows are filled from the research journal. What stays unestablished and says so: the documented persona order, and whether the activation-cost figures in upstream issue #1343 apply to 6.11.0, since that issue names no version and the skill consolidation postdates it.
b. The compliance mapping, held deliberately. Material was supplied claiming OWASP ASVS 5.0,
NIST SSDF, HIPAA/HITRUST, ISO 27001 and SOC 2 each require documented design rationale that ADRs
satisfy. The framing is sound; the control identifiers are unverified and are what an auditor
checks. Two are known risks: ASVS 5.0 reorganised its chapters, so a V1 citation carried over
from 4.0 may be stale, and ISO 27001's secure engineering control moved to A.8.27 in the 2022
revision from A.14.2.5 in 2013.
Its own author marks it unverified, in the source brief: the framing "came from the sending project's owner and was taken as given... nothing in this research verified it". Do not publish it as established on the strength of having been handed it twice.
Verification was attempted from this container on 2026-08-15 and is blocked. owasp.org is
refused by the network egress proxy (EGRESS_BLOCKED), so the ASVS chapter structure cannot be read
from the canonical source here. A guessed path into the OWASP/ASVS GitHub repository returned 404.
Do not repeat those two attempts. Either find the correct path inside that repository, which is
reachable, or verify from a machine without the egress restriction.
The context behind it, which is not itself a claim needing verification: the sending project is a pilot for building ASVS L3 applications in healthcare, so the record is the deliverable rather than documentation about the deliverable.
It belongs in
secure-development-standards, which owns
standards content and ships the ASVS 5.0 assessment. Publishing a mapping here would be the drift
the repository split exists to prevent. Cite that site by name and never by anchor:
CrossRepositoryAnchorsAreRefused bans #fragment links into it, because neither suite can check
the other's headings.
The Ultracode "bug" was not a bug. Four reports were drafted and all four were wrong. The
control is a six-position Effort slider whose top notch is Ultracode, sitting above Max, and it
is a separate control from the model picker. Official documentation states Ultracode "combines
xhigh reasoning effort with automatic workflow orchestration", so CLAUDE_EFFORT=xhigh while
Ultracode is on is documented and intended. Paired readings taken in the same turn as each mode
reminder: Ultracode gives xhigh, Max gives max, and MAX_THINKING_TOKENS is 31999 at both.
The keyword path is a separate, single-turn opt-in that does not change session effort, which is why it stays silent when the session setting is already on. Everything observed was correct behaviour.
The real lesson was methodological and is worth carrying: two observation channels that cannot be time-aligned cannot support an inference. The screenshots and the mode reminders were sampled at different moments, and every wrong report came from comparing them as though they were simultaneous.
Related and still open upstream, filed by someone else: #75732, open since 2026-07-08 with no maintainer response.
Verification kill lists are not established fact. docs/CASE-STUDY-refuted-but-true.md records
two audited kills from one research pass. One was a plain false negative on a claim the primary
source confirms verbatim. The other was a compound claim whose true conjunct was destroyed along
with its unsupported one. Both trace to a verdict field narrower than the thing being judged. If you
run adversarial verification here, use a three-valued verdict and split claims into atomic
assertions first.
- Prose is ASCII. No em dash, no smart quotes.
HS-11, and the pattern is easy to reintroduce by pasting. - A markdown link sits on one line, text and target both.
HS-16outranks the 100-character wrap, because a wrapped link is served as raw markdown by the published site. - Search before renaming a heading.
HS-13. Pages cite headings by name and link by anchor. - A count in prose needs the set enumerated on the same page.
HS-5. This is the rule new pages break most often. - No PR has been opened, and the owner asked for none. The branch is pushed and reviewed directly.