Skip to content

Commit ffd04b4

Browse files
kyle-sextonclaude
andauthored
fix(code-metrics): ground the principles skill in its primaries and add a skill-quality Next-section check (#4098)
Closes #4067 Closes #4066 ## Summary The `code-metrics:principles` skill was exercised through the Skill tool on this repository and every load-bearing citation checked against a primary. Seven findings came back (#4067): one factual overstatement baked into an eval, an ISO clause reading presented as the spec's words, a missing `## Next`, research narrative in two reference files, an ADR cited by number, no duplication entry, and a non-answer to the skill's most likely first question. The skill-quality gate passed the skill anyway because it has no check for the successor section (#4066). This PR fixes both, as code-metrics 0.2.2 and skill-quality 0.22.3, in one change set because the two fixes were found together and are each small. ## Fix code-metrics `principles`: - The Halstead quick-guide entry and eval 1 said difficulty "should not have" moved when a file was split. Difficulty `(n1/2) * (N2/n2)` carries no explicit length term, but both factors change per half on a split (a radon run gave 1.667 for the whole against 1.000 and 1.800 for the halves), so the entry, the measures.md paragraph, literature.md, and the eval now say per-file difficulty legitimately moves in either direction. - The ISO/IEC 5055 §8.2.115 percentage-of-file reading is labelled as the plugin's, with a four-part verification record (claim, basis, as-of 2026-09-11, recheck on a new OMG or ISO revision), in thresholds.md, measures.md, `reference/config.md`, and the provenance strings in `config-defaults.json`. The files name ISO/IEC 5055:2021 as the ISO publication of ASCQM v1.0, with v1.1 identical for the cited clauses, and record that the document's informative §6.3 Table 1 rows carry different defaults (1000 lines, 10%) from its detection patterns (5%, 90%). - The quick guide answers "which measure should I look at" with an intent-keyed tree, each branch grounded in its primary (McCabe 1976 and NIST SP 500-235 for cyclomatic, Campbell for cognitive, the ISO CWE-1080 and CWE-1041 patterns for size and duplication, and the mutation-testing presence gate for whether a suite would catch a fault, since the coverage-effectiveness literature is contested). A duplication entry and a duplication row in the thresholds table record that no reference ships and why; the routing table names the plugin's `report-schema.md` for the report vocabulary. - The no-verdict rule is stated once at the top; the ADR-by-number citation is gone; the operator's-list and research narrative in thresholds.md and literature.md is replaced by present-tense rows and per-source confidence with basis, per the skill-bodies rule, and the history moves to the 0.2.2 changelog entry. - `## Next` added in the sibling skills' shape. McCabe's "testable and maintainable" framing and Campbell's verbatim switch rule are carried in measures.md and literature.md; literature.md gains a coverage-and-test-effectiveness section citing the five primaries with DOIs. - The `file_lines` provenance sentence in `config-defaults.json` is carried verbatim by `audit-size/SKILL.md`, `principles/reference/thresholds.md`, and the `reference/config.md` row, which the audit-size test requires for the first two. skill-quality `check`: - Check 27 reads the `## Next` section: INFO when absent (226 of 258 skills here have none, and the rule leaves "has a successor" to the author), WARN when present but after `## Gotchas`, last in the file, or in neither the one-invocation nor the two-to-four-bullet shape. The single shape has to open with the invocation, and both shapes are read whole for Skill-tool, installed, fallback, and otherwise phrasing, so an operative chain dressed as a successor also warns. Seven cases in `check-skill.test.sh`; the gate stays advisory. Wording in the skill description, manifest, and generated catalog moves from twenty-six to twenty-seven checks. Every technical claim the skill now asserts was verified against fetched primaries by a dispatched research run and a separate fresh-context verifier, with Crossref confirmation of all seven paper citations; the record is in #4067's Evidence section. ## Verification - `check-skill.sh principles`: PASS, 0 errors, 0 warnings (was FAIL on the report-schema routing row until it was cited through `${CLAUDE_PLUGIN_ROOT}`, the sibling skills' form). `check-skill.sh audit-size`: PASS. `check-skill.sh check`: PASS, 0 errors, 1 pre-existing line-count warning. - `check-skill.test.sh`: all assertions passed, including the seven `## Next` cases and main's new check-26 cases after the merge. `audit-size.test.sh`: 13 cases, 0 failed. - Every repo skill that carries a `## Next` section (32) passes the tightened check with no warning, and every other code-metrics skill passes with no `## Next` warning. - `check-evals-quality.sh` on the principles evals: PASS. `check-code-metrics-config-reference.py`: config.md documents all 25 keys. - `check-changelog-parity.sh` in `--check`, `--check-bump origin/main`, and `--check-preserved origin/main`: clean. `generate-catalog.mjs --check`: in sync. - `markdownlint-cli2` over every changed markdown file: 0 issues. `check-purged-em-dashes.sh`: 383 declared paths, no em dashes. `shellcheck` and `shfmt -d` on both shell files: clean. All edited JSON parses. - `scripts/affected-tests.sh --run` on the merged tree: every selected Bash suite passed except two. `audit-size.test.sh` failed on the provenance sentence, which a later commit fixes (re-run above). `plugins/claude-ops/skills/plugins/scripts/cache-content-check.test.sh` fails its two process-budget cases because `PS4` does not reach `bash -x` in this container at all (`PS4='+${BASHPID}+ ' bash -x -c true` prints a bare `+ true`); this PR touches no claude-ops file, the failure reproduces identically on a standalone re-run, #4091 recorded the same environmental failure with CI green, and CI on main is green. The 18 Python and Node suites the Bash runner lists as NOT RUN include four code-metrics suites (`test_report.py`, `test_resolve_config.py`, `test_setup_apply.py`, `test_check_code_metrics_config_reference.py`), run through pytest: 77 passed. - Review lane: three findings (the config.md row, the test block's check number, the measures.md Halstead paragraph), each fixed in its own commit. Security lane: no findings. Codex: one P2 finding on check 27's single-shape validation, fixed with two added test cases. ## Related - Refs #3848 (the per-skill verification pass; the principles leg's outcome is recorded there) and #3768 (the code-metrics follow-ups container #4067 sits under). - Refs #3993, the audit-complexity leg of the same pass, whose issue shape #4067 follows. - Research artifacts for the claim verification live in this session's memory tier and are summarized in #4067. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01T3rGcaMrtBbnQrKCkW6J1t --- _Generated by [Claude Code](https://claude.ai/code/session_01T3rGcaMrtBbnQrKCkW6J1t)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 3b273a2 commit ffd04b4

17 files changed

Lines changed: 537 additions & 159 deletions

File tree

docs/CATALOG.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -88,7 +88,7 @@ plugin manifests and kept in sync by CI. Never hand-edit it; the category vocabu
8888
- [`context-guard`](../plugins/context-guard): Per-session context-window observability plus the first shipped consumer: a statusline wrapper tees each session's context_window fields to a per-session snapshot file, a zone resolver classifies usage into smart/acceptable/dumb bands (percentage bands plus window-class token bands, conservative-min combination, zones.json SSOT with shipped defaults), a reader contract fixes how consuming sessions interpret the snapshots, and zone-crossing hooks report once per transition into a worse zone across two channels: the continuation menu to the operator, who owns that choice, and to the model only the zone determination plus the counter-steer that a zone word is not a decay signal (advisory by default; an optional blocking mode gates new mutating work on a fresh dumb-zone snapshot with handoff-writing exempt), with a PostCompact hook persisting an evidence-degraded marker.
8989
- [`context-budget`](../plugins/context-budget): Measure a Claude Code session's fixed startup context payload per item, on the consumer's machine at a pinned, version-stamped binary, including per-tool attribution of the built-in tool pools that /context reports only as lump sums, derived live by A/B bare-name-deny differencing with enforced comparability rules (skill-listing signature, one mode, one binary), an SDK-primary exact meter degrading to a version-aware headless /context parser and then to an honest structured error (never a wrong number), and a per-project measure-toggle-remeasure ledger under the plugin data directory recording every lever's real before/after delta. Report-only: prints exact config, applies nothing.
9090
- [`plugin-quality`](../plugins/plugin-quality): Post-use behavioral audit of Claude Code plugin components: a six-step audit workflow (evidence capture, grounded mapping in a fresh subagent, blindspot pass, interactive contract lock, presence-gated review seams, work-item emit with draft+confirm) over any skill, agent, hook, command, or config you have actually used, zone-informed by context-guard snapshots when present, conservative when not.
91-
- [`skill-quality`](../plugins/skill-quality): Skill-authoring QA tooling: a static contract checker that runs twenty-six deterministic checks over a Claude Code skill (frontmatter, explicit invocation mode, description/verb-contract polarity, per-skill listing-entry cap, trigger-keyword preservation, line caps, broken internal refs, markdownlint, gotchas surface, evals presence, precompute opportunity, completion-criteria signal, injection shell-declaration, fresh-eyes declaration conformance), a shared skill-listing budget reporter across a set of skills, and a bundled evals.json schema plus a deterministic eval-quality lint (duplicate case identities, missing fixtures, empty or vague grading criteria, set-coverage warnings). Runs against any repo's skills directory via the convention-resolution ladder, with no baked layout.
91+
- [`skill-quality`](../plugins/skill-quality): Skill-authoring QA tooling: a static contract checker that runs twenty-seven deterministic checks over a Claude Code skill (frontmatter, explicit invocation mode, description/verb-contract polarity, per-skill listing-entry cap, trigger-keyword preservation, line caps, broken internal refs, markdownlint, gotchas surface, evals presence, precompute opportunity, completion-criteria signal, injection shell-declaration, fresh-eyes declaration conformance), a shared skill-listing budget reporter across a set of skills, and a bundled evals.json schema plus a deterministic eval-quality lint (duplicate case identities, missing fixtures, empty or vague grading criteria, set-coverage warnings). Runs against any repo's skills directory via the convention-resolution ladder, with no baked layout.
9292
- [`computer-use`](../plugins/computer-use): Operating knowledge for Claude Code's built-in computer-use MCP server, the desktop screen-control surface. `/computer-use:diagnose` resolves a symptom to a cause instead of retrying: why every screenshot is downscaled to a fixed pixel budget and why zoom (not a bigger display) is the way back to detail, how to read a capture or input failure, and the per-OS quirks that make a synthesized key or menu behave unlike a human's. `/computer-use:setup` verifies the prerequisites the surface cannot verify for itself and reports the environment settings that end a session mid-run.
9393
- [`fleet`](../plugins/fleet): Run a Claude Code agent turn on another machine in a personal fleet, over SSH on the tailnet. Every machine signs into its own Claude account, so the built-in peer tools are same-account and never span machines; this carries the SSH path that does, with target resolution from the rendered fleet manifest, one-shot and multi-turn headless recipes, the Windows-side relay that reaches a target's own sessions, and the permission posture that keeps remote agent launches prompting.
9494
- [`instruction-placement`](../plugins/instruction-placement): Routes agent-instruction content to the surface that loads it at the right moment. The audit skill sweeps a repository's instruction layer and its ordinary markdown for content whose scope is narrower than the surface carrying it, meaning conventions keyed to one file type or one subtree sitting in an always-loaded CLAUDE.md or AGENTS.md, and for normative conventions stranded in documentation Claude never loads at all, then classifies each against a routing rubric and proposes a destination whose `paths:` glob is machine-validated before it is ever offered. Safety-class content (irreversible actions, secrets, data integrity, external publication, compliance, agent authority) is hard-denied from demotion and reported as held back rather than proposed, because demotion trades guaranteed presence for conditional presence and deferred surfaces are invisible inside subagents and absent after compaction until re-triggered. Every accepted move regenerates an always-loaded index of deferred surfaces, which is what keeps a demoted rule reachable from a subagent that never receives its injection. The audit is read-only and emits a diffable findings artifact; realignment is a separate skill gated per item with no blanket-approve path; a deterministic check skill gates that every rule glob still resolves and the index is current; and a setup skill verifies the one thing no other gate can see: that the index target is a file Claude Code will actually read, since it reads CLAUDE.md and not AGENTS.md.

plugins/code-metrics/.claude-plugin/plugin.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
33
"name": "code-metrics",
4-
"version": "0.2.1",
4+
"version": "0.2.2",
55
"description": "Read-only code measures for a change, with cited references and no verdict: lines per file (audit-size), cyclomatic, cognitive, and Halstead complexity (audit-complexity), duplication with sanctioned-replication exclusions (audit-duplication), coverage per function with CRAP from existing lcov, Cobertura, coverage.py, or Go artifacts (audit-coverage), type debt for TypeScript and Python (audit-type-debt), the literacy router for what each number can and cannot say (principles), and a setup skill for the consumer's .claude/code-metrics.yaml. Runs external collectors only when they already resolve, never installs, never runs tests, never emits a finding.",
66
"author": {
77
"name": "Melodic Software",

plugins/code-metrics/CHANGELOG.md

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,59 @@
33
All notable changes to the `code-metrics` plugin are documented here. Format follows
44
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning.
55

6+
## [0.2.2]
7+
8+
### Fixed
9+
10+
- **`principles`: the Halstead split-file answer overstated what the formula supports.** The
11+
quick guide said difficulty "should not have" moved when a file was split, and eval 1 expected
12+
the same. Difficulty `(n1/2) * (N2/n2)` carries no explicit length term, but both factors change
13+
per half on a split, and a radon run on two functions measured together and apart gave 1.667 for
14+
the whole against 1.000 and 1.800 for the halves. The entry and the eval now say per-file
15+
difficulty legitimately moves on a split, in either direction.
16+
- **`principles`: the §8.2.115 reading is labelled as the plugin's.** thresholds.md, measures.md,
17+
and the configuration reference presented "a function's non-empty lines as a percentage of the
18+
file's" as the clause's words. The clause states `MaxNumberOfNonEmptyLinesOfCode` with a default
19+
of "5%" and names no base; the percentage-of-file base is this plugin's reading, and thresholds.md
20+
now carries the four-part verification record for it (OMG ASCQM v1.1 and the ISO edition, as of
21+
2026-09-11, recheck on a new revision). The same files now name ISO/IEC 5055:2021 as the ISO
22+
publication of ASCQM v1.0, with v1.1 identical for the cited clauses, and note that the
23+
document's informative CWE summary rows carry different defaults (1000 lines per file, 10%) from
24+
its detection patterns (5%, 90%).
25+
- **`principles`: McCabe's framing and Campbell's switch rule are quoted as written.** McCabe 1976
26+
frames cyclomatic complexity for modules that are "testable and maintainable", not testability
27+
alone; Campbell v1.7 states "a switch and all its cases combined incurs a single structural
28+
increment". measures.md and literature.md carry both verbatim.
29+
30+
### Changed
31+
32+
- **`principles`: the quick guide answers "which measure should I look at" with an intent-keyed
33+
tree**, each branch grounded in its primary: testing burden to cyclomatic (McCabe; NIST SP 500-235
34+
sets the test count equal to it), readability to cognitive (Campbell), diff size and copying to
35+
lines per file and duplication (ISO/IEC 5055 CWE-1080 and CWE-1041), and whether a suite would
36+
catch a fault to the mutation-testing presence gate, because coverage records execution and the
37+
primary literature disagrees on how well it predicts fault detection (Inozemtseva and Holmes
38+
2014 against Gopinath, Jensen and Groce 2014 and Kochhar, Thung and Lo 2015). A duplication entry
39+
states that no reference ships and that the percentage moves with `duplication.min_tokens`, and
40+
the routing table names the plugin's report-schema reference for the report vocabulary.
41+
- **`principles`: the no-verdict rule is stated once**, at the top of the skill body, and the
42+
reference files no longer cite the marketplace's ADR by number, which a consumer of the installed
43+
plugin cannot read.
44+
- **`principles`: the reference files state present-tense facts and carry no research narrative.**
45+
The thresholds file's account of how ten candidate values were commissioned from a social post,
46+
scrutinized at an interview, and full-text searched is replaced by a table of popular numbers with
47+
no found source, naming what was checked and what was not; literature.md states each source's
48+
confidence and its basis without narrating the pass that established it. For the record, that
49+
candidate list was 22 (cyclomatic), 22 (cognitive), 80 (Halstead difficulty), 500 (lines per
50+
file), 100 (coverage), 25 (CRAP), and four zeros for count-based concerns; 20 and 1000 survived
51+
as shipped defaults because a citation exists for them, and the rest traced to no source.
52+
- **`principles`: a `## Next` section** names the audit skill for the measure in question and
53+
`/code-metrics:setup` for setting the reader's own reference values, in the mention-only shape
54+
the sibling skills use.
55+
- **`principles`: literature.md gains a coverage-and-test-effectiveness section** citing the five
56+
primaries above with their DOIs, and a duplication row in the thresholds table records that no
57+
duplication reference ships and why.
58+
659
## [0.2.1]
760

861
### Changed

plugins/code-metrics/reference/config.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -63,9 +63,9 @@ The third column is written by hand and is not derived from anything. A row whos
6363
| `complexity.cyclomatic.reference` | `20` | ISO/IEC 5055:2021 §8.2.117 (normative). Cited alternatives: 10 (McCabe 1976, "reasonable, but not magical") and 15 (NIST SP 500-235, with its six practices) |
6464
| `complexity.cognitive.reference` | `null` | Campbell, SonarSource; no standard sets a threshold |
6565
| `complexity.halstead.difficulty` | `null` | Halstead 1977; no standard sets a threshold |
66-
| `size.mode` | `file-lines` | `file-lines` compares each file's non-blank lines to `size.file_lines`; `iso-8.2.115` adds each function's non-empty lines as a percentage of the file's, from a collector that reports function ranges |
67-
| `size.file_lines` | `1000` | The plugin's own number. It coincides with an informative figure in ISO/IEC 5055:2021 §6.3 Table 1, which is not normative; 500, the operator-list figure, is selectable |
68-
| `size.function_lines_pct` | `5` | ISO/IEC 5055:2021 §8.2.115 (normative), used in `iso-8.2.115` mode |
66+
| `size.mode` | `file-lines` | `file-lines` compares each file's non-blank lines to `size.file_lines`; `iso-8.2.115` adds each function's non-empty lines as a percentage of the file's (the plugin's reading of the clause, which states 5% with no base), from a collector that reports function ranges |
67+
| `size.file_lines` | `1000` | The plugin's own number. It coincides with an informative figure in ISO/IEC 5055:2021 §6.3 Table 1, which is not normative; 500 is selectable |
68+
| `size.function_lines_pct` | `5` | ISO/IEC 5055:2021 §8.2.115 (normative) states the 5%; the percentage-of-file base is the plugin's reading; used in `iso-8.2.115` mode |
6969
| `duplication.min_tokens` | `50` | Passed to the clone collector |
7070
| `duplication.min_lines` | `5` | Passed to the clone collector |
7171
| `duplication.ignore` | `[]` | Collector ignore globs |

plugins/code-metrics/scripts/config-defaults.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -65,14 +65,14 @@
6565
"config_key": "size.file_lines",
6666
"value_key": "lines_non_blank",
6767
"direction": "at_or_above",
68-
"provenance": "the plugin's own number, not ISO-backed: it coincides with the informative figure in ISO/IEC 5055:2021 §6.3 Table 1, which is not normative, while the normative form (§8.2.115) is a function-level percentage; 500 is the operator-list figure, selectable; size.mode iso-8.2.115 selects the normative function-percentage alternative"
68+
"provenance": "the plugin's own number, not ISO-backed: it coincides with the informative figure in ISO/IEC 5055:2021 §6.3 Table 1, which is not normative, while the normative form (§8.2.115) is a function-level percentage; 500 is selectable; size.mode iso-8.2.115 selects the normative function-percentage alternative"
6969
},
7070
{
7171
"measure": "function_lines_pct",
7272
"config_key": "size.function_lines_pct",
7373
"value_key": "function_lines_pct",
7474
"direction": "at_or_above",
75-
"provenance": "ISO/IEC 5055:2021 §8.2.115 (normative): a function whose non-empty lines exceed this percentage of the file's; used when size.mode is iso-8.2.115"
75+
"provenance": "ISO/IEC 5055:2021 §8.2.115 (normative) states the 5% with no base; the percentage-of-file reading is the plugin's; used when size.mode is iso-8.2.115"
7676
},
7777
{
7878
"measure": "coverage",

plugins/code-metrics/skills/audit-size/SKILL.md

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -59,9 +59,8 @@ side as INCONCLUSIVE); otherwise keep the JSON beside your notes and compare by
5959
- The reference compares against `lines_non_blank`. The bundled default is 1000, and its
6060
provenance is the one sentence the report prints beside it: the plugin's own number, not
6161
ISO-backed: it coincides with the informative figure in ISO/IEC 5055:2021 §6.3 Table 1, which is
62-
not normative, while the normative form (§8.2.115) is a function-level percentage; 500 is the
63-
operator-list figure, selectable; size.mode iso-8.2.115 selects the normative
64-
function-percentage alternative.
62+
not normative, while the normative form (§8.2.115) is a function-level percentage; 500 is
63+
selectable; size.mode iso-8.2.115 selects the normative function-percentage alternative.
6564
- A `null` value means the collector did not produce it (the bundled counter has no comment
6665
count); it is never zero.
6766
- `status` is `complete` when every lane in scope was measured, `partial` when one was not, and

0 commit comments

Comments
 (0)