Skip to content

Commit d1a9dd1

Browse files
kyle-sextonclaude
andauthored
fix(skill-quality,playbooks): correct the reserved-word basis and severity, record the load probes, anchor every record (#4072)
No related issue: follow-up corrections to #4069 after an independent audit of its accepted decisions; no issue tracks them. ## Summary Two fresh-context validators audited every decision behind #4069 against the official pages with the original rationale withheld. Nine rows were challenged, all on evidence that checks out against the platform Skills guide, the Agent Skills specification, the Claude Code skills page, and two load probes. This PR applies the resulting corrections. Nothing here adds a rule; it corrects where existing rules say they come from, downgrades one over-broad FAIL, and replaces an assumed harness claim with a measured one. ## Fix - **skill-quality 0.22.1** - Check 1's reserved-word limb (`anthropic`, `claude`) is a Skills API upload requirement ([skills guide, "Creating a Skill"](https://platform.claude.com/docs/en/build-with-claude/skills-guide#creating-a-skill), repeated under "Limits and constraints"; [overview, "Skill structure"](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview#skill-structure)), not an Agent Skills spec rule: the spec's `skills-ref` validator has no word list. Claude Code loads such names and ships bundled skills named `claude-api` and `claude-in-chrome`. The limb is now a WARN naming the upload surface; the 64-codepoint cap stays a FAIL on the spec basis. The script comment, error text, README, and CHANGELOG name the correct source per limb (the 0.22.0 entry had attributed both to the spec). - "Claude Code loads it" in the check 1 and check 4 comments now rests on a recorded `--plugin-dir` load probe on Claude Code 2.1.263 (an 88-codepoint name containing `claude` and a 608-line SKILL.md both loaded and were invoked through `claude -p`), instead of on `claude plugin validate` exiting 0, which proves only what validate inspects. - Check 4 and check 26 comments cite anchored URLs and the skill-creator SKILL.md in anthropics/skills, per the upstream-drift record form. - Tests 17c and 17d expect the WARN and the exit 0. - **playbooks 0.10.1** (`skill-authoring`) - The 1,024 description cap is stated as the spec's, enforced by `skills-ref` and stated as a Skills API upload requirement; the earlier "enforced on upload paths (claude.ai, the Skills API)" named claude.ai with no source. Eval case 3 follows. - The dependency paragraph carries the overview's second Claude Code bullet (installs stay local to the project, never global) and scopes its `pip install` example; the network sentence separates the Claude API sandbox (no network, no runtime installs) from claude.ai (varies with admin settings) instead of one "platform sandbox". - The `/skill-doctor` mention is presence-gated ("v2.1.252 or later, in a session that fetches feature flags, in the terminal") per the native-references convention; the Record notes the CHANGELOG lists it under 2.1.261. - The 500-line rule's "advisory on every surface" cites the load probe as well as the docs. - Every Record line cites an anchored URL instead of a quoted section name, and the spoke's intro says "anchor" rather than "anchor or section". - The pre-share checklist gains the presence-gated skill-creator row in its Testing group, which the decision placed there and the first change put only in the sibling spoke. Review follow-up (`cf8b0171a`): Codex's one finding fixed. The project-local install examples no longer offer `pip install --user`, which writes to the shared user site; they name a project virtualenv, a project `node_modules`, or an explicitly project-scoped target, and say why the user site is excluded. One audited row was reclassified to a human decision and is not changed here: whether third-person description voice applies to new skills only (current guidance), to the whole fleet through a voice sweep of 258 descriptions, or is recorded as a deliberate deviation like the gerund naming one. ## Verification Run on Linux, branch head `995c9a2e2` (re-run on the one file `cf8b0171a` touches: `markdownlint-cli2` 0 issues, `typos` clean, no em dashes, `check-skill.sh --require-evals` PASS with the same 2 pre-existing WARNs): - `bash plugins/skill-quality/scripts/check-skill.test.sh`: all assertions passed (17c and 17d updated: a leaf or declared name carrying a reserved word WARNs and exits 0). - `shellcheck plugins/skill-quality/scripts/check-skill.sh`: clean. - `check-skill.sh --require-evals` over `skill-quality/check` (PASS, 1 pre-existing WARN) and `playbooks/skill-authoring` (PASS, 2 pre-existing WARNs); `check-evals-quality.sh` PASS on the four cases. - `markdownlint-cli2` over the five changed markdown files: 0 issues. `typos` over the diff: clean. `check-purged-em-dashes.sh`: no em dashes. - `check-changelog-parity.sh --check`, `--check-bump origin/main`, `--check-order`: pass. `generate-catalog.mjs --check`: in sync (no description changed). - Every new anchor was checked against the fetched page snapshots: the headings exist on the skills guide, the overview, the Claude Code skills, MCP, permissions, plugins-reference and sub-agents pages, and the best-practices page. - `scripts/affected-tests.sh --run`: every selected suite passed except one, with 15 NOT RUN (Python and PowerShell suites the shell runner does not execute; CI's own lanes cover them). The one FAIL is `plugins/claude-ops/skills/plugins/scripts/cache-content-check.test.sh`, the process-budget trace probe that counts nothing in this container; it failed identically on #4069 and standalone, involves no file this PR touches, and is host-specific. - CI on `995c9a2e2`: `ci-status` and every lane green on the draft run; Claude code-review and security-review lanes green with no findings after the ready flip. ## Related - Follow-up to #4069; the audit verdicts and their resolution live in the untracked `.work/skill-authoring-best-practices/` memory tier of the authoring checkout (`AUDIT-ANSWERS-A.md`, `AUDIT-ANSWERS-B.md`, and the "Audit resolution" section of `INTERVIEW-ACCEPTED.md`). - #4070 and #4071 remain the deferred follow-ups from #4069; nothing here changes their scope. - `docs/conventions/upstream-drift/README.md` (record form) and `docs/conventions/native-references/README.md` (presence gate) are the conventions the corrections apply. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01MSGD4WGhUXepHq3LTmpQML --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 18c85c2 commit d1a9dd1

10 files changed

Lines changed: 188 additions & 88 deletions

File tree

plugins/playbooks/.claude-plugin/plugin.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
33
"name": "playbooks",
4-
"version": "0.10.0",
4+
"version": "0.10.1",
55
"description": "Doctrine and knowledge playbooks as on-demand skills, plus a maintainer-facing update skill. boris — Boris Cherny's Claude Code workflow tips (howborisusesclaudecode.com); skill-authoring — Anthropic's internal skill-authoring playbook; fable-5 — Claude Fable 5's operating doctrine (self-authored, no upstream). The boris and skill-authoring packs vendor a verbatim upstream baseline; /playbooks:update drift-checks and syncs those baselines centrally (maintainers).",
66
"author": {
77
"name": "Melodic Software",

plugins/playbooks/CHANGELOG.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,28 @@ All notable changes to the `playbooks` plugin are recorded here. The `version` i
44
`.claude-plugin/plugin.json` is the delivery vehicle — a consumer receives a change
55
only after that version increases.
66

7+
## [0.10.1]
8+
9+
### Fixed
10+
11+
- **`skill-authoring`**: `reference/authoring-guidance.md` corrections after an independent audit
12+
of its claims against the official pages. The 1,024-character description cap is stated as the
13+
Agent Skills specification's, enforced by its `skills-ref` validator and stated as a Skills API
14+
upload requirement (the earlier "enforced on upload paths (claude.ai, the Skills API)" named a
15+
surface no source documents); the dependency paragraph carries the platform overview's second
16+
Claude Code bullet (installs stay local to the project, never global) and scopes its example
17+
accordingly; the network sentence separates the Claude API sandbox (no network, no runtime
18+
installs, pre-installed list only) from claude.ai (varies with admin settings) instead of one
19+
"platform sandbox"; the `/skill-doctor` mention is presence-gated with its version floor stated
20+
as "v2.1.252 or later" and its feature-flag and terminal conditions, per the native-references
21+
convention; the 500-line rule's "advisory on every surface" now rests on a recorded
22+
`--plugin-dir` load probe (Claude Code 2.1.263, 2026-09-11, a 608-line SKILL.md loaded and
23+
invoked) as well as the docs; and every Record line cites its basis by anchored URL rather than a
24+
quoted section name, with the spoke's intro tightened to match the upstream-drift record form.
25+
`evals/evals.json` case 3 follows the cap wording.
26+
- **`skill-authoring`**: `reference/authoring-checklist.md` gains the presence-gated skill-creator
27+
row in its Testing group (attestation), which the guidance spoke carried but the checklist did not.
28+
729
## [0.10.0]
830

931
### Added

plugins/playbooks/skills/skill-authoring/evals/evals.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@
2929
"id": 3,
3030
"name": "description-caps-and-their-sources",
3131
"prompt": "What length limits apply to a skill description and where does the 1,024 number come from?",
32-
"expected_output": "Names both caps with their sources: 1,024 characters is the Agent Skills specification's validation limit on the description field, enforced on upload paths and not by Claude Code; 1,536 characters is Claude Code's skill-listing truncation of description plus when_to_use (skillListingMaxDescChars), under a listing budget of 1% of the context window that shortens least-invoked skills first. States that when_to_use extends the one description in the listing and that the key use case goes first because truncation is tail-first. Serves reference/authoring-guidance.md rather than restating the page as an instruction.",
32+
"expected_output": "Names both caps with their sources: 1,024 characters is the Agent Skills specification's validation limit on the description field, enforced by the spec's skills-ref validator and stated as a Skills API upload requirement, not by Claude Code; 1,536 characters is Claude Code's skill-listing truncation of description plus when_to_use (skillListingMaxDescChars), under a listing budget of 1% of the context window that shortens least-invoked skills first. States that when_to_use extends the one description in the listing and that the key use case goes first because truncation is tail-first. Serves reference/authoring-guidance.md rather than restating the page as an instruction.",
3333
"files": [],
3434
"expectations": [
3535
"Output names 1,024 as the Agent Skills specification validation cap on the description field alone and says Claude Code does not validate it",

plugins/playbooks/skills/skill-authoring/reference/authoring-checklist.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,7 @@ have checked a judgment row is misreporting.
6060
| `evals/evals.json` is present | mechanical (check 14) |
6161
| `evals/evals.json` validates against the schema and passes the eval-quality lint | mechanical (`/skill-quality:check validate-evals <skill>`) |
6262
| Three or more eval cases | mechanical (advisory) |
63+
| Where the bundled skill-creator plugin is installed, its eval modes ran the cases with a subagent per case; otherwise the fresh-session loop below stands in | attestation |
6364
| Fresh-session baseline captured with the skill disabled, then enabled | attestation |
6465
| Tested on real tasks, not contrived scenarios | attestation |
6566
| Models exercised: which of `haiku`, `sonnet`, `opus`, `fable` | attestation |

0 commit comments

Comments
 (0)