docs: schema-validated documentation examples

[igrigorik]

Validation harness that ensures every JSON example in the spec docs is validated against UCP schemas. Catches documentation drift — when schemas change, stale examples break CI and docs builds.

Contract

Every json block requires an HTML comment annotation on the preceding line:

<!-- ucp:example schema=shopping/checkout op=read -->
<!-- ucp:example schema=shopping/checkout path=$.totals op=read -->
<!-- ucp:example schema=shopping/cart op=create direction=request -->
<!-- ucp:example skip reason="JSON-RPC transport binding" -->

Unannotated blocks are hard failures.

Core rule: "if you open it, you own it." When an example opens an object's braces, every required field (per the resolved schema for the declared op + direction) must be shown or acknowledged with an ellipsis. This applies recursively.

Three-tier value contract for annotated (non-skip) examples:

Tier	Syntax	Meaning
Full value	Real JSON	Validated against schema (types, constraints, enums)
Required field	{ ... }, [ ... ], "..."	Field must exist; value not shown; scaffold fills it for validation
Absent	Not in example	Only permitted for optional schema fields

Pipeline

1. Extract — find annotated ```json blocks (handles MkDocs tab indentation)
2. Unwrap — strip HTTP headers from request/response examples
3. Expand — substitute {{ ucp_version }} with valid date
4. Preprocess — convert bare ... to valid JSON ({ ... } → {"...":"..."})
5. Coverage — walk resolved schema, verify required fields acknowledged
6. Strip — remove ellipsis markers
7. Merge — deep-merge into scaffold (example wins, scaffold fills gaps)
8. Validate — ucp-schema validate on merged payload
9. Report — map errors to source file:line

Bugs caught and fixed

• checkout.md: item missing required price field in disclosure example
• checkout.md: totals array missing required subtotal entry
• checkout-rest.md: business outcome used non-schema available_quantity field on line_item; replaced with message-based pattern
• discount.md (4 instances): quantity wrongly nested inside item object instead of on line_item — schema requires quantity at line_item level
• discount.md: price and title shown in create requests where schema omits them (request-only fields: only item.id is valid)

Current state

python3 scripts/validate_examples.py --schema-base source/schemas/ — 52 passed, 0 failed, 0 errors

268 blocks across 39 files:

• 52 validated — REST request/response pairs, core spec examples, error responses, extension examples
• 216 skipped — transport bindings, schema definitions, configs (future: payload_path= for JSON-RPC unwrap)

🟢 Pausing fixing the rest (skipped) to get feedback: 🟢

• Does this strategy make sense?
• Anything we want to change or improve?

---

Type of change

• Documentation update
• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings

[drewolson-google]

Looks good, one small comment on empty request bodies.

[drewolson-google]

Can we make the validator / annotation handle empty request bodies? This is a valid request, so having to skip it feels off.