Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 27 additions & 2 deletions .github/workflows/book.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,23 @@ on:
paths:
- "book/**"
- ".github/workflows/book.yml"
# FALSIFY-BOOK-CLI-PARITY-001 asserts every `apr` subcommand has a chapter.
# Filtering on book/** alone meant it only ran when the BOOK changed, never
# when the CLI gained a command — so `apr beat-run` shipped in #1995 with no
# chapter and the gate stayed green for months, until an unrelated batch that
# happened to touch book/ woke it. A parity gate must run when EITHER side
# moves; watching only one of them is how it comes to certify nothing.
- "crates/apr-cli/src/commands_enum.rs"
- "crates/apr-cli/src/extended_commands.rs"
- "crates/apr-cli/src/tool_commands.rs"
pull_request:
branches: [main]
paths:
- "book/**"
- ".github/workflows/book.yml"
- "crates/apr-cli/src/commands_enum.rs"
- "crates/apr-cli/src/extended_commands.rs"
- "crates/apr-cli/src/tool_commands.rs"

permissions:
contents: read
Expand Down Expand Up @@ -78,8 +90,21 @@ jobs:
if ! command -v pmat >/dev/null 2>&1; then
cargo install pmat --locked --quiet || true
fi
# Validate the book-completeness contract via pv (in-tree dogfood per CLAUDE.md)
pv validate contracts/apr-book-completeness-v1.yaml
# Validate the book-completeness contract via pv.
#
# This line used to call a bare `pv` and exited 127 (command not found)
# the first time the step ever ran: every earlier run died at the parity
# gate above, so nothing reached it. Note the shape of the bug — the step
# guards `pmat` with `command -v` and a comment saying it "tolerates
# absence", then invokes `pv` unguarded one line earlier, in a step whose
# entire purpose is running pv.
#
# `pv` is IN-TREE (crates/aprender-contracts-cli, [[bin]] name = "pv"), so
# build it from the tree instead of depending on an installed copy — that
# is the dogfood rule in CLAUDE.md, and it cannot drift from the contracts
# it is validating.
cargo run --quiet -p aprender-contracts-cli --bin pv -- \
validate contracts/apr-book-completeness-v1.yaml
# pmat comply check — single binary signal; tolerates absence (pmat may not be available on every runner)
if command -v pmat >/dev/null 2>&1; then
pmat comply check 2>&1 | tail -10 || echo "::warning::pmat comply check returned non-zero (advisory)"
Expand Down
5 changes: 4 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ SHELL := /bin/bash
# Multi-line recipes execute in same shell
.ONESHELL:

.PHONY: all build test test-smoke test-fast test-quick test-full test-heavy lint lint-current fmt clean doc book book-build book-serve book-test tier1 tier2 tier3 tier4 coverage coverage-fast profile hooks-install hooks-verify lint-scripts bashrs-score bashrs-lint-makefile chaos-test chaos-test-full chaos-test-lite fuzz bench dev pre-push ci check run-ci run-bench audit deps-validate deny pmat-score pmat-gates quality-report semantic-search examples mutants mutants-fast property-test install-alsa test-alsa test-audio-full contract-validate contract-test contract-audit contract-regen contract-check dev-setup check-siblings
.PHONY: all build test test-smoke test-fast test-quick test-full test-heavy lint lint-current fmt clean doc book book-build book-serve book-test tier1 tier2 tier3 tier4 coverage coverage-fast profile hooks-install hooks-verify lint-scripts bashrs-score bashrs-lint-makefile chaos-test chaos-test-full chaos-test-lite fuzz bench dev pre-push ci check run-ci run-bench audit deps-validate deny pmat-score pmat-gates quality-report semantic-search examples mutants mutants-fast property-test install-alsa test-alsa test-audio-full contract-validate contract-test contract-audit contract-regen contract-check dev-setup check-siblings check-wasm32

# Default target
all: tier2
Expand Down Expand Up @@ -985,6 +985,9 @@ publish: ## Publish crate(s) to crates.io — strips [patch], publishes, then ve
echo "POST-PUBLISH VERIFICATION: PASSED"; \
fi

check-wasm32: ## Verify aprender-core still compiles for wasm32-unknown-unknown (aprender#2310)
@bash scripts/check_wasm32_core_builds.sh

check-siblings: ## Verify sibling repos exist and versions are compatible
@echo "Checking sibling repositories..."
@all_ok=true; \
Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,9 @@ publishing — all backed by YAML provable contracts that fail CI on drift.
| Metric | Count | Source of truth |
|-------:|------:|---|
| Workspace crates | **78** workspace crates | `cargo metadata --no-deps` (NOT `ls crates/` — 4 are `exclude`d, 1 has no Cargo.toml) |
| Provable contracts | **1770** provable contracts | `find contracts/ -name '*.yaml'` |
| CLI commands | **103** CLI commands | `apr --help` |
| Book CLI chapters | **103** chapters | `ls book/src/cli/*.md` (parity with CLI) |
| Provable contracts | **1771** provable contracts | `find contracts/ -name '*.yaml'` |
| CLI commands | **105** CLI commands | `apr --help` |
| Book CLI chapters | **105** chapters | `ls book/src/cli/*.md` (parity with CLI) |
| Book lib chapters | **69** chapters | `ls book/src/lib/*.md` (parity with `pub mod`) |

These numbers are enforced by [`contracts/readme-claims-v1.yaml`](contracts/readme-claims-v1.yaml).
Expand Down Expand Up @@ -212,7 +212,7 @@ paiml/aprender/
├── Cargo.toml # Workspace root + `cargo install aprender`
├── crates/
│ ├── aprender-core/ # ML library (use aprender::*)
│ ├── apr-cli/ # CLI logic (103 subcommands)
│ ├── apr-cli/ # CLI logic (105 subcommands)
│ ├── aprender-compute/ # SIMD/GPU compute kernels
│ ├── aprender-gpu/ # CUDA PTX
│ ├── aprender-serve/ # Inference server
Expand Down
6 changes: 6 additions & 0 deletions book/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,7 @@
- [apr attn-viz-lint](./cli/attn-viz-lint.md)
- [apr audio-inspect-lint](./cli/audio-inspect-lint.md)
- [apr awq-lint](./cli/awq-lint.md)
- [apr beat-run](./cli/beat-run.md)
- [apr bench](./cli/bench.md)
- [apr canary](./cli/canary.md)
- [apr cbtop](./cli/cbtop.md)
Expand All @@ -291,13 +292,15 @@
- [apr compile](./cli/compile.md)
- [apr convert](./cli/convert.md)
- [apr data](./cli/data.md)
- [apr dataset](./cli/dataset.md)
- [apr ddp-metrics-lint](./cli/ddp-metrics-lint.md)
- [apr debug](./cli/debug.md)
- [apr decrypt](./cli/decrypt.md)
- [apr diagnose](./cli/diagnose.md)
- [apr diff](./cli/diff.md)
- [apr distill](./cli/distill.md)
- [apr dry-sampling-lint](./cli/dry-sampling-lint.md)
- [apr debug embed-viz](./cli/embed-viz.md)
- [apr embed-viz-lint](./cli/embed-viz-lint.md)
- [apr embed](./cli/embed.md)
- [apr embeddings-lint](./cli/embeddings-lint.md)
Expand All @@ -321,6 +324,7 @@
- [apr imatrix-lint](./cli/imatrix-lint.md)
- [apr import](./cli/import.md)
- [apr inspect](./cli/inspect.md)
- [apr kernel](./cli/kernel.md)
- [apr kv-timeline-lint](./cli/kv-timeline-lint.md)
- [apr lint](./cli/lint.md)
- [apr list](./cli/list.md)
Expand Down Expand Up @@ -399,6 +403,7 @@
- [aprender::code](./lib/code.md)
- [aprender::compute](./lib/compute.md)
- [aprender::data](./lib/data.md)
- [aprender::datasets](./lib/datasets.md)
- [aprender::decomposition](./lib/decomposition.md)
- [aprender::demo](./lib/demo.md)
- [aprender::embed](./lib/embed.md)
Expand Down Expand Up @@ -427,6 +432,7 @@
- [aprender::nn](./lib/nn.md)
- [aprender::online](./lib/online.md)
- [aprender::optim](./lib/optim.md)
- [aprender::pipeline](./lib/pipeline.md)
- [aprender::prelude](./lib/prelude.md)
- [aprender::preprocessing](./lib/preprocessing.md)
- [aprender::primitives](./lib/primitives.md)
Expand Down
75 changes: 75 additions & 0 deletions book/src/cli/beat-run.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
<!-- PCU: cli-beat-run | contract: contracts/apr-cli-commands-v1.yaml -->

# apr beat-run

Evaluate a beat-benchmark contract against a measured value (PMAT-741)

**Category**: Quality

## Synopsis

```text
apr beat-run <CONTRACT> [--measured <VALUE>] [--json]
```

`apr beat-run` is the falsifiable runner behind the four-pillar "replace **and**
beat" mission. A beat contract pins an incumbent's baseline — the number
scikit-learn, PyTorch, Unsloth or Ollama actually produces — together with the
threshold `apr` must clear to claim a win. This command reads that contract and,
given a measurement, returns the verdict.

Two modes:

- **Without `--measured`** it reports the contract's pinned parameters and exits
0. Use this to see what a beat currently claims.
- **With `--measured`** it computes the verdict and exits **non-zero on a
regression or an unjudgeable contract**, so it can gate CI directly.

The verdict is not computed here. It comes from
`aprender_contracts::schema::Beat::evaluate`, the single source of truth, so the
CLI and the contract engine cannot drift into disagreeing about whether a beat
was won.

## Examples

<!-- example-cost: trivial -->
```bash
apr beat-run --help
```

Report what a beat contract pins, without judging anything:

<!-- example-cost: trivial -->
```bash
apr beat-run contracts/beat-sklearn-iris-v1.yaml
```

Judge a measurement and gate on it — this is the form CI uses:

<!-- example-cost: trivial -->
```bash
apr beat-run contracts/beat-sklearn-iris-v1.yaml --measured 0.973
echo "exit=$?" # non-zero means REGRESSED or unjudgeable
```

## Exit codes

| exit | meaning |
|-----:|---------|
| 0 | the beat was WON, or no `--measured` value was supplied |
| non-zero | REGRESSED, or the contract cannot judge the value it was given |

An **unjudgeable** contract exits non-zero on purpose. A beat that cannot decide
is not a beat that passed: silently treating "I could not tell" as a win is the
failure mode this runner exists to prevent.

A contract path that does not exist reports `File not found`, not a format
error — an earlier version printed `Invalid APR format:` and sent readers
looking for a model that was never involved.

## See also

- [`apr bench`](./bench.md) — produce the measurement this command judges
- [`apr qa`](./qa.md) — falsifiable QA gates on a model artifact
- Source: [`crates/apr-cli/src/commands/beat_run.rs`](https://github.com/paiml/aprender/blob/main/crates/apr-cli/src/commands/beat_run.rs)
- Contract: [`contracts/apr-cli-commands-v1.yaml`](https://github.com/paiml/aprender/blob/main/contracts/apr-cli-commands-v1.yaml)
59 changes: 59 additions & 0 deletions book/src/cli/dataset.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
<!-- PCU: cli-dataset | contract: contracts/apr-lint-producers-v1.yaml -->

# apr dataset

Dataset inspection tools.

**Category**: Inspection

## Synopsis

```text
apr dataset audio-inspect <FILE> [--format json|text] [-o FILE] [--force]
```

## What `audio-inspect` measures

It decodes an uncompressed RIFF/WAVE file and reports the shape and amplitude
extrema it actually measured:

| Field | Meaning |
|-------|---------|
| `sample_rate` | Hz, from the `fmt ` chunk — never resampled |
| `channels` | channel count, from the `fmt ` chunk — never mixed down |
| `samples` | frames per channel (torchaudio's `num_frames`) |
| `min` / `max` | amplitude extrema over every decoded sample |
| `codec` | `pcm_u8`, `pcm_s16le`, `pcm_s24le`, `pcm_s32le` or `pcm_f32le` |

Integer PCM is normalised by the negative full-scale magnitude, the
`torchaudio.load(normalize=True)` convention. Float payloads are reported as
stored, so a float WAV that overshoots ±1 shows up as such.

A container or codec it cannot decode — FLAC, MP3, Ogg, ADPCM, a truncated
`data` chunk, an empty stream — is **refused** with a non-zero exit and a
message naming what was found. It never estimates.

## Example

<!-- example-cost: trivial -->
```bash
apr dataset audio-inspect --help
```

Producing the observation `apr audio-inspect-lint` reads:

<!-- example-cost: interactive -->
```bash
apr dataset audio-inspect clip.wav --format json -o audio.json
apr audio-inspect-lint --json-file audio.json --expected-sample-rate 16000
```

## Full help

Run `apr dataset audio-inspect --help` for the complete option list.

## See also

- Consumer: [`apr audio-inspect-lint`](./audio-inspect-lint.md)
- Source: [`crates/apr-cli/src/commands/audio_inspect.rs`](https://github.com/paiml/aprender/blob/main/crates/apr-cli/src/commands/audio_inspect.rs)
- Contract: [`contracts/apr-lint-producers-v1.yaml`](https://github.com/paiml/aprender/blob/main/contracts/apr-lint-producers-v1.yaml)
8 changes: 7 additions & 1 deletion book/src/cli/debug.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,14 @@ Simple debugging output ("drama" mode available)
## Synopsis

```text
apr debug [OPTIONS]
apr debug [FILE] [OPTIONS]
apr debug embed-viz --model <MODEL> [OPTIONS]
```

`FILE` is optional because a subcommand brings its own input. `apr debug` with
neither a file nor a subcommand refuses rather than exiting 0 having done
nothing.

## Example

<!-- example-cost: model-required model: qwen2.5-coder-1.5b-instruct-q4_k_m.gguf -->
Expand All @@ -25,5 +30,6 @@ Run `apr debug --help` for the complete option list.

## See also

- Subcommand: [`apr debug embed-viz`](./embed-viz.md)
- Source: [`crates/apr-cli/src/commands/debug.rs`](https://github.com/paiml/aprender/blob/main/crates/apr-cli/src/commands/debug.rs)
- Contract: [`contracts/apr-page-cli-debug-v1.yaml`](https://github.com/paiml/aprender/blob/main/contracts/apr-page-cli-debug-v1.yaml)
1 change: 1 addition & 0 deletions book/src/cli/embed-viz-lint.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,5 +25,6 @@ Run `apr embed-viz-lint --help` for the complete option list.

## See also

- Producer: [`apr debug embed-viz`](./embed-viz.md)
- Source: [`crates/apr-cli/src/commands/embed_viz_lint.rs`](https://github.com/paiml/aprender/blob/main/crates/apr-cli/src/commands/embed_viz_lint.rs)
- Contract: [`contracts/apr-page-cli-embed-viz-lint-v1.yaml`](https://github.com/paiml/aprender/blob/main/contracts/apr-page-cli-embed-viz-lint-v1.yaml)
Loading
Loading