Skip to content

Testing surface has four names for one thing, and no statement of what it tests — rename to apr test while it is still nearly free #2525

Description

@noahgift

The problem

One capability, four names:

layer name
crate aprender-test-cli
[lib] name probador
apr subcommand apr probar
upstream dep jugar-probar

None of them says what it tests. probar is Spanish for "to try/test" — it names the verb, not the subject, so apr probar --help tells a user nothing.

Compare the pattern that already works in this repo:

apr data    Data quality pipeline (audit, split, balance) — powered by alimentar

Plain-English noun for the subcommand, Spanish name kept for the standalone binary and credited in the description. A user reading apr data knows what it does; a user reading apr probar does not.

Why now: the rename is nearly free today

apr probar currently routes one subcommand:

Commands:
  tensor  Export tensor activations for visual regression testing (PMAT-481)

Its own help says the rest — test, record, coverage, playbook, comply, av-sync, audio, video, animation, stress, llm — "land in follow-up PRs that delegate to the probador library."

So 16 of 17 subcommands are not yet exposed through apr. Renaming now touches one routed path. After the follow-up PRs land, it is a breaking change across the whole testing surface.

What we actually test (currently unstated)

The 17 subcommands are a grab-bag with no organising principle. They fall into four coherent groups:

group subcommands what is under test
web serve, build, watch, comply (WASM C001–C010), stress the WASM/browser build and its runtime behaviour
llm llm test/load/bench/sweep/score/experiment/data-audit/gen-dataset/report inference correctness, throughput, and cost against an endpoint
media av-sync, audio, video, animation rendered output against ground truth (EDL, levels, codec, easing)
harness test, record, report, coverage, init, config, playbook the runner itself — recording, replay, state machines, reporting

That table is the "what do we test and why" that is missing. It should be the top of the crate docs and the shape of the CLI.

Proposal

  1. apr test — the subcommand. apr test is currently free (verified against the built binary; apr data, apr qa, apr bench, apr eval are all taken, apr test is not).
  2. Description follows the apr data precedent: "Test harness for web, LLM, media and replay — powered by probador."
  3. Group the subcommands as above so apr test --help states the four things it tests rather than listing 17 flat verbs.
  4. Keep probador as the [lib] name and the standalone binary identity — same as alimentar behind apr data. The Spanish name is fine as an engine name; it is only wrong as the user-facing verb.
  5. Rename the crate aprender-test-cli → keep as-is, or align to aprender-test. Lower priority; the user-visible name is what matters.

Open question for the owner: whether apr probar should remain as a hidden alias for one release or be removed outright. Given only tensor is routed, removing outright is defensible.

Related

The binary-surface audit (#2503) found this crate is also the one whose [lib] name (probador) differs from its binary name (aprender-test-cli) — the same class that left 126 dead cargo_bin("renacer") references in aprender-profile (#2516) and 1 dead cargo_bin("probador") here.

Refs #2503

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions