Skip to content

Catalog and pricing: one authority for model/provider facts (supersedes #4173) #6396

Description

@Hmbown

Supersedes #4173, which was opened from a 2026-07 architecture doc with counts that have since drifted (it says 81 models / 31 providers; both are now far larger). Its acceptance criterion was "there is an inventory of which registries are product contracts vs generated/catalog data vs internal implementation." That inventory now exists and is below, so the old issue is closed in favour of this scoped one.

Root cause, verified

offerings_from_models_dev (crates/config/src/catalog.rs:344) enumerates catalog offerings from catalog.providers[*].models only.

models.dev deliberately splits facts about a model from facts about a provider serving it. We implemented one half. A model that exists only in the top-level models map therefore produces no offering and can never be listed — so we are structurally late for every vendor, not just the one that surfaced it.

Two precisions worth recording, because they change the fix:

  1. The top-level map is not entirely unread — crates/config/src/models_dev.rs:86 reads it for reasoning-support enrichment. The defect is in enumeration, not deserialization. The fix is "enumerate from the union", not "read a field nobody reads".
  2. ModelsDevModel (models_dev.rs:199) carries no provider fieldid, name, family, capabilities, limit, modalities, and nothing naming who serves it. The join must therefore come from the map key (xiaomi/mimo-v2.6-pro). That works against live upstream data, but our bundled snapshot keys models bare (its only two entries are deepseek-v4-pro, deepseek-v4-flash). So the ingestion fix alone does not repair offline/first-boot until crates/config/assets/models_dev.bundled.json is regenerated in upstream shape.

Consequence for 0.10.0: commit 2dca7fcdda (hand-seeded MiMo 2.6 rows) is a human doing the join the code refuses to do, and it stays load-bearing for cold offline start. Retire it when the seed is regenerated, not when the ingestion fix lands.

Inventory: seven independent declarations of the same model facts

# Source Size
1 crates/models/src/lib.rs match tables ~150 ids
2 crates/models/assets/model_catalog.bundled.json 56 entries
3 crates/config/assets/models_dev.bundled.json 99 rows / 21 providers
4 crates/agent/src/lib.rs ModelInfo 155 entries — the only home for aliases + supports_tools
5 crates/config/src/provider_defaults.rs 169 consts, duplicated verbatim in crates/tui/src/config/models.rs
6 crates/tui/src/pricing.rs 5,920 lines, 287 price match arms
7 web/lib/facts.generated.ts 50 providers / 77 models

Plus two parallel provider enums — ProviderKind (52) and ApiProvider (53) — bridged by a fixed-size KIND_LOOKUP: [_; 53] in crates/tui/src/config.rs.

Adding one provider today costs ~14 hand edits across provider_kind.rs, provider.rs, lib.rs, the TUI enum and that fixed-size array. That cost is why vendors go missing: adding one is a code change, so it competes with feature work and loses.

Slices

A — model-only ingestion. Enumerate offerings from the union of providers[*].models and namespaced top-level models keys. ~30 lines plus a test. Stops every vendor being late. Land first, independent of everything else.

B — regenerate the offline seed in upstream key shape, so A also fixes cold start, then retire the hand-seeded rows.

C — collapse to one authority. Migrate the last consumer of each of the seven, deleting as you go; move aliases + supports_tools out of crates/agent into the catalog (their only home today); collapse ApiProvider into ProviderKind so KIND_LOOKUP dies. Large blast radius — exhaustive matches across two crates — so it is deliberately not a release-window change.

D — upstream the thin vendors to models.dev (AICraft, MiMo, SenseNova, StepFun, DashScope, command-code). It is a public repo; the live refresh then picks them up with no infrastructure, and every other tool benefits. Keep a local overlay only for what models.dev structurally cannot carry: transport facts and prices we bill against.

Pricing: hold it to billing standard before billing exists

Display-only today, but PRODUCT_PRD.md:220 commits to "an evidence-backed model catalog". A wrong cost number is a false product claim regardless of whether money moves.

There are currently three pricing authorities: pricing.rs (display), the platform rate cards (CWC_MANAGED_MODEL_RATE_CARDS_JSON), and cwc_usage_rate_cards (Postgres). When they disagree we show one number and would charge another. Managed inference is gated off today, so fixing the divergence now costs nothing; after it is enabled it means migrating historical charges.

The rate-card shape in wrangler.jsonc is already the professional one — revision, sourceRef, pricingPolicy, checkedAt, expiresAt, integer micros, customer vs provider rates, CWC_RATE_CARD_MAX_AGE_DAYS. The work is propagating it, not inventing it:

  1. Integer micros end to end, never f64.
  2. sourceRef / checkedAt / expiresAt on every rate; past expiry reads unpriced, never stale-priced.
  3. Unknown is a value, not zero — Quote::Unpriced { reason } as a type, so no surface can render a guess as $0.00. The MiMo seed already gets this right in data; make it unrepresentable to get wrong.
  4. Separate rate data (catalog rows) from billing-surface detection (credential logic, stays local) from rate rules (time-of-day, context tiers) as a few typed variants over data rather than 287 arms.
  5. Freeze the quote at dispatch and persist it in the receipt; a later refresh must never reprice history. Already correct on the cloud-facts path — generalize it.
  6. One quote function, one stated rounding rule, an audit trail per number.
  7. Golden vectors: (route, usage, timestamp) → exact amount, covering the DeepSeek time-of-day windows and the 512k/272k/200k tier edges.
  8. Reconcile against real invoices. The evidence file records invoice reconciliation as "unqualified". 287 arms nobody has diffed against a bill are 287 guesses, however tidy. This is the only item that needs something we do not already have, and it is the one that separates professional from merely organized.

Items 1–7 are engineering. Item 8 is ground truth, and no amount of 1–7 substitutes for it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    • Status
      Backlog

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions