fix(pricing): price OpenRouter turns from the lake and declared rates - #6691
Conversation
Cause: OpenRouter's /v1/models now lists `~`-prefixed "latest" alias ids (`~deepseek/deepseek-pro-latest`). They fail `valid_catalog_model_id`, and one invalid row fails the whole roster closed, so the OpenRouter lake scope was permanently `failed/invalid_response` and `fresh_provider_live_pricing_quote_at` never returned a rate: every OpenRouter turn showed "rate unavailable". Separately, the main interactive turn froze its dispatch quote only from the lake and never consulted the operator-declared `[[custom_models]]` rate that background/review envelopes (`effective_route_envelope`) already honor. Fix: `parse_openrouter_models_response` drops `~` alias rows (moving aliases, not billing identities); the fail-closed validator is unchanged for every other id. A new `CodewhaleClient::configured_pricing_quote_at` is shared by `effective_route_envelope` and the turn-loop dispatch boundary, which now prefers a declared rate (scoped to the installed client's endpoint fingerprint) before the lake quote. Not covered: server-tag suffixed ids (`:nitro`, `:free`) still need an exact catalog or declared row; their price can differ from the base id, so they are not stripped. Tests (each fails with its fix hunk reverted, passes with it): - client::tests::fetch_catalog_delta_skips_openrouter_latest_aliases_and_keeps_prices - core::engine::tests::main_turn_dispatch_freezes_declared_custom_model_rate test result: ok. 2 passed; 0 failed (reverted: 0 passed; 2 failed) Related filters (fetch_catalog, openrouter, configured_model_client_tests, exact_turn_snapshot, effective_route, provider_live_pricing): test result: ok. 46 passed; 0 failed cost_status, provider_catalog_live, core::engine::tests::exact: test result: ok. 60 passed; 0 failed Refs #6690 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014ZwqatxgVFxHvovngywnks
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
️✅ There are no secrets present in this pull request anymore.If these secrets were true positive and are still valid, we highly recommend you to revoke them. 🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request. |
Review follow-up for #6691. The live OpenRouter /v1/models lists six routers (openrouter/auto, auto-beta, fusion, pareto-code, bodybuilder, typesafe/jev-router) at "prompt":"-1","completion":"-1". parse_price rejected negatives and the conversion collected with `?`, so the whole refresh still failed and every turn stayed "rate unavailable". - A negative OpenRouter price now means "no fixed rate": the row is kept with cost None. Rows that do not decode, carry an id the catalog cannot hold, or an unreadable/implausible price are skipped and counted with a warning. A response that is not a `{"data": [...]}` list, or whose rows all fail, is still rejected. - declared_or_catalog_quote: a [[custom_models]] row with no rates yields to the exact endpoint's catalog price at every dispatch boundary (main turn, background envelope, EffectiveRouteEnvelope::capture); it stays frozen rate-less only when the catalog has none. The turn-loop logic moved into client::main_turn_pricing_quote_at so it is testable. - A pinned OpenRouter vendor no longer blocks an explicit declared rate for the exact route; the aggregate catalog price stays blocked. Against the saved live list (458 rows): refresh succeeds with 440 rows, 434 priced; deepseek/deepseek-v4-pro prices at 0.95526/1.91052 per M. Focused (cargo test -p codewhale-tui --lib): fixed: test result: ok. 10 passed; 0 failed; 0 ignored; 0 measured; 13793 filtered out reverted (neg price, rate-less fallthrough, vendor pin): test result: FAILED. 0 passed; 3 failed; 0 ignored; 0 measured; 13800 filtered out reverted (per-row decode skip): test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 13802 filtered out related (fetch_catalog openrouter configured_model effective_route provider_live_pricing exact_turn_snapshot main_turn cost_status provider_catalog_live): test result: ok. 120 passed; 0 failed; 0 ignored; 0 measured; 13683 filtered out Fixes #6690 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014ZwqatxgVFxHvovngywnks
…6693 Hosted CI on 776e3fe (PR #6672) failed two codewhale-tui tests. 1. client::catalog_tests::raw_rows_keep_duplicate_known_fields_invalid_in_existing_parsers Cause: #6691 (79d9815) made OpenRouter parsing per-row by decoding `data` as Vec<serde_json::Value>. A Value map keeps the last of a duplicated key, so `{"id":"second","id":"replacement"}` was silently accepted as "replacement" (and a duplicated pricing.prompt took the last rate) instead of being rejected. Fix (product): keep rows as Box<RawValue> and decode each with from_str, so serde's duplicate-field error still fires and the row is skipped and counted as malformed like any other undecodable row. The test now asserts the per-row contract: the ambiguous row is dropped, "first" survives; Baseten still rejects the whole response. 2. tui::ui::tests::first_run_ollama_choice_survives_restart_from_canonical_config Cause: #6693 (1394c29, eb48db7) moves a root default_text_model onto the leaf of the outgoing route that was resolving it. The test seeded default_text_model = "deepseek-v4-pro" with DeepSeek active and no leaf, then asserted the root key survived the switch to Ollama ("index not found"). That encoded superseded behavior. Fix (test): assert the root alias is gone and [providers.deepseek].model = "deepseek-v4-pro", which is the #6693 contract (switching back keeps the choice; Ollama never inherits it). Tests (CARGO_BUILD_JOBS=4, cargo test -p codewhale-tui --lib): - raw_rows_keep_duplicate_known_fields_invalid_in_existing_parsers: test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 13814 filtered out - first_run_ollama_choice_survives_restart_from_canonical_config: test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 13814 filtered out - filters catalog openrouter fetch_catalog first_run_ provider_picker local_ollama: test result: ok. 450 passed; 0 failed; 2 ignored; 0 measured; 13363 filtered out (a first run of the same filters had 1 failure in provider_picker::tests::credential_draft_is_masked_and_escape_drops_it_without_persistence, which passed 3/3 alone and in the rerun; it counts files in a home dir and is untouched by this change) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014ZwqatxgVFxHvovngywnks
Fixes #6690
Cause
/v1/modelsrefresh failed as a whole on any single bad row, so the OpenRouter lake scope stayedfailed/invalid_responseandfresh_provider_live_pricing_quote_atnever returned a rate. Every OpenRouter turn showed "rate unavailable". The live list has two kinds of row that did this:~-prefixed "latest" aliases (~deepseek/deepseek-pro-latest, 18 in the live list), which failvalid_catalog_model_id.openrouter/auto,auto-beta,fusion,pareto-code,bodybuilder,typesafe/jev-router) that publish"prompt":"-1","completion":"-1".parse_pricerejected negatives and the conversion collected with?.turn_loopdispatch boundary) froze its quote only from the lake, never from the operator-declared[[custom_models]]rate thateffective_route_envelope(background/review) already honors.Fix
parse_openrouter_models_responsestill requires a{"data": [...]}list but decodes each row on its own. It drops~alias rows, and it skips and counts (with atracing::warn!) rows that do not decode or whose id the catalog cannot hold.fetch_catalog_deltaalso skips and counts rows with an unreadable or implausible price. A structurally invalid response, or one where no row survives, is stillInvalidResponse. The secret-in-id guard still rejects the whole list.cost: None, never with a partial rate.CodewhaleClient::configured_pricing_quote_atandclient::main_turn_pricing_quote_at. The turn loop now prefers a declared rate, scoped to the installed client's endpoint fingerprint, before the lake quote.provider_catalog_live::declared_or_catalog_quote, used at all three dispatch boundaries (main turn,effective_route_envelope,EffectiveRouteEnvelope::capture). A declared rate wins. A declaration with no rates yields to the same endpoint's catalog price, and stays frozen rate-less only when the catalog has none, so a same-named bundled price still cannot fill it.EffectiveRouteEnvelope::audit: a pinnedopenrouter_vendorstill blocks the aggregate catalog price (RoutingDependentPrice), but no longer blocks an explicit declared (UserOverride) rate for the exact route.Against the saved live list (458 rows), the refresh now succeeds with 440 rows, 434 of them priced.
deepseek/deepseek-v4-proprices at 0.95526 / 1.91052 USD per M (cache read 0.079605).Still open: a server-tag-suffixed route the catalog does not list (the reporter's
deepseek/deepseek-v4.1-flash:nitro) still needs an exact catalog row or a declared[[custom_models]]rate, which now prices it on the main turn. Tag prices can differ from the base id, so they are not stripped.Tests
Each new or strengthened test fails with its fix reverted and passes with it. They assert the actual rates and cost, not just that a quote exists:
client::tests::fetch_catalog_delta_skips_openrouter_latest_aliases_and_keeps_prices: live-shaped fixture (~alias,-1router, undecodable row, unreadable price, bad id, realdeepseek/deepseek-v4-prorow). It also checks that structurally invalid bodies are still rejected.client::tests::provider_live_price_parsers_reject_present_bad_rates_but_keep_zero_and_omission: updated so a negative OpenRouter price givescost: None.cost_status::tests::main_turn_rateless_declaration_yields_to_the_endpoint_catalog_pricecost_status::tests::openrouter_vendor_pin_is_priced_by_an_explicit_declared_ratecore::engine::tests::main_turn_dispatch_freezes_declared_custom_model_rate: now asserts the frozen rates.Only focused
cargo test -p codewhale-tui --libruns happened locally. The full suite is left to CI.🤖 Generated with Claude Code
https://claude.ai/code/session_014ZwqatxgVFxHvovngywnks