Add OpenRouter as a model provider - #487
Open
dsaad68 wants to merge 5 commits into
Open
Conversation
fx reaches models through three providers today: Vercel AI Gateway, Codex, and Grok. All three need either Vercel billing or a paid consumer subscription, and every base-URL override is restricted to loopback HTTP, so there is no way to point fx at another endpoint. OpenRouter adds ~400 models behind a single API key, including a set that cost nothing to run. That is the motivating outcome: a user with no Vercel billing and no ChatGPT or Grok subscription can now run fx for free. OpenRouter speaks the OpenAI Chat Completions format, which neither existing protocol module covers (vercel_protocol.zig is Vercel v3, responses_protocol.zig is the OpenAI Responses API). The new chat_completions_protocol.zig carries no vendor identity so any future OpenAI-compatible route can share it. It accumulates streamed tool calls by their `index` field and skips SSE comment lines, which OpenRouter emits as `: OPENROUTER PROCESSING` keep-alives mid-stream. Auth is a plain API key from OPENROUTER_API_KEY, following the existing ai_gateway_api_key shape rather than adding an OAuth flow. There is no stored session, so `fx logout openrouter` says so instead of falling through to the Vercel logout. Because fx calls tools on every turn, the catalog fetches only tool-capable models (?supported_parameters=tools); a model that cannot call tools breaks on the first step. Free models are identified by published pricing, sorted first, and surfaced four ways: a `Free` fact in the /model menu, a marker in `fx models`, a `free` field in its JSON, and a `--free` filter. The catalog's `is_free` flag is authoritative; the `:free` id suffix is a display convenience for surfaces that only carry id strings, and a fixture test pins the two together. Usage is exact: OpenRouter reports token counts and credit cost inline on the terminal chunk, so no deferred reconciliation is needed. The 402, 429, and 503 responses carry plain-language detail, since a negative balance blocks even free models and free-tier requests are capped at 20/min and 50/day. Ordering is this provider's own. compareModelCatalogEntries and projectPickerModelCatalog encode Vercel-specific product policy and are gateway-only, so OpenRouter is deliberately not routed through them. Verified against the live OpenRouter API and, offline, against a fake in tests/e2e/openrouter-stream.test.ts covering catalog filtering, free-first ordering, a streamed tool-call round trip, and the error paths. Unit tests cover the reducer, catalog parsing, and request serialization. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The first commit wired OpenRouter through the CLI and the /model menu but missed /setup entirely, so the provider was unreachable from the TUI: the Model provider screen enumerated a hardcoded three entries and stopped at Grok. Selecting OpenRouter required `fx provider openrouter` from a shell. The provider stage now offers every ProviderId, and a test walks std.meta.tags to assert that, so a future provider cannot be added without appearing here. Connections gains an OpenRouter row reporting whether the key is present in the environment. It starts no sign-in flow, because there is none: selecting it explains that OPENROUTER_API_KEY is read from the environment. The row also appears during onboarding, where a user with no Vercel billing and no subscription most needs to learn a free path exists. credential_source_order was missing the key, so probing never recorded it and Connections could not have reported its status. Adding it exposed that the Credential source screen filtered subscriptions by listing them individually; that is now an isGatewaySource predicate, which keeps provider-scoped keys out of a screen that picks a Gateway credential and fails closed for any source added later. Verified in a real terminal: Model provider lists OpenRouter as current, Connections reports "environment" when the key is set, and selecting that row renders the guidance notice. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A sweep for the same defect class as the setup-hub miss found four more places that enumerate providers by hand, where the compiler cannot help: - The ACP provider config option was a hardcoded JSON list, so an ACP client could report OpenRouter as current but never offer it as a choice. - fx status and doctor built connected_providers from three explicit checks, omitting OpenRouter even while reporting OPENROUTER_API_KEY as the auth source. - A tools-disabled host profile cleared the reviewer for three providers by name, leaving OpenRouter's active in a profile with no tools. - Missing-credential guidance fell through an if/else chain to the Gateway message, telling an OpenRouter user to run `fx login`. That chain is now missingCredentialMessage/missingInteractiveCredentialMessage, a switch, so a new provider must state its own guidance rather than inherit Vercel's. Token facts printed the raw window for values that are not round decimals. Gateway models are advertised in round numbers so this never showed, but OpenRouter reports powers of two, rendering "1048576 context · 230400 output". Non-round values now round to the nearest unit; exact multiples are untouched. Verified interactively: the /model menu shows "1M context · 230K output · Free" with the OpenRouter catalog status line, ACP lists all four providers with openrouter current, and fx status reports OpenRouter under connected_providers in both text and JSON. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
dsaad68
force-pushed
the
feat/openrouter-provider
branch
from
August 28, 2026 00:13
cd4d0a6 to
e5855e3
Compare
The upstream draft was closed, so the rationale it carried — why a new chat-completions protocol module was needed, why the catalog filters to tool-capable models, and how free models are surfaced — would otherwise only survive in a closed pull request. Not part of the feature: it documents the change rather than shipping with it, and should be removed before any future upstream submission so it does not appear in the diff. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds OpenRouter as a model provider, alongside Vercel AI Gateway, Codex, and Grok.
All three existing routes need either Vercel billing or a paid consumer subscription, and every base-URL override is restricted to loopback HTTP, so there is no way to point fx at another endpoint. OpenRouter adds ~400 models behind a single API key, including a set that cost nothing to run — so a user with no Vercel billing and no ChatGPT or Grok subscription can run fx for free.
Design notes
A new wire format. OpenRouter speaks OpenAI Chat Completions, which neither existing protocol module covers (
vercel_protocol.zigis Vercel v3,responses_protocol.zigis the OpenAI Responses API).chat_completions_protocol.zigcarries no vendor identity so any future OpenAI-compatible route can share it. It accumulates streamed tool calls by theirindexfield and skips SSE comment lines, which OpenRouter emits as: OPENROUTER PROCESSINGkeep-alives mid-stream — feeding one to a JSON parser aborts an otherwise healthy stream.API key, no OAuth. Auth follows the existing
ai_gateway_api_keyshape viaOPENROUTER_API_KEY. There is no stored session, sofx logout openroutersays so rather than falling through to the Vercel logout.Tool-capable models only. fx calls tools every turn, so the catalog fetches with
?supported_parameters=tools; a model that cannot call tools breaks on the first step.Free models are surfaced deliberately. Identified by published pricing, sorted first, and shown four ways: a
Freefact in the/modelmenu, a marker infx models, afreefield in its JSON, and a--freefilter.is_freeis authoritative; the:freeid suffix is a display convenience for surfaces that only carry id strings, and a fixture test pins the two together.Exact usage. OpenRouter reports token counts and credit cost inline on the terminal chunk, so no deferred reconciliation is needed. The 402, 429, and 503 responses carry plain-language detail, since a negative balance blocks even free models and free-tier requests are capped at 20/min and 50/day.
Ordering stays provider-owned.
compareModelCatalogEntriesandprojectPickerModelCatalogencode Vercel-specific product policy and are gateway-only, so OpenRouter is deliberately not routed through them.Commits
ProviderIdplumbing./setupenumerated a hardcoded three providers, so it was unreachable from the TUI.fx status/doctor, the tools-disabled profile, and credential guidance.Where a hand-maintained list caused a miss, it was replaced with something the compiler or a test enforces:
isGatewaySource,missingCredentialMessage, and a test that walksstd.meta.tags(ProviderId)to assert every provider is reachable from the setup picker.Testing
tests/e2e/openrouter-stream.test.ts— catalog filtering, free-first ordering, the--freefilter, a streamed tool-call round trip, and the 402/429 paths. Classified incorpus.jsonas verification-only with a shard weight./setup→ Model provider and Connections,/model, and ACP config options driven in a real terminal.Known gap
FailureKindhas no payment-required variant, so a 402 maps toforbidden— correct non-retryable semantics, but it surfaces as "HTTP 403". The detail names the real status so the message cannot mislead. Happy to add apayment_requiredvariant instead if you'd prefer it fixed at the enum.Note on scope
This is a fork-driven contribution and I recognize OpenRouter is a model router, so it overlaps with AI Gateway in a way that Codex and Grok do not. Opening as a draft to check appetite before polishing further — happy to close if it is not a direction you want.
🤖 Generated with Claude Code