feat(provider): add free model resolution for --model free#34060
Open
caretak3r wants to merge 3 commits into
Open
feat(provider): add free model resolution for --model free#34060caretak3r wants to merge 3 commits into
caretak3r wants to merge 3 commits into
Conversation
Resolves --model free to the cheapest zero-cost listed model across all configured providers. Wires through CLI run, TUI args/app/thread, and provider service. Squashed: - feat: add free model resolution for --model free - refactor(provider): clean up free-model resolution - test(provider): align provider tests with WithInstance rename - fix(tui): wrap free-model resolution in WithInstance.provide - docs: tighten free-model resolution comments - test(provider): stabilize free model resolution assertions after rebase - refactor(provider): detect free models by cost only, remove isListed naming guard
Contributor
|
Thanks for updating your PR! It now meets our contributing guidelines. 👍 |
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue for this PR
Closes #21863
Type of change
What does this PR do?
Adds
--model freetoopencode runand the TUI. It picks one of the OpenCode Zen zero-cost models at random per invocation, so you don't have to type the model id when you just want "any free one".The filter is structural, not name-based:
providerID === opencode && all cost dimensions === 0. There is notier/zenfield in models.dev, andapi.urlis the sameopencode.ai/zen/v1for both the-freemodels and the zero-cost promo (big-pickle), so the id suffix isn't a contract worth coding against. Treating "free" as "zero-cost opencode model" means the resolver follows the live catalog without code changes when upstream rotates models in or out (saw this happen between rebases — caughtnorth-mini-code-freeandnemotron-3-ultra-freeautomatically).Also adds
--variant(and--variant anyto randomize across a model's listed variants), and threads model + variant resolution through bothopencode runandopencode --mini.Implementation notes worth flagging for review:
Provider.resolveSelectionrequires an explicitInstanceContextargument because it callsService.list()which needsInstanceRefin the Effect fiber. CLI handlers that run outsideeffectCmd(the TUI) lose the fiber, so the caller threads the ctx in. Without this, the resolver dies at runtime with "InstanceRef not provided" — and the unit tests don't catch it becauseit.instancealready provides the fiber. This is whybootstrap()now passes its loaded ctx to the callback.--minipath intui.tsdoesn't call the resolver itself;runMinidelegates toRunCommand.handler, which already has the fiber context. TheMiniCommandInputtype just gainedvariant?: stringso the TUI can pass it through.How did you verify your code works?
bun test test/provider/provider.test.ts test/cli/cmd/run.test.ts test/cli/tui/thread.test.ts— 114/114 pass. The provider tests log the live catalog count on startup so drift is visible.bun run buildsucceeds across all 12 targets, smoke test green.opencode models opencodeagainst the built binary returns 5 zero-cost models (4-free+big-pickle).opencode run --model free "say one word"produced all 5 unique models, distribution roughly uniform within statistical noise, and the LLM responded each time.opencode run --model free --variant any "hi"resolves correctly.packages/opencode/script/stress-free-models.shas a parametric stress test ([DURATION_SECONDS] [WORKERS]) so this stays easy to re-verify after future catalog rolls.One thing I noticed but didn't fix:
nemotron-3-ultra-freeconsistently takes ~13s to first token vs ~4s for the rest. Upstream-side latency on that specific model, not something the resolver controls. Heavy concurrent stress (>3 workers) gets throttled by the Zen endpoint, so the stress script defaults to 3 workers.Screenshots / recordings
n/a — CLI behavior only.
Checklist