Skip to content

feat(provider): add free model resolution for --model free#34060

Open
caretak3r wants to merge 3 commits into
anomalyco:devfrom
caretak3r:feat/free-model-resolution
Open

feat(provider): add free model resolution for --model free#34060
caretak3r wants to merge 3 commits into
anomalyco:devfrom
caretak3r:feat/free-model-resolution

Conversation

@caretak3r

@caretak3r caretak3r commented Jun 26, 2026

Copy link
Copy Markdown

Issue for this PR

Closes #21863

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Adds --model free to opencode run and the TUI. It picks one of the OpenCode Zen zero-cost models at random per invocation, so you don't have to type the model id when you just want "any free one".

The filter is structural, not name-based: providerID === opencode && all cost dimensions === 0. There is no tier / zen field in models.dev, and api.url is the same opencode.ai/zen/v1 for both the -free models and the zero-cost promo (big-pickle), so the id suffix isn't a contract worth coding against. Treating "free" as "zero-cost opencode model" means the resolver follows the live catalog without code changes when upstream rotates models in or out (saw this happen between rebases — caught north-mini-code-free and nemotron-3-ultra-free automatically).

Also adds --variant (and --variant any to randomize across a model's listed variants), and threads model + variant resolution through both opencode run and opencode --mini.

Implementation notes worth flagging for review:

  • Provider.resolveSelection requires an explicit InstanceContext argument because it calls Service.list() which needs InstanceRef in the Effect fiber. CLI handlers that run outside effectCmd (the TUI) lose the fiber, so the caller threads the ctx in. Without this, the resolver dies at runtime with "InstanceRef not provided" — and the unit tests don't catch it because it.instance already provides the fiber. This is why bootstrap() now passes its loaded ctx to the callback.
  • --mini path in tui.ts doesn't call the resolver itself; runMini delegates to RunCommand.handler, which already has the fiber context. The MiniCommandInput type just gained variant?: string so the TUI can pass it through.

How did you verify your code works?

  • bun test test/provider/provider.test.ts test/cli/cmd/run.test.ts test/cli/tui/thread.test.ts — 114/114 pass. The provider tests log the live catalog count on startup so drift is visible.
  • bun run build succeeds across all 12 targets, smoke test green.
  • opencode models opencode against the built binary returns 5 zero-cost models (4 -free + big-pickle).
  • Ran the binary in a loop: 12 calls of opencode run --model free "say one word" produced all 5 unique models, distribution roughly uniform within statistical noise, and the LLM responded each time.
  • opencode run --model free --variant any "hi" resolves correctly.
  • Committed packages/opencode/script/stress-free-models.sh as a parametric stress test ([DURATION_SECONDS] [WORKERS]) so this stays easy to re-verify after future catalog rolls.

One thing I noticed but didn't fix: nemotron-3-ultra-free consistently takes ~13s to first token vs ~4s for the rest. Upstream-side latency on that specific model, not something the resolver controls. Heavy concurrent stress (>3 workers) gets throttled by the Zen endpoint, so the stress script defaults to 3 workers.

Screenshots / recordings

n/a — CLI behavior only.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

Resolves --model free to the cheapest zero-cost listed model across all
configured providers. Wires through CLI run, TUI args/app/thread, and
provider service.

Squashed:
- feat: add free model resolution for --model free
- refactor(provider): clean up free-model resolution
- test(provider): align provider tests with WithInstance rename
- fix(tui): wrap free-model resolution in WithInstance.provide
- docs: tighten free-model resolution comments
- test(provider): stabilize free model resolution assertions after rebase
- refactor(provider): detect free models by cost only, remove isListed naming guard
@github-actions github-actions Bot added the needs:compliance This means the issue will auto-close after 2 hours. label Jun 26, 2026
@github-actions github-actions Bot removed the needs:compliance This means the issue will auto-close after 2 hours. label Jun 26, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for updating your PR! It now meets our contributing guidelines. 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE]: Allow --model free to randomly select free models

1 participant