Skip to content

fix(opencode): honest context arithmetic for small and unreported model limits - #44898

Open
yanglinfang wants to merge 8 commits into
anomalyco:devfrom
yanglinfang:pr1-context-arithmetic
Open

fix(opencode): honest context arithmetic for small and unreported model limits#44898
yanglinfang wants to merge 8 commits into
anomalyco:devfrom
yanglinfang:pr1-context-arithmetic

Conversation

@yanglinfang

@yanglinfang yanglinfang commented Aug 25, 2026

Copy link
Copy Markdown

Issue for this PR

Closes #41372

Second slice of #44242 (being split into small PRs per review feedback; first slice #44895). #41372 is the umbrella issue for the series — if this merge auto-closes it before the remaining slices land, it should be reopened.

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Fixes the context arithmetic that makes opencode unusable with small local models:

  1. Unset limit.context disabled compaction entirely. Routers and local providers frequently report 0; sessions then ran straight into provider overflow errors. Now a conservative 32k usable window applies, shrunk further by a per-session cap learned from actual provider overflow rejections (logged once per session).
  2. Unset limit.output was assumed to be 32k. usable = context − 32000 reaches 0 on any window under 32k, so every completed step counts as overflow and compaction re-triggers after every single step. The fallback is now window-proportional (25%, floor 1024), and each request additionally clamps its output budget to the headroom the estimated input actually leaves, so a long history can't push a request over the window.
  3. The compaction reserve was effectively fixed at 20k — 36% of a 56k local window. Now proportional: min(20k, max(2048, 15% of window)). compaction.reserved config keeps absolute priority and windows ≥133k keep today's 20k unchanged.

How did you verify your code works?

New unit suites for the reserve formula, usable-window derivation, learned caps, and the per-request clamp; the changed upstream expectation (context: 0 → conservative window instead of never-overflow) is updated with both directions asserted. Full packages/opencode suite green on current dev.

End-to-end A/B on the same machine, model (qwen3:4b-instruct via ollama), config (context: 16384, output: 0), and coding task:

  • without this fix: 30 compactions in 15 minutes — one after every step; the model re-reads the same file forever, zero files changed, killed at the cap
  • with it: 9 steps, 0 compactions, task completed and verified in ~7 minutes

Screenshots / recordings

Evidence panels for the A/B run to be attached below (no UI change in the code itself).

before and after fix

05-study-stock-compaction-loop 06-study-ours-task-complete

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

…el limits

Three fixes to the budget arithmetic that breaks small local models:

- unset/zero limit.context disabled proactive compaction entirely; now a
  conservative 32k usable window applies, shrunk by a per-session cap
  learned from provider overflow rejections, with a one-time warning
- the compaction reserve was min(20k, maxOutputTokens) — 36%% of a 56k
  local window; now proportional min(20k, max(2048, 15%% of the window)),
  with compaction.reserved config keeping absolute priority
- unset limit.output was assumed to be 32k, so usable = context - 32000
  reached 0 on any window under 32k and compaction re-triggered after
  every step; now the fallback is window-proportional (25%%, floor 1024)
  and each request additionally clamps its output budget to the headroom
  the estimated input actually leaves

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for your contribution!

This PR doesn't have a linked issue. All PRs must reference an existing issue.

Please:

  1. Open an issue describing the bug/feature (if one doesn't exist)
  2. Add Fixes #<number> or Closes #<number> to this PR description

See CONTRIBUTING.md for details.

@github-actions

Copy link
Copy Markdown
Contributor

The following comment was made by an LLM, it may be inaccurate:

Based on the search results, I found one related PR (excluding the current PR #44898 which appears in results):

Related PR:

However, this is not a duplicate but rather a parent/predecessor PR as part of the same feature split. The current PR is a focused bug fix for context arithmetic that's part of a larger effort to support small/local models.

No duplicate PRs found

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE]: Model capability class in metadata — size-appropriate prompts and tool behavior for small/local models

1 participant