Skip to content

feat(provider): add maxInputTokens window override and maxTokensAsOutputReserve - #234

Open
nam278 wants to merge 1 commit into
Vizards:mainfrom
nam278:feat/provider-advertised-token-window
Open

feat(provider): add maxInputTokens window override and maxTokensAsOutputReserve#234
nam278 wants to merge 1 commit into
Vizards:mainfrom
nam278:feat/provider-advertised-token-window

Conversation

@nam278

@nam278 nam278 commented Aug 13, 2026

Copy link
Copy Markdown

Summary

  • add deepseek-copilot.maxInputTokens — overrides the context window (input + output) advertised to VS Code; 0 keeps each model's configured window (1M for DeepSeek V4)
  • add deepseek-copilot.maxTokensAsOutputReserve — when enabled, maxTokens is also advertised as the output reserve, so the advertised input budget becomes window − maxTokens
  • cap the advertised output reserve at half the window so advertised output never exceeds input, even when the override is smaller than the model's maximum output
  • refresh the model picker immediately when any token-limit setting changes

Why

Copilot Chat computes auto-compact thresholds against the advertised maxInputTokens (background ~0.78, emergency ~0.90). The advertised limits were fixed model constants (input 655,360 / output 393,216), so history kept growing until compaction kicked in near the full 1M window — well past the region where DeepSeek V4 reliably uses its context. At that depth, long conversations degrade: the model starts losing fidelity on older turns and drifting ("lost-at-the-edges" effect), while every request carries hundreds of thousands of tokens of history.

This override lets users set the effective window to ~200K: compaction then kicks in around 156K, keeping the conversation inside the reliable region, with fresh summaries instead of a bloated tail of raw messages. #225 asks for exactly this control.

User impact

  • recommended recipe to combat lost-at-the-edges degradation: enable maxTokensAsOutputReserve, set maxTokens: 65000 and maxInputTokens: 265000 → advertised input 200,000, so auto-compact triggers at ~156K and the conversation stays inside the reliable context region
  • maxInputTokens: 200000 alone compacts even sooner (~78K trigger) for users who want aggressive cost control
  • users who prefer deep context keep 0 and get the previous behavior (advertised input 606,784 / output 393,216, ~473K trigger)
  • max_tokens sent to the API is untouched — only the advertised values change

Validation

  • npx tsc --noEmit — exit 0
  • npm run lint (oxlint) — 0 warnings, 0 errors
  • npm run format:check — pass
  • hand-verified advertised combos: defaults → 606,784 / 393,216 (trigger ~473K); 200000 → 100,000 / 100,000 (trigger 78K); 265000 + 65K reserve → 200,000 / 65,000 (trigger 156K)

Closes #225

… advertising

- deepseek-copilot.maxInputTokens overrides the advertised context window
  (0 = model default), driving Copilot Chat auto-compact thresholds
- deepseek-copilot.maxTokensAsOutputReserve advertises maxTokens as the
  output reserve subtracted from the advertised input budget
- advertise output capped at half the window so output never exceeds
  input when the override is smaller than the model's maximum output
- refresh the model picker immediately when token-limit settings change
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

怎么调整 copilot 第三方模型的保留用于响应值设置,溢出直接开始压缩

1 participant