feat(provider): add maxInputTokens window override and maxTokensAsOutputReserve - #234
Open
nam278 wants to merge 1 commit into
Open
feat(provider): add maxInputTokens window override and maxTokensAsOutputReserve#234nam278 wants to merge 1 commit into
nam278 wants to merge 1 commit into
Conversation
… advertising - deepseek-copilot.maxInputTokens overrides the advertised context window (0 = model default), driving Copilot Chat auto-compact thresholds - deepseek-copilot.maxTokensAsOutputReserve advertises maxTokens as the output reserve subtracted from the advertised input budget - advertise output capped at half the window so output never exceeds input when the override is smaller than the model's maximum output - refresh the model picker immediately when token-limit settings change
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
deepseek-copilot.maxInputTokens— overrides the context window (input + output) advertised to VS Code;0keeps each model's configured window (1M for DeepSeek V4)deepseek-copilot.maxTokensAsOutputReserve— when enabled,maxTokensis also advertised as the output reserve, so the advertised input budget becomeswindow − maxTokensWhy
Copilot Chat computes auto-compact thresholds against the advertised
maxInputTokens(background ~0.78, emergency ~0.90). The advertised limits were fixed model constants (input 655,360 / output 393,216), so history kept growing until compaction kicked in near the full 1M window — well past the region where DeepSeek V4 reliably uses its context. At that depth, long conversations degrade: the model starts losing fidelity on older turns and drifting ("lost-at-the-edges" effect), while every request carries hundreds of thousands of tokens of history.This override lets users set the effective window to ~200K: compaction then kicks in around 156K, keeping the conversation inside the reliable region, with fresh summaries instead of a bloated tail of raw messages. #225 asks for exactly this control.
User impact
maxTokensAsOutputReserve, setmaxTokens: 65000andmaxInputTokens: 265000→ advertised input 200,000, so auto-compact triggers at ~156K and the conversation stays inside the reliable context regionmaxInputTokens: 200000alone compacts even sooner (~78K trigger) for users who want aggressive cost control0and get the previous behavior (advertised input 606,784 / output 393,216, ~473K trigger)max_tokenssent to the API is untouched — only the advertised values changeValidation
npx tsc --noEmit— exit 0npm run lint(oxlint) — 0 warnings, 0 errorsnpm run format:check— pass200000→ 100,000 / 100,000 (trigger 78K);265000+ 65K reserve → 200,000 / 65,000 (trigger 156K)Closes #225