Skip to content

feat(mcp): token budget management with progressive disclosure#76

Merged
prosdev merged 1 commit intomainfrom
feat/71-token-budget-management
Nov 27, 2025
Merged

feat(mcp): token budget management with progressive disclosure#76
prosdev merged 1 commit intomainfrom
feat/71-token-budget-management

Conversation

@prosdev
Copy link
Copy Markdown
Collaborator

@prosdev prosdev commented Nov 27, 2025

Summary

Implements #71 - Token budget management for search results (sub-issue 5/5 of Epic #66)

Changes

New Parameters

  • Added tokenBudget parameter to dev_search tool (500-10000 tokens)
    • Compact mode default: 2000 tokens
    • Verbose mode default: 5000 tokens

Progressive Disclosure Strategy

  • Full detail (top 3 results): Complete snippet + imports + metadata
  • Signature detail (next 4 results): Signature + metadata (no snippet body)
  • Minimal detail (remaining): Header + location only
  • Truncation notice when budget exceeded

Formatter Options

Added to FormatterOptions:

  • progressiveDisclosure: Enable/disable (default: true)
  • fullDetailCount: Top N results with full detail (default: 3)
  • signatureDetailCount: Next M results with signatures (default: 4)
  • DetailLevel type: 'full' | 'signature' | 'minimal'

Token Estimation

Improved estimateTokensForText() with:

  • estimateCodeRatio() for code-heavy text detection
  • More accurate token estimation (3.5 chars/token for code vs 4 chars/token for prose)

Testing

  • 10 new tests for token budget management
  • All 1292 tests passing

Example Output

With budget of 2000 tokens and 10 results:

# Search Results (10 found, showing 7 within budget)

## 1. createPlan (function) - packages/subagents/src/planner/planner.ts
[Full snippet + imports + metadata]

## 2. handleMessage (method) - packages/subagents/src/planner/planner.ts
[Full snippet + imports + metadata]

## 3. PlannerAgent (class) - packages/subagents/src/planner/planner.ts
[Full snippet + imports + metadata]

## 4. breakdownTasks (function) - packages/subagents/src/planner/utils/breakdown.ts
[Signature only + metadata]

## 5. estimateEffort (function) - packages/subagents/src/planner/utils/estimation.ts
[Signature only + metadata]

...

---
*3 more results (token budget reached)*

Closes #71

Implements #71 - Richer Search Results (sub-issue 5/5)

Changes:
- Add tokenBudget parameter to dev_search tool (500-10000, default: 2000/5000)
- Add DetailLevel type: 'full' | 'signature' | 'minimal'
- Add progressive disclosure options to FormatterOptions:
  - progressiveDisclosure: enable/disable (default: true)
  - fullDetailCount: top N results with full detail (default: 3)
  - signatureDetailCount: next M results with signatures (default: 4)

CompactFormatter:
- formatResultWithDetail() for detail-level-aware formatting
- getDetailLevel() determines detail based on position and budget
- Truncation notice when budget exceeded

VerboseFormatter:
- Same progressive disclosure support
- Full: snippet + imports + signature + metadata
- Signature: signature + metadata (no snippet)
- Minimal: header + location only

SearchAdapter:
- tokenBudget parameter in tool definition
- Creates formatter with custom budget when specified
- Enables snippets/imports by default

Testing:
- 10 new tests for token budget management
- All 1292 tests passing

Closes #71
@prosdev prosdev merged commit 99ea625 into main Nov 27, 2025
1 check passed
@prosdev prosdev mentioned this pull request Nov 27, 2025
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Token budget management for search results

1 participant