Skip to content

Per-session rate limit on tool calls #262

@esengine

Description

@esengine

Background

The budget system (cost / token caps, landed in 0.12.x) prevents runaway spending, but not runaway frequency. A loop bug or a model going in circles can fire hundreds of run_command calls in seconds — fine on cost terms (each call is cheap) but bad for filesystem / disk / spawn pressure, and a real DoW vector when the loop also makes API calls.

OWASP AI Agent Cheat Sheet § 5 cites "max_calls": 100, "window_seconds": 60 as the canonical pattern.

Proposal

Sliding-window rate limiter at the tool-dispatch boundary. Defaults (suggested, not load-bearing):

  • run_command / run_background: 60 calls / 60s
  • All tools combined: 200 calls / 60s

When the limit fires, the tool call returns an error to the model ("rate-limited, slow down"); the next loop tick can retry once the window slides. UI surfaces the throttle state.

Open design questions

This is rfc-tagged — please discuss in the issue before opening a PR:

  • Per-tool vs aggregate vs both?
  • Sliding window vs token bucket?
  • What does the model see — a real error, or a synthesized "wait" instruction the model can comply with?
  • How does this interact with the existing budget system — combine, or stay separate?
  • Configurable in user config? Per-project override?

Acceptance criteria (post-RFC)

  • Rate-limiter at ToolRegistry.dispatch (or wherever the boundary lands after design)
  • Configurable defaults
  • Clear error → model when throttled
  • Tests: window expiration, burst handling, multi-tool aggregation
  • Doctor / dashboard surface (optional, follow-up)

Out of scope

  • Cross-session limits (this is per-session)
  • Network-level rate limiting (handled by upstream API providers)

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is neededrfcArchitecture proposal / request for comments

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions