[Feature]: Add per-agent rate limiting middleware to prevent abuse and ensure fair usage

[janhavitupe]

Overview

Currently, there is no rate limiting or request throttling mechanism in the system. This allows any agent or client to send an unbounded number of requests, which can lead to resource exhaustion, degraded performance, and potential abuse.

Problem

• No per-agent request limits exist
• A single agent/client can monopolize system resources
• No protection against accidental or malicious high-frequency requests
• System stability and fairness are not enforced

Proposed Solution

Introduce a per-agent rate limiting middleware that enforces request limits using a standard algorithm such as:

• Token Bucket (preferred for flexibility), or
• Sliding Window

Expected Behavior

• Each agent has a configurable request rate limit (e.g., X requests per second/minute)
• Requests exceeding the limit are:
  • Rejected with a clear error response (e.g., HTTP 429), or
  • Delayed (optional, depending on design)
• Rate limits should be configurable via settings/environment variables
• System should support independent limits per agent

Implementation Ideas

• Middleware layer that intercepts incoming requests
• Maintain in-memory counters or token buckets per agent ID
• Optional extensibility for Redis-backed storage (for distributed setups)
• Lightweight and non-blocking design

Additional Considerations

• Logging when rate limits are exceeded
• Clear error messaging for clients
• Avoid introducing significant latency
• Ensure thread-safe or async-safe implementation

Why this matters

Rate limiting is a fundamental requirement for production systems to:

• Prevent abuse
• Ensure fairness across agents
• Protect system resources
• Improve reliability under load

This would significantly improve the robustness of the framework.