Skip to content

fix: compaction hangs when compaction model is rate-limited #2062

@youngbinkim0

Description

@youngbinkim0

Context

Reported by @rothnic in PR #1777 — when the compaction/summarization model is rate-limited, the session hangs indefinitely instead of recovering.

Related PR: #1777

Problem

When context window recovery triggers compaction and the compaction model is rate-limited:

  1. runSummarizeRetryStrategy in src/hooks/anthropic-context-window-limit-recovery/summarize-retry-strategy.ts has only 2 retries with 2s/4s backoff. There is no rate-limit-specific handling and no model fallback. After exhausting retries, the session hangs.

  2. Preemptive compaction in src/hooks/preemptive-compaction.ts has zero retry on compaction failure — if the first attempt is rate-limited, it silently fails and the session continues until it hits the hard context limit.

Neither path has awareness of rate-limit errors or the ability to fall back to an alternative compaction model.

Root Cause

  • summarize-retry-strategy.ts uses a generic retry loop (2 attempts, 2s/4s delays) that treats all errors identically. Rate-limit errors with retry-after headers are not detected or respected.
  • preemptive-compaction.ts wraps the compaction call in a try/catch that logs and returns on failure — no retry at all.
  • compaction-model-resolver.ts resolves a single model from config with no fallback chain.
  • The runtime-fallback hook (src/hooks/runtime-fallback/) only handles the main chat model, not compaction/summarization models.

Suggested Fix

  • Add rate-limit detection in the summarize retry strategy (check for 429 status, retry-after header)
  • Implement a fallback compaction model resolution (try primary → fallback model)
  • Add at least basic retry to preemptive compaction
  • Consider extending runtime-fallback awareness to compaction calls

Key Files

  • src/hooks/anthropic-context-window-limit-recovery/summarize-retry-strategy.ts — 2 retries, no rate-limit handling
  • src/hooks/preemptive-compaction.ts — Zero retry on failure
  • src/shared/compaction-model-resolver.ts — Single model, no fallback chain
  • src/hooks/runtime-fallback/ — Only handles main chat model

cc @rothnic @davidakerr

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions