Mid-turn compaction fails with ModelHttpError: Bad Request on large transcripts

[haliphax]

Summary

Mid-turn compaction repeatedly fails with ModelHttpError: Bad Request when the session transcript is large enough that the summarization payload exceeds the compaction model's input limits.

Observed Behavior

During a tool-loop turn that started with 124 messages / ~477K chars of content, mid-turn compaction triggered 3 times and failed each time:

{"ts":"2026-04-22T03:28:39.788Z","level":"warn","msg":"mid-turn compaction failed, proceeding with current messages","component":"session-tool-loop-model-client","sessionId":"agent:developer:discord:channel:1480957862858719232","err":"ModelHttpError: Bad Request"}
{"ts":"2026-04-22T03:29:00.383Z","level":"warn","msg":"mid-turn compaction failed, proceeding with current messages",...}
{"ts":"2026-04-22T03:29:13.237Z","level":"warn","msg":"mid-turn compaction failed, proceeding with current messages",...}

The turn still completed (the catch block falls through gracefully), but compaction never succeeded, so context was never reduced.

Root Cause Analysis

Traced through the code path:

1. session-tool-loop-model-client.ts → complete() estimates tokens, triggers compaction when over threshold
2. transcript-compact.ts → compactSessionTranscript() loads transcript, strips images/thinking, calls compactTranscriptIfNeeded()
3. compaction.ts → compactTranscriptIfNeeded() splits messages into system prefix, middle (to summarize), and tail (preserveRecentMessages=8). The entire middle section is concatenated into a single user message and sent to the compaction model.

With 124 messages and ~477K chars total, the "middle" portion (116 messages) likely exceeded 400K chars. This entire blob was sent as one summarization request to kiro/claude-haiku-4.5. The upstream gateway returned HTTP 400, presumably because the payload exceeded the model's actual input token limit.

Each tool-loop iteration re-checks the token estimate, finds it still over threshold, and retries compaction — which fails the same way.

Issues Identified

1. No input size cap for compaction summarization

compactTranscriptIfNeeded() has no truncation, chunking, or size guard. It sends the entire middle section regardless of size. For very long sessions, this can exceed what the compaction model can handle.

File: packages/models/src/compaction.ts (line ~93, compactTranscriptIfNeeded)

2. Error logging loses the response body

The catch block in session-tool-loop-model-client.ts:165 logs String(e) which yields ModelHttpError: Bad Request. The ModelHttpError class has a bodySnippet field with the actual rejection reason from the gateway, but it's never logged.

File: packages/daemon/src/sessions/session-tool-loop-model-client.ts (line ~165)

3. Repeated futile retries

Each tool-loop iteration re-triggers compaction even though nothing changed since the last failure. There's no backoff or "already failed this turn" flag.

Suggested Fixes

• Truncate or chunk the compaction input to stay within the compaction model's context window (could use contextWindowTokens from the model config)
• Log e.bodySnippet alongside the error message for better diagnostics
• Set a per-turn failure flag so compaction isn't retried after it fails once in the same tool loop
• Consider a maxCompactionInputTokens config knob for operator control

Config Context

{
  "models": {
    "compaction": {
      "model": "kiro/claude-haiku-4.5"
    }
  }
}

preserveRecentMessages was at the default of 8. contextWindowReserveTokens was at the default of 20,000.

Environment

• Shoggoth running in Docker
• Model provider: anthropic-messages kind via gateway at http://kiro:8000
• Session: agent:developer:discord:channel:1480957862858719232