[Bug] Remote conversation got stuck: monologue detector false positive on extended thinking models

[VascoSch92]

Summary

Certain GAIA benchmark instances deterministically fail with Remote conversation got stuck across all retries, all SDK versions, and both Claude Sonnet 4.5 and Opus 4.6. The root cause is a conflict between the stuck detector's monologue threshold (3 consecutive agent MessageEvents) and extended thinking models that produce reasoning-only responses (no text, no tool calls) before their first action.

Affected instances

Instance	Question	Sonnet (run 1)	Sonnet (run 2)	Sonnet (run 3)	Opus (run 4)	Opus (run 5)
2d83110e	Reversed text puzzle	STUCK	STUCK	STUCK	STUCK	STUCK
2a649bb1	EC numbers for virus testing	PASS	STUCK	STUCK	STUCK	STUCK
6359a0b1	Green polygon area	FAIL	FAIL	FAIL	STUCK	STUCK
983bba7c	Animals in alvei papers	PASS	PASS	FAIL	STUCK	PASS

2d83110e fails in 100% of runs (5/5). 2a649bb1 fails in 4/5. The Opus runs are more affected (up to 4 stuck instances vs 1–2 for Sonnet).

Reproduction

Run the GAIA benchmark with Claude Sonnet 4.5 or Opus 4.6 with extended thinking enabled and OpenHands agent, Instance 2d83110e will fail every time.

Root cause

The code path (agent.py:298–382)

1. The LLM returns a valid response (chatcmpl-* ID) containing only thinking/reasoning blocks — no text content, no tool calls.
2. agent.py:302–309 detects has_reasoning and/or has_content:

   has_reasoning = (
       message.responses_reasoning_item is not None
       or message.reasoning_content is not None
       or (message.thinking_blocks and len(message.thinking_blocks) > 0)
   )
   has_content = any(
       isinstance(c, TextContent) and c.text.strip() for c in message.content
   )

3. Since there are no tool calls, execution falls through to line 356. If neither reasoning nor content is found, a warning is logged. But in all cases, a MessageEvent is emitted unconditionally (lines 359–372):

   msg_event = MessageEvent(source="agent", llm_message=message, ...)
   on_event(msg_event)  # Emitted even when empty

4. Since has_content is False, execution_status is not set to FINISHED (line 379–382), so the agent loop continues and calls the LLM again.
5. The LLM produces the same reasoning-only response. This repeats 3 times.

The stuck detection (stuck_detector.py:200–225)

6. _is_stuck_monologue() counts consecutive MessageEvents with source="agent":

   for event in reversed(events):
       if isinstance(event, MessageEvent):
           if event.source == "agent":
               agent_message_count += 1

7. The monologue threshold is 3 (types.py:40–41). After 3 empty MessageEvents, the conversation is marked STUCK.

The error propagation (remote_conversation.py:1035–1039)

8. The remote client detects the STUCK status and raises ConversationRunError("Remote conversation got stuck").

Why these specific instances

These questions cause the model to reason internally without producing a tool call:

• 2d83110e: Reversed text (.rewsna eht sa "tfel" drow eht fo etisoppo eht etirw). The model decodes it in thinking but doesn't know which tool to call for a pure-text answer — it just needs to call finish, but extended thinking fills the response with reasoning instead.
• 2a649bb1: Multi-step research question. The model plans which searches to run but doesn't commit to a tool call in the first 3 responses.

Evidence from conversation archives

Each retry shows the identical event sequence:

event-00000  SystemPromptEvent (agent)
event-00001  MessageEvent (user) — question delivered, content_len=1
event-00002  ConversationStateUpdateEvent — execution_status=running
event-00004  MessageEvent (agent) — content=[], thinking_blocks=[], resp_id=chatcmpl-...  [+2.0s]
event-00005  MessageEvent (agent) — content=[], thinking_blocks=[], resp_id=chatcmpl-...  [+1.6s]
event-00006  MessageEvent (agent) — content=[], thinking_blocks=[], resp_id=chatcmpl-...  [+1.6s]
event-00007  ConversationStateUpdateEvent — execution_status=stuck

Each empty response has a unique chatcmpl-* ID (real LLM calls, not cached), arrives in ~1.5–2s (vs 4–94s for normal first responses), and the thinking_blocks field is [] in the archived events (likely stripped or encrypted).

Why retries don't help

The retry mechanism escalates resource_factor (1→2→4→8), which increases pod CPU/memory. But the failure is in the model's response pattern, not resource exhaustion. Each retry gets a fresh runtime, fresh session, fresh conversation — but the same system prompt + question always produces the same model behavior.

Proposed solutions

Option 1: Don't emit MessageEvent for empty responses (minimal change)

In agent.py, skip the on_event(msg_event) call when the response has no content and no reasoning. This prevents empty messages from counting toward the monologue threshold.

# Line 356-372: only emit if there's something to emit
if has_reasoning or has_content:
    msg_event = MessageEvent(source="agent", llm_message=message, ...)
    on_event(msg_event)

Option 2: Inject a nudge after empty responses (more robust)

When the LLM returns an empty response, append a system/user message reminding it to use a tool:

if not has_reasoning and not has_content:
    logger.warning("LLM produced empty response - nudging to use tools")
    # Don't emit empty MessageEvent, instead continue loop
    # The next LLM call will include context about available tools
    continue

Option 3: Exclude empty MessageEvents from monologue detection (targeted fix)

In stuck_detector.py:200–225, skip MessageEvents that have no content:

if isinstance(event, MessageEvent):
    if event.source == "agent":
        # Don't count empty messages toward monologue
        if not event.llm_message or not event.llm_message.content:
            continue
        agent_message_count += 1

Option 4: Increase monologue threshold for extended thinking (configuration)

A threshold of 3 is too aggressive for extended thinking models that may need several reasoning passes. Increasing it to 5–6 would give the model more room while still catching genuine monologue loops.

Recommendation

Option 1 is the safest minimal fix — an empty response with no content, no reasoning, and no tool calls provides no value as a MessageEvent and should not be emitted. Combined with Option 4 (raising the threshold to 5 for extended thinking configs), this would eliminate the false positives without masking real stuck scenarios.

[enyst]

Re:

event-00000 SystemPromptEvent (agent)
event-00001 MessageEvent (user) — question delivered, content_len=1
event-00002 ConversationStateUpdateEvent — execution_status=running
event-00004 MessageEvent (agent) — content=[], thinking_blocks=[], resp_id=chatcmpl-... [+2.0s]
event-00005 MessageEvent (agent) — content=[], thinking_blocks=[], resp_id=chatcmpl-... [+1.6s]
event-00006 MessageEvent (agent) — content=[], thinking_blocks=[], resp_id=chatcmpl-... [+1.6s]
event-00007 ConversationStateUpdateEvent — execution_status=stuck

The empty thinking_blocks looks a bit strange. It would be great if we can capture the LLM responses in these 3 cases as they were, and then maybe try to understand what happened here.

Re: stuck on MessageEvent
If we need to, we could, I guess, make them 10 instead of 3, to see if it continues. I kinda doubt it helps though, if we saw nothing, it sounds like a bug on our side reading the LLM response...

Just to note, in my recollection we really wanted/needed to send thinking_blocks back wherever possible. So IMHO it would be worth checking why apparently we don't.

[VascoSch92]

Instance 2d83110e — Opus (conv 04c99959, last retry)

Event	Response ID	Latency	Completion Tokens	Reasoning Tokens	Content	Thinking Blocks
00004	chatcmpl-24fdcbad-91a6-4ba2-8580-bdffa5bb99ec	2.05s	2	0	[]	[]
00005	chatcmpl-0df1ade7-165a-4a60-85f0-cdd34727b68c	2.29s	2	0	[]	[]
00006	chatcmpl-27db0dbc-f675-40e9-8adb-3e74d7876254	1.97s	2	0	[]	[]

Instance 2d83110e — Sonnet (conv e2c80e8b)

Event	Response ID	Latency	Completion Tokens	Reasoning Tokens
00004	chatcmpl-8386c68d-5444-463b-8ede-74ee878cd6a2	1.10s	1	0
00005	chatcmpl-59f90522-c264-46e4-af0e-be3a265a0d1e	1.71s	1	0
00006	chatcmpl-a5eaede1-844a-4c02-b3cc-e5733950d863	2.56s	1	0

Instance 2a649bb1 — Opus (conv 5f112d0c)

Same pattern: 3 empty MessageEvents with content: [], thinking_blocks: [], reasoning_tokens: 0, completion_tokens: 2.

All responses have unique chatcmpl-* IDs (real LLM calls, not cached).

Key Finding: thinking_blocks: [] Is Accurate

The base_state.json token usage confirms:

• reasoning_tokens: 0 on every stuck response — the model produced no thinking blocks
• completion_tokens: 1–2 — just end-of-turn tokens
• Latency: 1.1–2.8s — too fast for actual extended thinking

This contradicts the issue hypothesis that the model returns "reasoning-only responses." The model returns completely empty responses: no reasoning, no content, no tool calls.

Comparison With Working Responses

Instance 983bba7c — Opus (finished successfully)

Response	Latency	Completion	Reasoning
1st	2.37s	4	0
2nd	9.27s	490	187
3rd	5.80s	425	0
4th	11.99s	505	187
5th	6.04s	346	0

The 1st response is also near-empty (4 tokens, 0 reasoning, 2.37s), but subsequent responses are normal. The model intermittently returns empty responses; 2d83110e triggers it 3+ times in a row.

Instance 6359a0b1 — Sonnet (finished normally)

• 1st response: 14.86s, completion=698, reasoning=471 — normal from the start

LLM Config (both models)

• enable_encrypted_reasoning: true
• extended_thinking_budget: 200000
• reasoning_effort: "high"
• stream: false
• native_tool_calling: true
• Routed through litellm_proxy at llm-proxy.eval.all-hands.dev

Since reasoning_tokens: 0, encryption is irrelevant — there was nothing to encrypt.

Revised Root Cause Hypothesis

The issue is not "extended thinking models producing reasoning-only responses." The LLM (via litellm proxy) returns completely empty responses (1–2 completion tokens, 0 reasoning tokens) in ~1–2s. For certain prompts (reversed text puzzle in 2d83110e), this happens deterministically, hitting the monologue threshold of 3.

The chatcmpl-* response ID prefix indicates litellm translates Anthropic responses to OpenAI format. The most likely cause is that litellm's translation layer drops thinking blocks and/or content, returning an effectively empty response. The ~2s latency supports this — a truly empty model response would be faster or error out.

Reproducing the Bug

Hard to reproduce outside the eval infra because:

• Responses go through litellm_proxy with OpenAI-format translation (chatcmpl-* IDs)
• enable_encrypted_reasoning: true is a server-side eval feature
• Model routing (litellm_proxy/anthropic/claude-opus-4-6) may differ from direct API calls

To isolate the layer:

1. Call the Anthropic API directly with the same prompt + extended thinking — if it works, the bug is in litellm
2. Call through litellm locally with enable_encrypted_reasoning + extended_thinking_budget — tests litellm's OpenAI format translation
3. Check litellm proxy logs for the specific chatcmpl-* IDs — shows the raw Anthropic response before translation (fastest path to confirmation)

Answers to @enyst's Questions

The empty thinking_blocks looks a bit strange. Can we capture the LLM responses as they were?

The token stats prove the model genuinely returned empty responses: reasoning_tokens: 0, completion_tokens: 1–2. The thinking_blocks: [] is the accurate raw state, not a serialization artifact.

Why apparently we don't send thinking_blocks back?

The model never produced thinking blocks for these responses. The question should shift to: why does litellm return completely empty responses when extended thinking is enabled? The litellm proxy logs for those chatcmpl-* IDs would show whether the Anthropic API response had content that was then dropped during format translation.

[juanmichelini]

@VascoSch92

The LLM returns a valid response (chatcmpl-* ID) containing only thinking/reasoning blocks — no text content, no tool calls.

I noticed this with the preflight hooks, nemotron-3-super was returning thinking blocks but empty messages. See #2420

The question was "1+1=" and nemotron answered 2 in the thinking block.

If that is the case,

Option 5

we could use the thinking block as a message block

[juanmichelini]

@VascoSch92

Remote conversation got stuck across all retries, all SDK versions, and both Claude Sonnet 4.5 and Opus 4.6.

1. Does this happen versions <v1.10 like v1.8.* ?
2. Did you test with new experiments or did you look at the old runs?

[VascoSch92]

@juanmichelini

1. with new versions of the sdk, i.e., strictly bigger than v1.11.5
2. new experiments/evals I was doing

I think there is a small issue coming from LiteLLM. I opened an issue in their repo (ref).

But this will take time. We want probably a solution from our side.

[VascoSch92]

I noticed this with the preflight hooks, nemotron-3-super was returning thinking blocks but empty messages. See #2420

Actually, I was evaluationg on a feature branch. I should rebase my branch and re-run the benchmark for the above instance to see if your fix is fixing that also.

[VascoSch92]

Let's summarize what we are doing in agent.py

LLM Response Pattern Matrix

Reference: agent.py L348-L433

Definitions

• tool_calls: message.tool_calls is non-empty
• content: at least one TextContent with non-blank .text in message.content
• reasoning: any of message.thinking_blocks (Anthropic), message.reasoning_content (o1/DeepSeek R1/Claude), or message.responses_reasoning_item (OpenAI Responses API) is present

Decision matrix

#	tool_calls	content	reasoning	Branch	Behavior	Terminal status
1	yes	yes	yes	L362	Execute tools; content attached as thought, reasoning attached to 1st action	— (continues)
2	yes	yes	no	L362	Execute tools; content attached as thought	— (continues)
3	yes	no	yes	L362	Execute tools; reasoning attached to 1st action, no thought	— (continues)
4	yes	no	no	L362	Execute tools; no thought, no reasoning	— (continues)
5	no	yes	yes	L410→L430	Emit MessageEvent, set FINISHED	FINISHED
6	no	yes	no	L410→L430	Emit MessageEvent, set FINISHED	FINISHED
7	no	no	yes	L410	Emit MessageEvent with empty content, agent loops	STUCK (after 3)
8	no	no	no	L407→L410	Log warning, emit MessageEvent with empty content, agent loops	STUCK (after 3)

Notes

• Rows 1–4: tool_calls take priority. The code enters L362 and returns after execution. Content and reasoning are metadata on the first ActionEvent only.
• Rows 5–6: has_content=True triggers FINISHED at L430-432. Conversation stops and awaits user input.
• Row 7 ⚠️: The model produced reasoning (e.g. thinking blocks) but no visible content. The MessageEvent is emitted but execution doesn't finish. After 3 consecutive agent MessageEvents the stuck detector fires. This is
  the pattern from issue [Bug] Remote conversation got stuck: monologue detector false positive on extended thinking models #2482 — litellm drops content but preserves thinking blocks, or the model answers only in the thinking block (nemotron "1+1=" case from PR fix: preflight check now validates reasoning_content for thinking models #2420).
• Row 8 ⚠️: Completely empty response (no reasoning, no content, no tools). Same stuck outcome as row 7. This is the pattern from instance 2d83110e — completion_tokens=2, reasoning_tokens=0.

This issue is in row 8 -> so i really need a fix in LiteLLM or I can augment the monologue.

Putting reasoning as content: I don't know if it is a good solution actually.

[enyst]

This issue is in row 8 -> so i really need a fix in LiteLLM or I can augment the monologue.

We could make the monologue at 10, if it helps. They're effectively retries, really, if there was nothing new sent in the next prompt.

An alternative, I think maybe we could also implement this as actual retries of llm.py completion(), maybe make the next time the temperature higher if it's possible with extended thinking, though I seem to recall it's not possible with most reasoning model providers. (we had a weird hack like that for Gemini 2.5 at some point, ... actually we might still have it in the codebase, LLMEmptyResponse handling or so, when we nudged the LLM to hopefully vary its responses so it's not empty anymore)

Probably best, if possible, we could ask litellm to prioritize that bug.