Skip to content

Bug: SSE chat completions report normal stop after agent task crashes #12422

@NewTurn2017

Description

@NewTurn2017

Bug Description

APIServerAdapter._write_sse_chat_completion() swallows agent exceptions and still terminates the SSE stream with finish_reason: "stop" plus [DONE]. A client receives what looks like a successful completion even when the underlying agent task crashed.

Affected Files / Lines

  • gateway/platforms/api_server.py:905-925

Why This Is a Bug

The code does:

try:
    result, agent_usage = await agent_task
    usage = agent_usage or usage
except Exception:
    pass

and then always emits a normal finish chunk with finish_reason: "stop" followed by [DONE].

This is a silent-failure path: downstream OpenAI-compatible clients cannot distinguish a real success from an internal agent crash.

Minimal Reproduction

A minimal reproduction with an agent_task that raises RuntimeError('boom from agent') and a queue that immediately sends the end-of-stream sentinel produced this SSE output:

data: {"choices": [{"delta": {"role": "assistant"}, ...}]}

data: {"choices": [{"delta": {}, "finish_reason": "stop"}], "usage": {"prompt_tokens": 0, ...}}

data: [DONE]

No error event or non-success status was emitted.

Expected Behavior

If the agent task fails before producing a valid final result, the SSE endpoint should surface an error (or at minimum avoid claiming a normal stop completion).

Actual Behavior

The stream ends as a successful completion even though the agent crashed.

Suggested Investigation Direction

  • Preserve and surface the exception from agent_task.
  • Align SSE error behavior with the non-streaming error path so API clients can detect failures.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High — major feature broken, no workaroundcomp/gatewayGateway runner, session dispatch, deliverytype/bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions