[bug] reasoning_content stripped from requests: breaks reasoning-required upstreams (DeepSeek thinking) on multi-turn

[DuanZGit]

Symptom

Multi-turn chat with a reasoning-required OpenAI-compatible upstream (e.g. DeepSeek's thinking models) fails on the second turn with:

400 invalid_request_error: The `reasoning_content` in the thinking mode must be passed back to the API.

and, in tool-calling flows:

400 invalid_request_error: Messages with role 'tool' must be a response to a preceding message with 'tool_calls'

The first turn works fine; any follow-up turn that carries back the assistant message from turn 1 fails.

Reproduction

1. Configure a route whose backend is a reasoning/thinking OpenAI-compatible model that requires reasoning_content to be passed back verbatim on subsequent turns (DeepSeek v4 flash/pro "deepseek-reasoner"-style models do this).
2. Send a first request; the response includes reasoning_content on the assistant message (switchyard's encode_response correctly emits it).
3. Send a second request whose messages array includes that assistant message with its reasoning_content (i.e. a client that faithfully passes history back).

Expected: second turn succeeds.

Actual: switchyard strips the reasoning field from the request before forwarding, so the upstream rejects the turn with the 400 above.

# turn 1 (ok) — response contains reasoning_content
curl -s http://127.0.0.1:4000/v1/chat/completions \
  -d '{"model":"smart1","messages":[{"role":"user","content":"hi"}],"stream":false}'

# turn 2 (400) — history includes assistant reasoning_content from turn 1
curl -s http://127.0.0.1:4000/v1/chat/completions \
  -d '{"model":"smart1","messages":[{"role":"user","content":"hi"},
       {"role":"assistant","content":"...","reasoning_content":"thinking..."}],"stream":false}'

Expected vs. actual

• Expected: the assistant message's reasoning_content is forwarded to the upstream so reasoning-required providers can validate the turn.
• Actual: reasoning_content is silently dropped on the request path; providers that enforce "reasoning_content must be passed back" reject with 400.

Root cause (code pointer)

crates/switchyard-translation/src/codecs/openai_chat/buffered.rs, encode_message_without_tool_results_to_openai:

let content_blocks = message
    .content
    .iter()
    .filter(|block| {
        !matches!(
            block,
            ContentBlock::ToolCall(_) | ContentBlock::Reasoning { .. }
        )
    })

ContentBlock::Reasoning is explicitly filtered out of the encoded request, even though the response encoder (encode_response, same file) re-emits it as reasoning_content. This asymmetry breaks round-tripping for providers that require reasoning to be echoed back (DeepSeek thinking, some vLLM reasoning templates).

Note: prepend_openai_reasoning_blocks merges incoming reasoning into content blocks on the decode side, but there is no corresponding encode-side path that restores a standalone reasoning_content field on assistant messages in the request.

Suggested direction

When encoding a request for an OpenAI-compatible upstream, preserve ContentBlock::Reasoning from assistant messages as a top-level reasoning_content (or reasoning) field instead of dropping it — ideally gated by a policy/extension so providers that reject unknown fields are unaffected. This mirrors the response-side behavior and makes switchyard safe for reasoning-required providers.

Environment

• Switchyard: main @ 2026-08-14 (recent commits: feat(libsy-llm-client) etc.)
• Inbound format: Chat Completions
• Backend: OpenAI-compatible reasoning model (DeepSeek thinking)
• Route type: deterministic routing with a classifier LLM

Additional context

Workaround used in production: a thin gateway in front of switchyard caches the last assistant reasoning_content per session and re-injects it into outbound requests when the client omits it. This confirms the upstream requirement and that the fix belongs in the translation layer. Happy to prepare a patch + tests if the direction is approved.

[ting-hong-shieh]

I'd like to take this one.
Confirmed against main (read 2026-08-16), crates/switchyard-translation/src/codecs/openai_chat/buffered.rs:

• Decode side: prepend_openai_reasoning_blocks (L391) reads reasoning_content and reasoning off the wire and turns them into ContentBlock::Reasoning.
• Encode side: encode_message_without_tool_results_to_openai (L795-L800) filters ContentBlock::Reasoning out of content_blocks, and the multipart branch in encode_content_blocks (L935-L937) drops it as well.

So the round trip is lossy in one direction, which matches the 400 on the second turn.
One detail to add to the report: the drop is silent. The audio, video, and unknown-block arms in the same function call push_lossy, so LossyConversionPolicy can reject or record them; the Reasoning arm returns {} with no diagnostic. A gateway operator gets no signal that reasoning was discarded.

Plan, unless a maintainer wants a different shape:

1. Encode ContentBlock::Reasoning as a top-level reasoning_content string on assistant messages, mirroring the decode path. Emit reasoning_content only; keep reasoning read-only on decode for compatibility.
2. Gate the field on target capability rather than emitting it unconditionally, since providers with strict unknown-field handling will reject it.
3. Call push_lossy when the block is dropped because the gate is off.
4. Round-trip tests: decode a DeepSeek-style response carrying reasoning_content, re-encode it as request history, assert the field survives; plus a gated-off case asserting the field is absent and the diagnostic is present.

Open question before I write it. For step 2, do you want a new TargetCapabilities flag (supports_reasoning_content, next to the existing supports_reasoning_effort), or should this key off ProviderProfile.provider? I lean toward the capability flag so it stays provider-agnostic, but it changes the public surface, so it's your call.

I have not reproduced the 400 against a live DeepSeek endpoint; my evidence is the code path above plus @DuanZGit's report. A captured request/response pair would make the test fixture more faithful if anyone has one.

[DuanZGit]

Here is a faithful, redacted request/response pair from our production DeepSeek thinking setup (provider-agnostic structure; values replaced). This is the exact pattern that hits the 400 without the fix, and that the round-trip test should cover.

Turn 1 response (upstream → client, stream: false, simplified)

{
  "id": "chatcmpl-<redacted>",
  "object": "chat.completion",
  "created": 1755379200,
  "model": "deepseek-v4-flash",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "I'll call the tool.",
        "reasoning_content": "The user wants to compute 2+2. I have a calculator tool. Let me call it.\n\n<tool_plan>\n1. invoke calculator(2, 2)\n2. return result\n</tool_plan>",
        "tool_calls": [
          {
            "id": "call_abc123",
            "type": "function",
            "function": {"name": "calculator", "arguments": "{\"a\": 2, \"b\": 2}"}
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ]
}

Turn 2 request (client → gateway → upstream, as our workload sends it)

Note: the client does NOT carry reasoning_content back on the assistant history message (this is what triggers the flow). With switchyard stripping ContentBlock::Reasoning on encode, the upstream sees no reasoning_content at all.

{
  "model": "deepseek-v4-flash",
  "stream": false,
  "messages": [
    {"role": "user", "content": "What is 2+2? Use the calculator."},
    {
      "role": "assistant",
      "content": "I'll call the tool.",
      "tool_calls": [
        {
          "id": "call_abc123",
          "type": "function",
          "function": {"name": "calculator", "arguments": "{\"a\": 2, \"b\": 2}"}
        }
      ]
    },
    {"role": "tool", "tool_call_id": "call_abc123", "content": "4"}
  ]
}

Upstream 400 (both variants seen in the wild)

{
  "error": {
    "message": "The `reasoning_content` in the thinking mode must be passed back to the API",
    "type": "invalid_request_error",
    "code": "invalid_message"
  }
}

{
  "error": {
    "message": "Messages with role 'tool' must be a response to a preceding message with 'tool_calls'",
    "type": "invalid_request_error"
  }
}

Expected round-trip behavior after the fix: re-encoding the Turn 1 message as request history yields the assistant message with BOTH tool_calls intact AND a top-level reasoning_content string, so Turn 2 goes through.

Happy to provide a streamed variant (SSE with delta.reasoning_content deltas interleaved with delta.content) if useful — that's the other pattern our gateway has to cache/backfill, and it exercises the multipart branch at L935-937.

[DuanZGit]

Thanks for taking this on — and for the extra detail on the silent drop (+1: a push_lossy diagnostic on the gated-off path would definitely help operators).

On the open question: I'd vote for the capability flag over keying off ProviderProfile.provider. Our gateway routes to multiple upstreams from one config (DeepSeek today, others behind the same route), so a provider-agnostic capability keeps the encode path free of a provider table. supports_reasoning_content sitting next to supports_reasoning_effort feels consistent.

The fixture above is from a non-streamed turn. I can also produce a streamed variant (SSE with delta.reasoning_content interleaved with delta.content, finishing with a tool_calls delta) — that exercises the multipart branch at L935-937, and it's the other pattern our gateway has to cache/backfill today. Happy to attach it if it helps the round-trip tests.

Also happy to test the fix against our live DeepSeek endpoint once the PR branch exists — that would close the "not reproduced against a live endpoint" gap you mentioned.

[DuanZGit]

Heads-up in case it helps: there's an existing open PR touching the same encode path — #415 "fix(translation): preserve chat reasoning details" (opened 08-13, currently being reviewed; @grahamking has review comments). It rewrites prepend_openai_reasoning_blocks on the decode side (adds reasoning_details handling) and also changes encode_message_without_tool_results_to_openai to re-emit Reasoning blocks as a reasoning field on assistant history messages.

So the round-trip-loses-reasoning part is being addressed there. What #415 does not have — and what I'd still love to see land — is your steps 2 & 3: the capability gate (unknown-field-strict providers) and the push_lossy diagnostic on drop. Those are the operator-facing pieces our gateway had to build around (we cache + backfill reasoning_content ourselves in the deployment layer, silently, because there was no signal from the wire).

My fixture comment above is a reasoning_content-shaped pair; if #415's direction ends up being merged and the gate lands on top, that pair should exercise the exact case.

[ting-hong-shieh]

Update after re-reading the thread, plus one thing I'd rather raise before opening a PR.

The fixture

@DuanZGit — thanks, that's exactly the shape I was missing. I turned Turn 1 into a regression test in ting-hong-shieh/Switchyard@c7b7645 (branch fix/openai-chat-reasoning-content-request, based on 9ad6744): an assistant message carrying both reasoning_content and tool_calls, followed by the tool result. Re-encoded as request history, the assistant message keeps reasoning_content alongside tool_calls, and the tool message still follows the call it answers — so both 400 variants you quoted have assertions on that path.

Reproduction: cargo test -p switchyard-translation --test request_translation on Linux, cargo 1.96.1, no network. cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings, and cargo test --workspace are clean at that commit.

To be precise about what that is: deterministic regression coverage of the encoder, not external validation. I still have not run it against a live DeepSeek endpoint.

Overlap with #415, and a field-name question

I'd missed #415 — good catch. It does cover the round-trip half, so I don't want to open a competing PR into the same function. But reading its encode path raises something that matters for this issue specifically.

#415 encodes request-side reasoning as message["reasoning"], not reasoning_content:

// encode_openai_message_plaintext_reasoning
if !reasoning.is_empty() {
    message["reasoning"] = Value::String(reasoning);
}

For a Turn 1 message decoded from reasoning_content with no reasoning_details — the fixture above — details is empty, so the plaintext branch runs and the replayed history carries reasoning. The response encoder in the same PR still emits reasoning_content, so the two directions use different names.

The 400 in this issue names reasoning_content. Whether DeepSeek also accepts reasoning as an alias, I don't know and haven't tested. Two cases:

• If it does, fix(translation): preserve chat reasoning details #415 closes this issue on its own and my step 1 is redundant.
• If it doesn't, fix(translation): preserve chat reasoning details #415 fixes the general reasoning round trip but leaves [bug] reasoning_content stripped from requests: breaks reasoning-required upstreams (DeepSeek thinking) on multi-turn #449 open.

@DuanZGit — you offered a live-endpoint check. This one is cheaper than waiting for a branch: send your Turn 2 request with reasoning on the assistant history message instead of reasoning_content and see whether the 400 comes back. That answers it either way and is useful to #415 regardless of what happens here.

What's still outstanding either way

Steps 2 and 3 from my earlier comment are independent of the field-name question and aren't in #415:

• the capability gate, so the field isn't sent to targets that reject unknown message fields;
• a push_lossy call when reasoning is dropped, so the drop stops being silent.

On the gate, @DuanZGit's point about routing several upstreams from one config matches what I built: a TargetCapabilities flag rather than a ProviderProfile.provider table.

One design decision I'd like a maintainer to make. The other TargetCapabilities flags default to permissive and only gate on Some(false). Mine currently requires Some(true) before sending, on the assumption that unconditionally adding a message field for every OpenAI-compatible target would break the strict ones. That assumption is untested — I don't know which providers actually reject an unknown reasoning_content on a request — and it's the sole reason for the inconsistency. If you'd rather it match its neighbours, the flip is small and I'll do it.

How would you like this sequenced?

@grahamking — since you're reviewing #415:

1. Wait for fix(translation): preserve chat reasoning details #415, then rebase the gate and the diagnostic on top as a follow-up. My preference: no conflicts, and it keeps fix(translation): preserve chat reasoning details #415's review focused.
2. Open mine standalone now against main. It would conflict with fix(translation): preserve chat reasoning details #415 in encode_message_without_tool_results_to_openai, and whichever lands second gets a rebase.
3. Fold the gate and diagnostic into fix(translation): preserve chat reasoning details #415, if @pst2154 would rather carry them.

I'll hold my branch until you say. Happy to just hand the tests and the gate to #415 if that's the shorter path.

Scope limit worth stating

encode_request checks exact_preserved_request first, so when source and target are both OpenAiChat and PreservationPolicy is InMemory or Embed, the original body is replayed and no encoder runs — that combination was never affected. The loss needs cross-format translation, PreservationPolicy::Disabled, or a request built directly from the IR. The issue doesn't mention this, and it narrows who is actually hitting the bug.

Second, a user-visible side effect of the push_lossy call: callers already on LossyConversionPolicy::Reject will now get a translation error where reasoning used to be stripped silently and rejected later by the provider, if at all. Intended, but it is a behavior change for existing configs.

[DuanZGit]

Ran the live-endpoint check you asked for (production DeepSeek, deepseek-v4-flash, via our UG host — keys stayed server-side). Six requests, same shape as the fixture: user → assistant(tool_calls[+extra]) → tool("4"). Results:

#	extra on assistant history msg	result
A	(none) — switchyard status quo	400 reasoning_content in thinking mode must be passed back
B	reasoning (the name #415 encodes)	400 same message ❌
C	reasoning_content	200 ✅
D	reasoning_content: ""	200 ✅
E	reasoning_details: [...]	400 same message
F	totally_unknown_field_xyz	400 same message

So the field-name question is settled empirically for DeepSeek: reasoning does not satisfy it, reasoning_content does — even empty. And the "which providers reject unknown fields" assumption is moot on this one: DeepSeek's gate is a presence check, not strict unknown-field validation (E and F fail on the same message regardless of the bogus field present, because reasoning_content is missing). Set B also shows why the capability flag can't defensively default to "don't send": for DeepSeek the safe values are reasoning_content and reasoning_content: "" — there is no "absent" state that works, so a Some(true)-gated emission is correct, and dropping the field when the gate is off is exactly the 400 we see today.

On sequencing (my vote is option 1, and I've said so in my other comment, but restating since you asked directly): wait for #415, then carry the gate + push_lossy as a follow-up rebased on it. #415's encode name should be fixed to reasoning_content per the table above — as-is, it fixes the round trip for Anthropic-shaped targets but DeepSeek stays 400.

One caveat on the scope note: exact_preserved_request covers the two switchyard hops, but our deployment terminates the OpenAI client and re-emits (gateway) — so we hit PreservationPolicy::Disabled / direct-IR on the request, which is in your listed scope and matches what we observe. And the Reject-policy side effect you flagged is fine by us — we'd rather fail loudly at the translator than invisibly mid-conversation.

Two artifacts if useful for the regression tests:

• verified fixture: exactly the table above (Turn 2 request/response captured from the live endpoint);
• and if the maintainers want streaming coverage, I can attach an SSE delta.reasoning_content ↔ delta.content ↔ delta.tool_calls interleaved pair — that's the other pattern our gateway has to cache today, since the multipart branch (L935-937) drops it even without the request-history path.

Also: once a branch lands with the gate, happy to run the same A–D matrix against our live endpoint as CI-on-prod. Ping me on the PR when it exists.

[ting-hong-shieh]

@DuanZGit — that matrix answers it. Set B is the one I couldn't call: reasoning does not satisfy the check, so #415 as written fixes the round trip for Anthropic-shaped targets and leaves DeepSeek at 400. Thanks for running it.

Two things I found while checking my branch against your results. Both are maintainer calls rather than things I want to decide unilaterally.

1. My fix does not cover the "no reasoning on this turn" case

Your set D (reasoning_content: "" → 200) against set A (absent → 400) implies the field's presence is what matters. My encoder returns early when the reasoning text is empty, so it never emits the field in that case. Probed at ting-hong-shieh/Switchyard@c7b7645 with supports_reasoning_content: Some(true):

assistant turn with tool_calls, no reasoning at all
  -> reasoning_content = None   (0 diagnostics)

source assistant message carrying reasoning_content: ""
  -> reasoning_content = None

The second one has a second cause: prepend_openai_reasoning_blocks filters empty strings on decode, so an empty reasoning_content never becomes a ContentBlock::Reasoning and there is nothing left to re-encode.

So the branch fixes "reasoning existed upstream and was dropped on the way back" — the reported bug — but an assistant turn that genuinely carries no reasoning still produces your set A shape.

Closing that means emitting reasoning_content: "" whenever the target declares the capability and the message is an assistant message, regardless of whether any reasoning survived. That is a straightforward change, and I have not made it, because:

• it is a behavior change wider than the reported bug — it would put the field on assistant messages that never had reasoning;
• the evidence for it is DeepSeek-only. Set D says an empty string is safe there. I have no data on any other target that would declare this capability.

Happy to add it with a test if you want it; equally happy to leave it and let the operator's history carry real reasoning. Your call.

2. The capability flag is not reachable from the served path

Worth stating plainly, because it limits who the fix actually helps.

switchyard-server and libsy-llm-client reach the encoder through the helper in crates/switchyard-translation/src/helpers.rs:

pub fn encode_request(request: &LlmRequest, wire_format: WireFormat) -> Result<Value> {
    Ok(DEFAULT_TRANSLATION_ENGINE
        .encode_request(wire_format, request, &DEFAULT_TRANSLATION_POLICY)?
        .body)
}

DEFAULT_TRANSLATION_POLICY is TranslationPolicy::default(), so target_capabilities is always TargetCapabilities::default() and supports_reasoning_content is always None. The call site is crates/libsy-llm-client/src/client.rs:207. Nothing in the server's TOML maps onto TargetCapabilities — the route-level reasoning field feeds ModelCapabilities in crates/switchyard-server/src/lib.rs:96, which is a separate struct for advertising model metadata to clients and never reaches the translation policy.

Net effect: the flag can only be set by a caller that builds a TranslationPolicy and calls TranslationEngine directly. @DuanZGit, that is presumably also how your gateway reaches PreservationPolicy::Disabled, which would explain why the scope note matched what you observe. For an operator running switchyard-server off a TOML deployment, both the preservation policy and the capability flag are fixed at their defaults, so this fix would sit inert.

That bears directly on the gate-default question I asked earlier:

• keep it Some(true)-gated and the fix serves direct TranslationEngine consumers only, until someone plumbs a config surface;
• flip it to Some(false)-gated, matching every other flag in the struct, and cross-format traffic through switchyard-server starts working without new configuration — at the cost of sending the field to targets whose tolerance is still untested. Note @DuanZGit's sets E and F only establish that DeepSeek does a presence check; they say nothing about other OpenAI-compatible providers.

Plumbing TargetCapabilities through the server config is clearly a larger change than this bug fix, so I am flagging it rather than folding it in. If you want it as a follow-up I am happy to take it, but it should not ride along here.

Status

Branch fix/openai-chat-reasoning-content-request is at c7b7645, based on 9ad6744, with your fixture as a regression test. cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings, and cargo test --workspace are clean at that commit on Linux / cargo 1.96.1. Still no PR — holding for the sequencing call, and my preference remains rebasing onto #415 once it lands.

@DuanZGit — yes to the streamed SSE pair if the maintainers want streaming coverage. The multipart branch drops reasoning independently of the request-history path, so that is arguably its own issue rather than something to grow this one into.

[ting-hong-shieh]

Correction to my previous comment.

I wrote that the multipart branch drops reasoning independently of the request-history path and is "arguably its own issue." That looks wrong, and I should have checked it rather than repeating the framing.

On main (9ad6744), encode_openai_content has exactly one caller — encode_message_without_tool_results_to_openai — and the blocks it receives are already filtered:

// crates/switchyard-translation/src/codecs/openai_chat/buffered.rs:795
.filter(|block| {
    !matches!(
        block,
        ContentBlock::ToolCall(_) | ContentBlock::Reasoning { .. }
    )
})

So the ContentBlock::Reasoning { .. } => {} arm at L935 is unreachable from that path. It is a defensive catch-all, not a live drop, and encode_openai_content has no other caller in the workspace.

The streaming path also handles reasoning in both directions on main: codecs/openai_chat/stream.rs reads reasoning_content / reasoning off deltas (L108) and writes reasoning_content back out (L192).

@DuanZGit — so I don't think an SSE pair is needed to cover a second buffered drop, because the multipart arm isn't one. A streamed fixture may still be worth having for the reasoning_details work in #415, but that is that PR's call rather than a separate gap here. Apologies for passing the framing along without verifying it first.

Nothing else in my previous comment changes: the empty-reasoning gap and the capability flag being unreachable from the served path both still hold, and I have re-checked the second one since posting.

[DuanZGit]

Both checks are useful — they narrow the fix in ways that matter for operators.

1. The empty-reasoning gap

Yes, set D ("" → 200) vs set A (absent → 400) means DeepSeek is doing a presence check, not a content check. Your observation matches what we saw: the branch as written fixes "reasoning existed upstream and was dropped" but leaves "assistant turn genuinely has no reasoning" at 400 on DeepSeek.

For our workload that gap doesn't bite us — tool-calling turns carry reasoning, and non-tool-calling turns don't hit the multi-turn tool-result path. But it could be a hole for teams using DeepSeek thinking without reasoning on every turn.

Emitting reasoning_content: "" on capability-enabled assistant messages would cover it on DeepSeek, and we'd be comfortable with that (or the narrower tool_calls-only variant) — but it is a behavior change beyond the reported bug, so it feels like a maintainer call rather than something we'd push for. Either option works for us.

2. The unreachable flag

Agreed, that's the bigger finding — if switchyard-server always runs with TargetCapabilities::default() and there's no TOML surface, the gate is inert for the served path, and the fix only helps direct TranslationEngine consumers. We happen to be one of those consumers (our gateway builds a TranslationPolicy directly, which also explains why the scope note matched what we observe), so we'd benefit either way. Whether to expose it in config later is a maintainer decision; flagging it here just so it isn't lost from the record.

Sequencing

Option 1 still reads right to us: let #415 land (ideally with the field-name fix) and rebase on top, rather than open a competing PR. If maintainers prefer otherwise, no objection from us.

Streaming

Agreed — the multipart arm isn't a live drop; our SSE offer was only ever for #415's reasoning_details coverage if they want it. Their call.

Gate default

We'd lean toward matching the neighbours (permissive), purely because the flag is currently unreachable from the served path — our data only shows DeepSeek requires the field and tolerates empty; we can't speak for other providers. No strong opinion here: it's a maintainer decision and we're happy either way. If there's evidence of providers that 400 on unknown fields, permissive is obviously wrong and we'd defer to that.

Net: the branch looks ready to rebase whenever maintainers settle the sequencing. Happy to re-run the A–D matrix against our live endpoint on whatever branch ends up as the PR — just ping.