Skip to content

fix(core): convert every media content block in _formatForTracing - #11613

Open
Lubaoshuai wants to merge 3 commits into
langchain-ai:mainfrom
Lubaoshuai:fix/tracing-convert-all-media-blocks
Open

Lubaoshuai wants to merge 3 commits into
langchain-ai:mainfrom
Lubaoshuai:fix/tracing-convert-all-media-blocks

Conversation

@Lubaoshuai

Copy link
Copy Markdown

Description

_formatForTracing in @langchain/core only converted the first URL/base64 media block of each message: the clone guard (if (messageToTrace === message)) also gated the per-block conversion, so after the first block was rewritten the guard became false and every later media block stayed inline in the traced payload — only the first attachment got offloaded to a data URI, and remaining base64 stayed in traces (e.g. when a message carries two PDF attachments, as reported in #11291).

This change clones the message once when any media block is present and converts every matching block. Messages without media blocks keep their original reference, so behavior is unchanged for the common case. As before, this affects the tracing copy only — providers still receive the original message.

Fixes #11291

Test

  • Added a regression test in libs/langchain-core/src/language_models/tests/chat_models.test.ts that traces a HumanMessage with two base64 file blocks and asserts both are rewritten to image_url data URIs and that the original message is not mutated. It fails on main (second block left unconverted) and passes with this change.
  • vitest run src/language_models/tests/chat_models.test.ts in libs/langchain-core: 25 passed, no type errors.

The clone guard in _formatForTracing also gated the per-block conversion:
after the first media block was rewritten, messageToTrace !== message, so
every subsequent URL/base64 block in the same message was skipped and stayed
inline in the traced payload.

Clone the message once when any media block is present, then convert all
matching blocks. Messages without media blocks keep their original reference.

Fixes langchain-ai#11291
@changeset-bot

changeset-bot Bot commented Sep 11, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 2911fc1

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@langchain/core Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@jackjin1997

Copy link
Copy Markdown
Contributor

Thanks for the clear reproduction. This is the same #11291 fix already covered by #11292, which has been open since August 3 and changes the same function/test path. #11292 also includes the required changeset, avoids an any in the callback test, passed the full local core suite, and the author of the other duplicate #11450 has already agreed to consolidate there. Could we keep review on #11292 and close this duplicate unless there is a genuinely distinct case to carry over?

@jackjin1997

Copy link
Copy Markdown
Contributor

Update: with this becoming the fifth related implementation for #11291, I have closed #11292 to reduce the review queue rather than keep asking for consolidation. Its changeset, typed callback regression, and full local core validation remain available if maintainers want that coverage carried into the surviving PR.

@Lubaoshuai

Copy link
Copy Markdown
Author

Thanks Zexu Jin (@jackjin1997) — makes sense. Since #11292 is closed, I've added the missing changeset here so this one is complete against the bot requirements. If maintainers want the typed callback regression from your branch carried in, happy to review a PR to this branch or cherry-pick it myself — whatever keeps the review queue shortest.

@Lubaoshuai

Copy link
Copy Markdown
Author

Done — the capture handler now types _llm as Serialized (same shape as the typed regression in #11292), so the test carries no any anymore. Kept the extra assertion that tracing doesn't mutate the original message, and the changeset is already in place. Happy for this branch to be the surviving PR for #11291 — it's ready for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

_formatForTracing only converts the first base64 content block per message

2 participants