fix(core): convert every media content block in _formatForTracing - #11613
Lubaoshuai wants to merge 3 commits into
Conversation
The clone guard in _formatForTracing also gated the per-block conversion: after the first media block was rewritten, messageToTrace !== message, so every subsequent URL/base64 block in the same message was skipped and stayed inline in the traced payload. Clone the message once when any media block is present, then convert all matching blocks. Messages without media blocks keep their original reference. Fixes langchain-ai#11291
🦋 Changeset detectedLatest commit: 2911fc1 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Thanks for the clear reproduction. This is the same #11291 fix already covered by #11292, which has been open since August 3 and changes the same function/test path. #11292 also includes the required changeset, avoids an |
|
Update: with this becoming the fifth related implementation for #11291, I have closed #11292 to reduce the review queue rather than keep asking for consolidation. Its changeset, typed callback regression, and full local core validation remain available if maintainers want that coverage carried into the surviving PR. |
|
Thanks Zexu Jin (@jackjin1997) — makes sense. Since #11292 is closed, I've added the missing changeset here so this one is complete against the bot requirements. If maintainers want the typed callback regression from your branch carried in, happy to review a PR to this branch or cherry-pick it myself — whatever keeps the review queue shortest. |
|
Done — the capture handler now types |
Description
_formatForTracingin@langchain/coreonly converted the first URL/base64 media block of each message: the clone guard (if (messageToTrace === message)) also gated the per-block conversion, so after the first block was rewritten the guard became false and every later media block stayed inline in the traced payload — only the first attachment got offloaded to a data URI, and remaining base64 stayed in traces (e.g. when a message carries two PDF attachments, as reported in #11291).This change clones the message once when any media block is present and converts every matching block. Messages without media blocks keep their original reference, so behavior is unchanged for the common case. As before, this affects the tracing copy only — providers still receive the original message.
Fixes #11291
Test
libs/langchain-core/src/language_models/tests/chat_models.test.tsthat traces aHumanMessagewith two base64 file blocks and asserts both are rewritten toimage_urldata URIs and that the original message is not mutated. It fails onmain(second block left unconverted) and passes with this change.vitest run src/language_models/tests/chat_models.test.tsinlibs/langchain-core: 25 passed, no type errors.