fix(agent, llm): improve message handling to support image, document in extract_recent_actions - #111
Merged
Merged
Conversation
…xtract_recent_actions
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was changed
agent:part_texthelper renders a message's content part — its.textwhen present,[image]/[document]for media,[non-text content]otherwise. Used in all four branches ofextract_recent_actionsllm: the mapper tracks whether the merge succeeded and appends a text part when no text part exists, instead of dropping the instructionWhy?
Any tool returning an image to the calling agent through the
_imagestool-result channel killed the agent process.wippy.llm:prompt.get_messages()strips_imagesout of the function result and re-injects the pictures as{role = USER, content = collected_images}— a message whose parts are all images, with no.textanywhere. Two separate defects fire on that message.extract_recent_actionscrashes the agent process"user: " .. message.content[1].textconcatenated nil, killing the process mid-turn. The dataflow node was leftrunningand the failure surfaced asCOMPLETE_WORKFLOW must be the first command in a command batch— a downstream symptom, not the cause. Nothing reached the logger because the process was gone. Only agents with amemory_contractwere affected:extract_recent_actionsis reached only viaperform_memory_recall, which returns early without one. TheFUNCTION_RESULTbranch already guarded this withor "nil";USER,ASSISTANTandDEVELOPERdid notThe Claude mapper silently drops the developer instruction
agent:stepappends the memory recall as aDEVELOPERmessage last — directly after that image-only message.should_create_new_messageonly tests fortool_result, so an image-only message takes the merge path; the merge loop scans for a text part, finds none, and falls through. The recalled memory was discarded with no error, on every turn following an attached image