Skip to content

fix(agent, llm): improve message handling to support image, document in extract_recent_actions - #111

Merged
msmakouz merged 2 commits into
masterfrom
fix/image-in-extract-recent-actions
Aug 25, 2026
Merged

fix(agent, llm): improve message handling to support image, document in extract_recent_actions#111
msmakouz merged 2 commits into
masterfrom
fix/image-in-extract-recent-actions

Conversation

@msmakouz

Copy link
Copy Markdown
Contributor

What was changed

  • agent: part_text helper renders a message's content part — its .text when present, [image] / [document] for media, [non-text content] otherwise. Used in all four branches of extract_recent_actions
  • llm: the mapper tracks whether the merge succeeded and appends a text part when no text part exists, instead of dropping the instruction

Why?

Any tool returning an image to the calling agent through the _images tool-result channel killed the agent process. wippy.llm:prompt.get_messages() strips _images out of the function result and re-injects the pictures as {role = USER, content = collected_images} — a message whose parts are all images, with no .text anywhere. Two separate defects fire on that message.

  1. extract_recent_actions crashes the agent process "user: " .. message.content[1].text concatenated nil, killing the process mid-turn. The dataflow node was left running and the failure surfaced as COMPLETE_WORKFLOW must be the first command in a command batch — a downstream symptom, not the cause. Nothing reached the logger because the process was gone. Only agents with a memory_contract were affected: extract_recent_actions is reached only via perform_memory_recall, which returns early without one. The FUNCTION_RESULT branch already guarded this with or "nil"; USER, ASSISTANT and DEVELOPER did not

  2. The Claude mapper silently drops the developer instruction agent:step appends the memory recall as a DEVELOPER message last — directly after that image-only message. should_create_new_message only tests for tool_result, so an image-only message takes the merge path; the merge loop scans for a text part, finds none, and falls through. The recalled memory was discarded with no error, on every turn following an attached image

@msmakouz
msmakouz requested a review from wolfy-j August 25, 2026 12:41
@msmakouz msmakouz self-assigned this Aug 25, 2026
@msmakouz msmakouz changed the title Fix/image in extract recent actions fix(agent, llm): improve message handling to support image, document in extract_recent_actions Aug 25, 2026
@msmakouz
msmakouz merged commit 953973b into master Aug 25, 2026
21 checks passed
@msmakouz
msmakouz deleted the fix/image-in-extract-recent-actions branch August 25, 2026 13:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant