Skip to content

Show tool output previews in Copilot CLI conversation rendering #40116

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pelikhan merged 3 commits into from
Jun 19, 2026
+24 −3
Merged

Show tool output previews in Copilot CLI conversation rendering #40116

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
533f455
Show tool output previews in Copilot debug log rendering
Copilot Jun 18, 2026
b30a1c8
Address review feedback: named constant and JSON.parse for tool_call_id
Copilot Jun 18, 2026
30a0ebd
Render Copilot CLI conversation from events.jsonl, not raw debug logs
Copilot Jun 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions actions/setup/js/copilot_sdk_session.cjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
* Event mapping:
* SDK "user.message" → JSONL "user.message"
* SDK "tool.execution_start" → JSONL "tool.execution_start" (toolName, mcpServerName)
* SDK "tool.execution_complete" → JSONL "tool.execution_complete" (toolName, mcpServerName, success)
* SDK "tool.execution_complete" → JSONL "tool.execution_complete" (toolName, mcpServerName, success, result)
* SDK "assistant.message" → JSONL "assistant.message" (content)
*
* The JSONL file is written to:
Expand Down Expand Up @@ -265,9 +265,12 @@ async function runWithCopilotSDK({ sdkUri, prompt, logger, attempt = 0, model, c
const mcpServerName = pending?.mcpServerName ?? "";
if (toolCallId) pendingToolCalls.delete(toolCallId);
const success = event.data?.success ?? !event.data?.error;
// Include result.content (concise LLM-facing output) so that the log
// parser can render tool output previews from events.jsonl directly.
const result = event.data?.result ?? undefined;

@github-actions github-actions Bot Jun 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/diagnose] event.data?.result ?? undefined is a no-op: optional chaining already returns undefined when the property is absent. Just event.data?.result is identical and clearer.

const result = event.data?.result;

// max-tool-denials intentionally tracks permission denials only.
// Tool execution failures are still logged, but do not increment the guardrail counter.
Comment on lines +268 to 272
writeEvent("tool.execution_complete", { toolName, mcpServerName, success }, event.timestamp);
writeEvent("tool.execution_complete", { toolName, mcpServerName, success, result }, event.timestamp);

@github-actions github-actions Bot Jun 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Full ToolExecutionCompleteResult object written to JSONL, but only result.content is ever consumed: detailedContent and uiResource are serialized unnecessarily, bloating events.jsonl for tool-heavy runs.

💡 Suggested fix

Write only the fields the log parser actually reads:

// current
const result = event.data?.result ?? undefined;
writeEvent("tool.execution_complete", { toolName, mcpServerName, success, result }, event.timestamp);

// suggested
const resultContent = event.data?.result?.content;
const result = resultContent !== undefined ? { content: resultContent } : undefined;
writeEvent("tool.execution_complete", { toolName, mcpServerName, success, result }, event.timestamp);

The SDK's ToolExecutionCompleteResult has four fields: content (concise, truncated), contents (structured blocks), detailedContent (the full tool output including complete diffs — explicitly documented as potentially very large), and uiResource. Only result.content is ever read in log_parser_shared.cjs. Serializing the whole object via JSON.stringify means every tool call that edits a file will write an unbounded detailedContent string to disk, and the parser has to deserialize all of it for nothing.

break;
}

Expand Down
6 changes: 5 additions & 1 deletion actions/setup/js/log_parser_shared.cjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -928,8 +928,12 @@ function convertCopilotEventsToLegacyLogEntries(logEntries) {
output = data.output;
} else if (typeof data.result === "string") {
output = data.result;
} else if (data.result && typeof data.result.content === "string") {

@github-actions github-actions Bot Jun 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/diagnose] This branch only handles string content, but the Copilot SDK and MCP protocol can deliver result.content as an array of content blocks ([{type:"text",text:"..."}]). When that shape appears the branch silently falls through to "success" — the regression this PR aims to fix.

💡 Suggested guard

Add an array case below the string check:

} else if (data.result && Array.isArray(data.result.content)) {
  // MCP-style content blocks: [{type:"text",text:"..."}]
  output = data.result.content.map(c => (typeof c === "string" ? c : c.text || "")).join("\n");
}

This mirrors the pattern in log_parser_format.cjs line 250.

// Native Copilot CLI events.jsonl format: result.content is the concise
// tool result text sent to the LLM (may be truncated for token efficiency).
output = data.result.content;
} else if (data.error) {
Comment on lines +931 to 935
output = String(data.error);
output = typeof data.error === "object" && typeof data.error.message === "string" ? data.error.message : String(data.error);

@github-actions github-actions Bot Jun 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error-extraction fix (String(data.error) → .message) has no test coverage: this is a real behavior change — before, an Error object would render as [object Object]; after, it renders as the human-readable message — but the test suite only adds a happy-path case.

💡 Suggested test

Add a case to the existing describe block covering failed tool execution with an Error object:

it("renders error.message from a failed tool call in events.jsonl", () => {
  const eventsLog = [
    '{"type":"user.message","timestamp":"2026-06-05T00:44:01.367Z","data":{}}',
    '{"type":"tool.execution_start","timestamp":"2026-06-05T00:44:04.520Z","data":{"toolName":"bash","mcpServerName":""}}',
    '{"type":"tool.execution_complete","timestamp":"2026-06-05T00:44:04.700Z","data":{"toolName":"bash","mcpServerName":"","success":false,"error":{"message":"Permission denied"}}}',
    '{"type":"assistant.message","timestamp":"2026-06-05T00:44:59.769Z","data":{"content":"Failed"}}',
  ].join("\n");

  const result = parseCopilotLog(eventsLog);

  expect(result.markdown).toContain("Permission denied");
  // Must not contain the old serialization artifact
  expect(result.markdown).not.toContain("[object Object]");
});

Without this, a future refactor could revert the fix silently.

@github-actions github-actions Bot Jun 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] The error extraction fix (data.error.message instead of String(data.error)) has no test coverage. Without a test, the "[object Object]" regression could silently return.

💡 Suggested test

Add alongside the existing result.content test:

it("renders error message from data.error.message in tool.execution_complete", () => {
  const eventsLog = [
    '{"type":"user.message","timestamp":"2026-06-05T00:44:01.367Z","data":{}}',
    '{"type":"tool.execution_start","timestamp":"2026-06-05T00:44:04.520Z","data":{"toolName":"bash","mcpServerName":""}}',
    '{"type":"tool.execution_complete","timestamp":"2026-06-05T00:44:04.700Z","data":{"toolName":"bash","mcpServerName":"","success":false,"error":{"message":"Permission denied"}}}',
    '{"type":"assistant.message","timestamp":"2026-06-05T00:44:59.769Z","data":{"content":"Done"}}',
  ].join("\n");
  const result = parseCopilotLog(eventsLog);
  expect(result.markdown).toContain("Permission denied");
  expect(result.markdown).not.toContain("[object Object]");
});

} else if (success) {
output = "success";
} else {
Expand Down
14 changes: 14 additions & 0 deletions actions/setup/js/parse_copilot_log.test.cjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,20 @@ describe("parse_copilot_log.cjs", () => {
expect(resultData?.numTurns).toBe(1);
});

it("renders tool output preview from result.content in Copilot CLI events.jsonl", () => {
const eventsLog = [
'{"type":"user.message","timestamp":"2026-06-05T00:44:01.367Z","data":{}}',
'{"type":"tool.execution_start","timestamp":"2026-06-05T00:44:04.520Z","data":{"toolName":"bash","mcpServerName":""}}',
'{"type":"tool.execution_complete","timestamp":"2026-06-05T00:44:04.700Z","data":{"toolName":"bash","mcpServerName":"","success":true,"result":{"content":"file1.txt\\nfile2.txt\\nfile3.txt"}}}',
'{"type":"assistant.message","timestamp":"2026-06-05T00:44:59.769Z","data":{"content":"Done"}}',
].join("\n");

const result = parseCopilotLog(eventsLog);

expect(result.markdown).toContain("bash");
expect(result.markdown).toContain("file1.txt");

@github-actions github-actions Bot Jun 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] The two assertions cover the happy path but leave important behaviour untested: (1) no check that the old "success" placeholder is absent, and (2) only the first line is verified — file2.txt / file3.txt could silently be dropped.

💡 Strengthen the test 
expect(result.markdown).toContain("file1.txt");
expect(result.markdown).toContain("file2.txt");  // verify multiline output reaches markdown
expect(result.markdown).not.toContain("success"); // guard against fallthrough to placeholder

A spec-style name would also help: "renders multiline tool output from result.content without a 'success' placeholder".

});
Comment on lines +127 to +139

it("should handle tool calls with details in HTML format", () => {
const logWithHtmlDetails = JSON.stringify([
{ type: "system", subtype: "init", session_id: "html-test", tools: ["Bash"], model: "gpt-5" },
Expand Down
Loading