[safe-output-health] Safe Output Health Report - 2026-05-03

[github-actions[bot]]

Executive Summary

This audit covers safe output job health for the last 24 hours (2026-05-02 to 2026-05-03). Three safe output jobs failed out of 23 executed, all related to the same PR branch (copilot/migrate-agentic-workflows-to-cli, PR #29860) which was merged during the same window. The underlying infrastructure (MCP server startup, tool registration, GitHub API integration) was 100% healthy across all runs. Failures were limited to specific GitHub API operation edge cases.

• Period: Last 24 hours
• Runs Analyzed: 48 total workflow runs
• Safe Output Jobs Executed: 23
• Safe Output Jobs Failed: 3 (13%)
• Safe Output Jobs Succeeded: 20 (87%)
• Total Operations Processed: ~52
• Operations Failed: 3 (~6%)
• Error Clusters Identified: 2

---

Safe Output Job Statistics

Job Type	Executions	Failures	Success Rate
create_discussion	5	0	100%
create_issue	7	0	100%
create_pull_request	3	0	100%
add_comment	3	0	100%
submit_pull_request_review	2	0	100%
noop	6	0	100%
push_to_pull_request_branch	2	1	50%
dispatch_workflow	1	1	0%
resolve_pull_request_review_thread	1	1	0%
create_pull_request_review_comment	2	0	100%
add_labels	2	0	100%
update_pull_request	2	0	100%
Other operations	~15	0	100%

---

Error Clusters

Cluster 1: PR Branch Merged/Deleted Before Safe Output Execution (2 occurrences)

• Count: 2 occurrences
• Affected Jobs: push_to_pull_request_branch, dispatch_workflow
• Affected Workflows: Changeset Generator (Codex), Smoke Copilot

Sample Errors

push_to_pull_request_branch: Branch copilot/migrate-agentic-workflows-to-cli no longer exists on origin (it may have been deleted), can't push to it.

dispatch_workflow: Failed to dispatch workflow "haiku-printer": No ref found for: refs/heads/copilot/migrate-agentic-workflows-to-cli

• Root Cause: PR feat: migrate playwright workflows to CLI mode, deprecate MCP mode #29860 (copilot/migrate-agentic-workflows-to-cli) was merged to main before the safe output jobs for these runs completed. Both operations attempt to act on the PR branch after it no longer exists.
• Impact: Medium - cascading cancellations (Changeset Generator's update_pull_request was cancelled), but no data loss or corruption.

Cluster 2: Missing Permission for resolve_pull_request_review_thread (1 occurrence)

• Count: 1 occurrence
• Affected Workflows: Smoke Claude

Sample Error

resolve_pull_request_review_thread: Request failed due to following response errors:
 - Resource not accessible by integration

• Root Cause: The GitHub Actions token lacks permission to resolve PR review threads via the GraphQL API (pull-requests: write or equivalent).
• Impact: Medium - 1/13 operations failed; remaining 12 operations (review comments, push, issue creation, etc.) succeeded.

---

Root Cause Analysis

Branch Race Condition

Both push_to_pull_request_branch and dispatch_workflow failures trace to the same event: PR #29860 was merged during the smoke test execution window. The system correctly detects these failures, but does not distinguish between "branch was properly merged" and "unexpected deletion," causing the safe_outputs job to report failure status.

Permission Configuration

The resolve_pull_request_review_thread operation uses a GraphQL mutation requiring elevated permissions. The Smoke Claude workflow token has insufficient scope for this specific operation.

---

Recommendations

Bug Fixes Required

1. Add graceful handling for merged PR branch (push_to_pull_request_branch & dispatch_workflow)

   • Priority: Medium
   • Problem: Hard failure when PR branch is merged before safe output runs
   • Fix: Pre-flight branch existence check; if confirmed merged, log informational skip instead of failing
   • Estimated Effort: Small

2. Fix permissions for resolve_pull_request_review_thread in smoke test workflows

   • Priority: Medium
   • Problem: Token lacks permission to resolve review threads
   • Fix: Ensure pull-requests: write in smoke-claude workflow permissions block
   • Estimated Effort: Small

Process Improvements

1. Distinguish expected vs unexpected branch deletions — Add detection logic to reduce false-positive failure rates for PRs that merge quickly
2. Pre-validate token permissions at safe output job startup — Surface permission misconfigurations before operations are attempted

---

Work Item Plans

Work Item 1: Handle Merged PR Branch Gracefully

• Type: Bug Fix | Priority: Medium
• Description: push_to_pull_request_branch and dispatch_workflow fail hard when PR branch is merged before job runs
• Acceptance Criteria:
  • Merged branch detected and skipped gracefully with informational log
  • No cascading cancellation when PR is confirmed merged

Work Item 2: Fix resolve_pull_request_review_thread Permissions

• Type: Bug Fix | Priority: Medium
• Description: Smoke Claude safe_outputs job fails with "Resource not accessible by integration"
• Acceptance Criteria:
  • Operation succeeds in smoke test runs
  • Workflow permission block updated and documented

---

Metrics

• Safe Output Job Success Rate: 87% (20/23)
• Operation Success Rate: ~94% (49/52)
• MCP Server Startup Failures: 0/23 (100% healthy)
• Most Reliable: create_discussion, create_issue, create_pull_request, add_comment, noop (all 100%)
• Most Problematic: dispatch_workflow (0%), resolve_pull_request_review_thread (0%)

Historical Context

First audit run — no historical baseline available.

References:

• Smoke Claude §25268953602 — resolve_pull_request_review_thread permission failure
• Smoke Copilot §25268953596 — dispatch_workflow branch not found
• Changeset Generator §25268953603 — push_to_pull_request_branch branch deleted

Generated by Safe Output Health Monitor · ● 1.1M · ◷

• expires on May 4, 2026, 5:32 AM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-05-04T05:32:36.570Z.

Closed by Workflow