Skip to content

feat: Add Issue Link Checking to PR Helper Bot#1201

Merged
rwalworth merged 4 commits intomainfrom
add-issue-link-checking
Mar 5, 2026
Merged

feat: Add Issue Link Checking to PR Helper Bot#1201
rwalworth merged 4 commits intomainfrom
add-issue-link-checking

Conversation

@rwalworth
Copy link
Contributor

Summary

This PR adds issue link checking as a fourth validation check to the PR Helper Bot, ensuring every PR references at least one issue and that the PR author is assigned to all linked issues. Alongside this new feature, the bot architecture is refactored from a fragmented multi-step/multi-comment design into a unified single-comment dashboard with modular, independently testable helper modules.

Key Changes:

  • Add issue link detection via regex (closing/related keywords) with GraphQL closingIssuesReferences fallback
  • Verify PR author is assigned to all linked issues
  • Consolidate all checks (DCO, GPG, merge conflict, issue link) into a single, auto-updating dashboard comment
  • Refactor bot scripts into a clean modular architecture (checks.js, comments.js, api.js)
  • Split the on-commit + on-pr two-step workflow into dedicated on-pr-open and on-pr-update workflows
  • Add comprehensive test suites for every module

Motivation

Previously the bot ran DCO, GPG, and merge conflict checks independently, each posting its own comment on failure. There was no mechanism to verify that a PR was linked to a tracked issue, making it easy for contributions to bypass the issue-first workflow. The fragmented architecture also made testing and maintenance difficult.

This change:

  1. Enforces issue-first development -- PRs without a linked issue or without the author assigned are flagged
  2. Reduces comment noise -- All check results appear in a single dashboard comment that updates in place
  3. Improves maintainability -- Modular helpers with pure functions are easy to test and extend

Changes

1. Issue Link Check (New) ⭐

The core addition. checkIssueLink in helpers/checks.js validates that a PR references at least one issue and that the author is assigned to each linked issue.

Detection strategy:

  1. Regex scan of the PR body for closing keywords (closes, fixes, resolves) and related to references
  2. Falls back to GitHub's GraphQL closingIssuesReferences if the regex finds nothing
  3. Fetches each linked issue via REST API and verifies the PR author appears in the issue's assignees

Possible outcomes:

Result Reason Dashboard Display
Pass Issue linked and author assigned ✅ Linked to #N (assigned to you)
Fail No issue referenced ❌ Not linked to any issue
Fail Author not assigned to linked issue ❌ Not assigned to #N
Error API failure ⚠️ Internal error, review manually

Files:

  • helpers/checks.js -- checkIssueLink() function
  • helpers/api.js -- fetchIssue(), fetchClosingIssueNumbers() API wrappers

2. Unified Dashboard Comment

Replaced per-check individual comments with a single bot comment that renders all four checks together. The comment is identified by an HTML marker (<!-- bot:pr-helper -->) and is created on first run, then updated in place on subsequent events.

Comment structure:

  • Greeting with PR author mention
  • ### PR Checks section with DCO, GPG, Merge Conflict, and Issue Link results separated by horizontal rules
  • Footer indicating overall status (all passed vs. action required)

Files:

  • helpers/comments.js -- buildBotComment(), buildChecksSection(), and per-check section builders
  • helpers/api.js -- postOrUpdateComment() with pagination to find existing bot comment

3. Modular Architecture Refactor

Broke the monolithic bot-on-commit.js and bot-on-pr.js into focused modules:

Old File New Location Responsibility
bot-on-commit.js (deleted) helpers/checks.js Pure check functions (DCO, GPG, merge conflict, issue link)
bot-on-commit.js (deleted) helpers/api.js fetchPRCommits, runAllChecksAndComment
bot-on-pr.js (deleted) bot-on-pr-open.js PR open/reopen: all checks + auto-assign + force label
(new) bot-on-pr-update.js Synchronize/edit: all checks + conditional label swap
(new) helpers/comments.js Comment building (dashboard, per-check sections)

Check functions in checks.js are pure (no API calls, no side effects), making them trivially testable. API interactions are isolated in api.js.

4. Workflow Changes

  • on-pr.yaml renamed to Bot - On PR Open, simplified from a two-step job (on-commit then on-pr with env-var passing) to a single step calling bot-on-pr-open.js
  • on-pr-update.yaml (new) replaces on-commit.yaml, triggers on synchronize and edited events, calls bot-on-pr-update.js
  • on-commit.yaml deleted
  • Both workflows share a concurrency group keyed on PR number (pr-bot-${{ number }}) to prevent race conditions
  • Both skip draft PRs and bot-authored PRs

5. Test Infrastructure Overhaul

Replaced the old test-on-commit-bot.js and test-on-pr-bot.js with a comprehensive suite of focused test files:

Test File What It Covers Approach
tests/test-checks.js DCO, GPG, merge conflict, issue link checks Unit tests on pure functions + mock API
tests/test-comments.js Dashboard comment rendering Unit tests with snapshot verification
tests/test-api.js buildBotContext, postOrUpdateComment, labels, etc. Unit tests with mock GitHub
tests/test-on-pr-open-bot.js Full bot-on-pr-open.js integration End-to-end with mock GitHub
tests/test-on-pr-update-bot.js Full bot-on-pr-update.js integration End-to-end with mock GitHub
tests/test-assign-bot.js Comment-based assignment bot Moved from scripts/ to tests/

Shared utilities (tests/test-utils.js) provide commit helpers, a createMockGithub factory with issue/GraphQL support, comment snapshot verification, and a runTestSuite runner with summary printing.


Testing

node .github/scripts/tests/test-checks.js          # ✅ Unit + integration tests for all 4 checks
node .github/scripts/tests/test-comments.js         # ✅ Dashboard comment rendering tests
node .github/scripts/tests/test-api.js              # ✅ API helper unit tests
node .github/scripts/tests/test-on-pr-open-bot.js   # ✅ End-to-end on-PR-open scenarios
node .github/scripts/tests/test-on-pr-update-bot.js # ✅ End-to-end on-PR-update scenarios
node .github/scripts/tests/test-assign-bot.js       # ✅ Comment assignment bot tests

Test Plan:

  • All unit tests pass for pure check functions
  • Comment snapshot tests verify exact markdown output for all check states (pass, fail, error)
  • Integration tests cover issue link scenarios: no issue, issue linked but not assigned, issue linked and assigned, GraphQL fallback
  • Integration tests verify dashboard comment is created on first run and updated on subsequent runs
  • CI workflow (zxc-test-bot-scripts.yaml) updated to run all new test files

Files Changed Summary

Added (8 files)

  • .github/scripts/bot-on-pr-open.js -- PR open handler
  • .github/scripts/bot-on-pr-update.js -- PR update handler
  • .github/scripts/helpers/checks.js -- Pure check functions
  • .github/scripts/helpers/comments.js -- Dashboard comment builder
  • .github/scripts/tests/test-checks.js -- Check unit tests
  • .github/scripts/tests/test-comments.js -- Comment unit tests
  • .github/scripts/tests/test-api.js -- API helper tests
  • .github/scripts/tests/test-on-pr-open-bot.js -- On-PR-open integration tests
  • .github/scripts/tests/test-on-pr-update-bot.js -- On-PR-update integration tests
  • .github/workflows/on-pr-update.yaml -- New PR update workflow

Modified (4 files)

  • .github/scripts/helpers/api.js -- Added fetchIssue, fetchClosingIssueNumbers, postOrUpdateComment, runAllChecksAndComment, swapStatusLabel
  • .github/scripts/helpers/index.js -- Re-exports new modules
  • .github/scripts/tests/test-utils.js -- Added createMockGithub factory with issue/GraphQL mocking
  • .github/workflows/on-pr.yaml -- Renamed and simplified to single-step
  • .github/workflows/zxc-test-bot-scripts.yaml -- Updated to run all new test files

Removed (4 files)

  • .github/scripts/bot-on-commit.js -- Replaced by modular checks + bot-on-pr-update.js
  • .github/scripts/bot-on-pr.js -- Replaced by bot-on-pr-open.js
  • .github/scripts/test-on-commit-bot.js -- Replaced by tests/test-checks.js et al.
  • .github/scripts/test-on-pr-bot.js -- Replaced by tests/test-on-pr-open-bot.js et al.
  • .github/workflows/on-commit.yaml -- Replaced by on-pr-update.yaml
Category Count
New feature (issue link check) 1
New bot scripts 2
New helper modules 2
New test files 5
Modified files 5
Deleted files 5

Breaking Changes

None. All changes are internal to the CI bot infrastructure. No public SDK APIs are affected. The bot comment format changes from multiple individual comments to a single dashboard comment, but this is a UX improvement with no downstream impact.

Signed-off-by: Rob Walworth <robert.walworth@swirldslabs.com>
@rwalworth rwalworth self-assigned this Mar 3, 2026
@rwalworth rwalworth requested review from a team as code owners March 3, 2026 21:38
@rwalworth rwalworth added the status: needs review The pull request is ready for maintainer review label Mar 3, 2026
Signed-off-by: Rob Walworth <robert.walworth@swirldslabs.com>
Copy link
Contributor

@gsstoykov gsstoykov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Just one thing.

Signed-off-by: Rob Walworth <robert.walworth@swirldslabs.com>
gsstoykov
gsstoykov previously approved these changes Mar 4, 2026
@rwalworth rwalworth merged commit ba5d7fe into main Mar 5, 2026
14 checks passed
@rwalworth rwalworth deleted the add-issue-link-checking branch March 5, 2026 15:28
@rwalworth rwalworth added priority: medium Normal priority; to be addressed in the standard development cycle scope: ci Related to GitHub Actions or CI/CD skill: intermediate Requires familiarity with the codebase structure and SDK concepts and removed status: needs review The pull request is ready for maintainer review labels Mar 5, 2026
@rwalworth rwalworth added this to the v0.52.0 milestone Mar 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority: medium Normal priority; to be addressed in the standard development cycle scope: ci Related to GitHub Actions or CI/CD skill: intermediate Requires familiarity with the codebase structure and SDK concepts

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants