Skip to content

Add /tactics comment-triggered workflow for servicing PRs#53596

Draft
Copilot wants to merge 4 commits intomainfrom
copilot/create-workflow-for-tactics-command
Draft

Add /tactics comment-triggered workflow for servicing PRs#53596
Copilot wants to merge 4 commits intomainfrom
copilot/create-workflow-for-tactics-command

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 25, 2026

The goal of this PR is to add a workflow where users with write permission to the repo can comment /tactics on the PR. This will trigger the workflow to grab an AI enabled PAT, review the PR as well as the originating issue (if there is one), update the description to add the .NET tactics template based on the information gathered, and mark the issue with the servicing-consider label.

Some examples:
marcpopMSFT#5
marcpopMSFT#4
marcpopMSFT#2

A zizmor audit has been run on both yml files with no issues identified.

  • Add models: read permission to fix 401 from GitHub Models API
  • Add pull-requests: write to update job to fix 403 when posting comments on PRs (/issues/{pr_number}/comments returns "Resource not accessible by integration" without PR write permission)

Co-authored-by: marcpopMSFT <12663534+marcpopMSFT@users.noreply.github.com>
Agent-Logs-Url: https://github.com/dotnet/sdk/sessions/e9426e50-d112-4958-82fc-c010e834b310
Copy link
Copy Markdown
Member

@lbussell lbussell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@marcpopMSFT did you try this on your fork? what was the output?

@marcpopMSFT
Copy link
Copy Markdown
Member

@marcpopMSFT did you try this on your fork? what was the output?

Not great: https://github.com/marcpopMSFT/sdk/actions/runs/23564038929
Unhandled error: Error: AI API call failed (401): {"error":{"code":"unauthorized","message":"The models permission is required to access this endpoint","details":"The models permission is required to access this endpoint"}}

Co-authored-by: marcpopMSFT <12663534+marcpopMSFT@users.noreply.github.com>
Agent-Logs-Url: https://github.com/dotnet/sdk/sessions/769c23fd-4c04-415c-a470-007b89cbe31b
Co-authored-by: marcpopMSFT <12663534+marcpopMSFT@users.noreply.github.com>
Agent-Logs-Url: https://github.com/dotnet/sdk/sessions/1d3c9fb1-4478-456b-97fd-cf11947df9db
@akoeplinger
Copy link
Copy Markdown
Member

@marcpopMSFT what is this trying to do? was this created with github agentic workflows, it looks different to the ones I'm familiar with?

Cherry-picked changes from marcpopMSFT/sdk main branch:
- Switch from GitHub Models to Copilot PAT pool with select-copilot-pat action
- Upgrade AI model to gpt-4.1 and improve tactics prompt instructions
- Add required Copilot-Integration-Id header to API call
- Fix prompt variable reference (split system/user messages)
- Update PR description with tactics when no linked issue found
- Add pull-requests: write permission for PR body updates

Co-authored-by: Marc Paine <marcpop@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@marcpopMSFT marcpopMSFT force-pushed the copilot/create-workflow-for-tactics-command branch from bf61f1c to 4e5aa57 Compare March 27, 2026 18:29
@marcpopMSFT
Copy link
Copy Markdown
Member

marcpopMSFT commented Mar 27, 2026

@marcpopMSFT what is this trying to do? was this created with github agentic workflows, it looks different to the ones I'm familiar with?

@akoeplinger that was an older version. I went ahead and tried copying how @jeffhandley had done PAT token-based calls in the runtime repo. The latest version is the one using that method of calling AI. I updated the description to cover the goal here and included a few examples that I've tested over on my fork. It seems to be working pretty well over there.

Open questions:

  • Is this the recommended way to do this?
  • How do we ensure there's always a valid PAT?
  • Is the tactics summary accurate and sufficient to reduce tactics toil?
  • Is there any security risk to this (ie ensure no one without write permissions can trigger this)?
  • Should this be moved centrally so others can use it?

env:
REPO_NAME: ${{ github.event.repository.name }}
RUN_URL: https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}
AI_MODEL: gpt-4.1
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This model is ~1 year old, why?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants