Skip to content

fix(init): prevent false GitHub Copilot auto-detection from bare .github/ directory#917

Open
alleninnz wants to merge 1 commit intoFission-AI:mainfrom
alleninnz:fix/copilot-false-detection
Open

fix(init): prevent false GitHub Copilot auto-detection from bare .github/ directory#917
alleninnz wants to merge 1 commit intoFission-AI:mainfrom
alleninnz:fix/copilot-false-detection

Conversation

@alleninnz
Copy link
Copy Markdown

@alleninnz alleninnz commented Apr 4, 2026

Summary

  • Bug: openspec init falsely auto-detects GitHub Copilot in virtually every GitHub repository because the tool detection logic checks for .github/ directory existence, which is present in nearly all repos (for workflows, issue templates, etc.)
  • Fix: Adds an optional detectionPaths field to AIToolOption that overrides the default skillsDir-based detection. For GitHub Copilot, detection now requires Copilot-specific signals (e.g., copilot-instructions.md, .github/prompts/, .github/agents/) instead of a bare .github/ directory
  • Impact: In non-interactive mode, bare .github/ no longer silently selects Copilot. In interactive first-time setup, Copilot is no longer incorrectly pre-selected. All other tools are unaffected (backward-compatible opt-in field)

Detection signals for GitHub Copilot

The following Copilot-specific paths are checked (any one triggers detection):

Path Type Signal
.github/copilot-instructions.md file Main Copilot config file
.github/instructions/ directory Path-specific Copilot instructions
.github/workflows/copilot-setup-steps.yml file Copilot coding agent environment setup
.github/prompts/ directory Copilot custom slash commands
.github/agents/ directory Copilot custom agents
.github/skills/ directory Copilot skills (including OpenSpec-generated)
.github/.mcp.json file Copilot MCP server config

Changes

File Change
src/core/config.ts Added detectionPaths?: string[] to AIToolOption; added detection paths to GitHub Copilot entry
src/core/available-tools.ts When detectionPaths present, check those instead of bare skillsDir directory
test/core/available-tools.test.ts 7 new tests covering detection logic
test/core/init.test.ts Updated 2 tests to use Copilot-specific signal
test/core/update.test.ts Updated 1 test to use Copilot-specific signal

Test plan

  • Bare .github/ directory does NOT trigger Copilot detection
  • Each Copilot-specific signal correctly triggers detection (copilot-instructions.md, prompts/, agents/, skills/, copilot-setup-steps.yml)
  • Tools without detectionPaths still use skillsDir detection (backward compat)
  • All 112 tests pass in affected test files
  • No regressions in full test suite (23 pre-existing failures unchanged)
  • Manual smoke test: openspec init in temp dir with bare .github/ produces no Copilot artifacts

Generated with Claude Code

Summary by CodeRabbit

Release Notes

  • New Features

    • Added support for customizable tool detection paths configuration.
  • Improvements

    • GitHub Copilot is now detected based on specific configuration markers (e.g., copilot-instructions.md, .github/prompts/, .github/agents/) rather than just folder presence, improving detection accuracy.

…hub/ directory

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Apr 4, 2026

📝 Walkthrough

Walkthrough

The changes add a configurable detection mechanism for AI tools. A new optional detectionPaths field was introduced to allow tools to specify alternative file/directory paths for availability detection. GitHub Copilot is now configured to check for specific markers like .github/copilot-instructions.md, .github/prompts/, and related files. The detection logic falls back to the existing skillsDir behavior when detectionPaths is absent.

Changes

Cohort / File(s) Summary
Core Detection & Configuration
src/core/available-tools.ts, src/core/config.ts
Added detectionPaths?: string[] optional field to AIToolOption interface. Updated GitHub Copilot tool config with detection path list (.github/copilot-instructions.md, .github/prompts, .github/agents, etc.). Modified getAvailableTools() to check configured detection paths first; falls back to skillsDir check if detectionPaths absent/empty.
Tool Detection Tests
test/core/available-tools.test.ts
Added comprehensive test suite for GitHub Copilot detection behavior, verifying that bare .github/ directory does not trigger detection, but presence of Copilot-specific files/directories does. Included regression test for tools without detectionPaths (e.g., claude).
Init & Update Test Setup
test/core/init.test.ts, test/core/update.test.ts
Updated test filesystem setup to create .github/copilot-instructions.md file alongside .github/ directory, aligning test environment with new GitHub Copilot detection paths configuration.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 A path through the .github trees,
Where Copilot instructions dance in the breeze,
Detection paths bloom like clover so sweet,
Each marker a sign that the tools come to greet!

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately reflects the main fix: preventing false GitHub Copilot auto-detection from a bare .github/ directory by requiring specific Copilot-related paths instead.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@alleninnz alleninnz marked this pull request as ready for review April 4, 2026 21:22
@alleninnz alleninnz requested a review from TabishB as a code owner April 4, 2026 21:22
Copilot AI review requested due to automatic review settings April 4, 2026 21:22
Copy link
Copy Markdown

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes openspec init tool auto-detection so GitHub Copilot is no longer falsely detected in repositories that merely contain a standard .github/ directory, by introducing tool-specific detection signals.

Changes:

  • Added an optional detectionPaths?: string[] on AIToolOption to support tool-specific detection signals.
  • Updated getAvailableTools() to prefer detectionPaths (file/dir existence) over skillsDir directory checks when provided.
  • Updated/added tests to ensure bare .github/ does not trigger Copilot detection, while Copilot-specific signals do.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/core/config.ts Adds detectionPaths to the tool config type and defines Copilot-specific detection signals.
src/core/available-tools.ts Implements detection using detectionPaths (when present) instead of checking only skillsDir.
test/core/available-tools.test.ts Adds coverage for the new Copilot detection behavior and preserves legacy skillsDir detection for other tools.
test/core/init.test.ts Updates init detection-related tests to use a Copilot-specific signal.
test/core/update.test.ts Updates new-tool detection test setup to use a Copilot-specific signal.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
test/core/available-tools.test.ts (1)

91-150: Add coverage for the remaining configured Copilot markers.

Great set of detection tests overall. Consider adding explicit cases for .github/instructions and .github/.mcp.json too, since both are in detectionPaths.

📌 Suggested test additions
+    it('should detect GitHub Copilot when .github/instructions directory exists', async () => {
+      await fs.mkdir(path.join(testDir, '.github', 'instructions'), { recursive: true });
+
+      const tools = getAvailableTools(testDir);
+      const toolValues = tools.map((t) => t.value);
+      expect(toolValues).toContain('github-copilot');
+    });
+
+    it('should detect GitHub Copilot when .github/.mcp.json exists', async () => {
+      await fs.mkdir(path.join(testDir, '.github'), { recursive: true });
+      await fs.writeFile(path.join(testDir, '.github', '.mcp.json'), '{}');
+
+      const tools = getAvailableTools(testDir);
+      const toolValues = tools.map((t) => t.value);
+      expect(toolValues).toContain('github-copilot');
+    });
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/core/available-tools.test.ts` around lines 91 - 150, Add two new tests
to cover the remaining GitHub Copilot detection markers: create a test that
writes an empty file at path.join(testDir, '.github', 'instructions') and
asserts getAvailableTools(testDir).map(t=>t.value) contains 'github-copilot',
and another test that writes an empty file at path.join(testDir, '.github',
'.mcp.json') and asserts the same; use the existing test pattern
(getAvailableTools, testDir, mapping to toolValues and
expect(...).toContain('github-copilot')) so detectionPaths for Copilot are fully
covered.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@test/core/available-tools.test.ts`:
- Around line 91-150: Add two new tests to cover the remaining GitHub Copilot
detection markers: create a test that writes an empty file at path.join(testDir,
'.github', 'instructions') and asserts
getAvailableTools(testDir).map(t=>t.value) contains 'github-copilot', and
another test that writes an empty file at path.join(testDir, '.github',
'.mcp.json') and asserts the same; use the existing test pattern
(getAvailableTools, testDir, mapping to toolValues and
expect(...).toContain('github-copilot')) so detectionPaths for Copilot are fully
covered.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 2150923f-0c72-4d11-bb3c-349a68429d09

📥 Commits

Reviewing files that changed from the base of the PR and between afdca0d and 2ed427e.

📒 Files selected for processing (5)
  • src/core/available-tools.ts
  • src/core/config.ts
  • test/core/available-tools.test.ts
  • test/core/init.test.ts
  • test/core/update.test.ts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants