Migrate threat detection to external threat-detect binary behind feature flag#40166
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
…ure flag gh-aw-detection) Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
threat-detect binary behind feature flag
…inux-amd64 Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Updated |
|
@copilot enable feature flag for smoke-copilot and changeset generator agentic workflows |
…rkflows Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Added |
|
@copilot not seeing the new codegen in compiled changeset lock.yml file |
…nable detection job Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
The root cause was |
|
@copilot do not inject Python script in detection. Use bash or actions/github-script |
…n runs Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Fixed in the latest commit. Added |
|
📰 BREAKING: Smoke Copilot is now investigating this pull request. Sources say the story is developing... |
Smoke Test: Gemini - 27830499361\n\n- GitHub MCP Testing: ✅\n- Web Fetch Testing: ✅\n- File Writing Testing: ✅\n- Bash Tool Testing: ✅\n- Build gh-aw: ❌\n\nOverall status: FAILWarning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "localhost"See Network Configuration for more information.
|
Agent Container Tool Check
Result: 12/12 tools available ✅ Overall Status: PASS
|
|
Smoke test PASS Warning Firewall blocked 6 domainsThe following domains were blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "accounts.google.com"
- "android.clients.google.com"
- "clients2.google.com"
- "contentautofill.googleapis.com"
- "safebrowsingohttpgateway.googleapis.com"
- "www.google.com"See Network Configuration for more information.
|
🧪 Smoke Test: Claude — Run 27830499436Core #1-12: ✅ all passed Overall: PASS ✅ Warning Firewall blocked 6 domainsThe following domains were blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "accounts.google.com"
- "android.clients.google.com"
- "clients2.google.com"
- "contentautofill.googleapis.com"
- "safebrowsingohttpgateway.googleapis.com"
- "www.google.com"See Network Configuration for more information.
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
💥 Automated smoke test review - all systems nominal!
Warning
Firewall blocked 6 domains
The following domains were blocked by the firewall during workflow execution:
accounts.google.comandroid.clients.google.comclients2.google.comcontentautofill.googleapis.comsafebrowsingohttpgateway.googleapis.comwww.google.com
To allow these domains, add them to the
network.allowedlist in your workflow frontmatter:
network:
allowed:
- defaults
- "accounts.google.com"
- "android.clients.google.com"
- "clients2.google.com"
- "contentautofill.googleapis.com"
- "safebrowsingohttpgateway.googleapis.com"
- "www.google.com"See Network Configuration for more information.
💥 [THE END] — Illustrated by Smoke Claude · 69.2 AIC · ⌖ 28 AIC · ⊞ 8.5K
| engine: false | ||
| threat-detection: {} | ||
| timeout-minutes: 20 | ||
| features: |
There was a problem hiding this comment.
Smoke test: consider documenting this new features block in the workflow guide.
| threat-detection: {} | ||
| timeout-minutes: 20 | ||
| features: | ||
| gh-aw-detection: true |
There was a problem hiding this comment.
Smoke test: confirm gh-aw-detection: true is intended to be enabled by default here.
|
@copilot merge main, recompile |
There was a problem hiding this comment.
Smoke review: the requested PR review tools completed for run 27830567569.
Warning
Firewall blocked 5 domains
The following domains were blocked by the firewall during workflow execution:
accounts.google.comclients2.google.comcontentautofill.googleapis.comsafebrowsingohttpgateway.googleapis.comwww.google.com
To allow these domains, add them to the
network.allowedlist in your workflow frontmatter:
network:
allowed:
- defaults
- "accounts.google.com"
- "clients2.google.com"
- "contentautofill.googleapis.com"
- "safebrowsingohttpgateway.googleapis.com"
- "www.google.com"See Network Configuration for more information.
📰 BREAKING: Report filed by Smoke Copilot · 254.4 AIC · ⊞ 19.9K
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Done in |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Fixed in |
gh-aw currently runs threat detection inline — invoking the workflow's engine inside AWF and scraping the transcript for a
THREAT_DETECTION_RESULT:line, concluded byparse_threat_detection_results.cjs. This migrates to the standalone versionedthreat-detectbinary fromgithub/gh-aw-threat-detection, gated behindfeatures: gh-aw-detection: truefor incremental rollout.What changes
Feature flag & version pin
GHAWDetectionFeatureFlag = "gh-aw-detection"— opt-in, default off; existing compiled workflows unchangedDefaultThreatDetectVersion = "v0.2.2"inversion_constants.go— hard-pinned like AWF/mcpg, no lifecycle registryfeatures: gh-aw-detection: truenow emits the standard compiler warning:Using experimental feature: gh-aw-detectionNew constants
ThreatDetectionDir = "/tmp/gh-aw/threat-detection"ThreatDetectionResultPath = ".../detection_result.json"Install script (
actions/setup/sh/install_threat_detect_binary.sh)threat-detectfrom GitHub Releases, SHA256-verified, Linux amd64/arm64 and macOS x64/arm64; followsinstall_awf_binary.shpatternthreat-detect-linux-amd64matching the published release assetsCompiler (
threat_detection.go)buildDetectionJobStepsbranches at step 7 on the feature flagthreat-detect→ AWF execute with/tmp/gh-aw/threat-detectionmountedrw→ uploaddetection_result.json+detection.log→threat-detect conclude --result-file.cjs) unchangedExperimental warning coverage
pkg/workflow/compiler_validators_test.gofor enabled/disabled/absentgh-aw-detectioncases.Early adopter workflows
smoke-copilot.mdopts in viafeatures: gh-aw-detection: truechangeset.mdopts in viafeatures: gh-aw-detection: true; the pre-existingthreat-detection: engine: falseconfig was replaced withthreat-detection: {}so the detection job is compiled (when the inline engine is disabled with no custom steps,HasRunnableDetection()returnsfalseand skips the job entirely, making the feature flag a no-op)Gate contract preserved
The downstream interface is byte-identical to the inline path:
detection_conclusion/detection_reason/detection_successjob outputsGH_AW_DETECTION_CONCLUSION/GH_AW_DETECTION_REASONenv varsGH_AW_DETECTION_CONTINUE_ON_ERRORwarn-vs-strict semanticsOnly the producer changes: structured
detection_result.json+threat-detect concludeinstead of transcript-scrapedTHREAT_DETECTION_RESULT:+.cjs.Example
Compiled detection job (new path):
Notes
parse_threat_detection_results.cjsis not referenced in new-flag workflows but is not yet deleted (per rollout plan)✨ PR Review Safe Output Test - Run 27830499436
Warning
Firewall blocked 6 domains
The following domains were blocked by the firewall during workflow execution:
accounts.google.comandroid.clients.google.comclients2.google.comcontentautofill.googleapis.comsafebrowsingohttpgateway.googleapis.comwww.google.comSee Network Configuration for more information.