Skip to content

Commit 750d75f

Browse files
committed
docs: enhance domain parity guide with workshop agent and governance instructions for FinOps
1 parent c4cde5a commit 750d75f

1 file changed

Lines changed: 63 additions & 2 deletions

File tree

docs/domain-parity-and-contribution.md

Lines changed: 63 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ The demo-app repo owns all scanning logic, Copilot artifacts, and infrastructure
6060
| **Scanner deployment** | Azure App Service (Docker container) | GitHub Actions only (no deployed scanner app) |
6161
| **Open source tools** | axe-core 4.11, IBM Equal Access 4.0 | PSRule for Azure, Checkov, Cloud Custodian, Infracost |
6262
| **Custom tools** | 5 custom Playwright checks, CLI, SARIF generator, scoring engine, PDF/HTML reports | 2 Python SARIF converters, 4 Cloud Custodian policies |
63-
| **SARIF generation** | Native (built-in SARIF v2.1.0 generator) | Mixed: PSRule and Checkov native; Cloud Custodian and Infracost via Python converters |
63+
| **SARIF generation** | Native (built-in SARIF v2.1.0 generator) | Mixed: PSRule and Checkov native; Cloud Custodian and Infracost via 2 Python converters |
6464
| **SARIF upload** | `codeql-action/upload-sarif@v4` (same-repo) | Cross-repo upload via GitHub REST API (`gh api`) |
6565
| **Copilot agents** | 2 (a11y-detector, a11y-resolver) | 5 (CostAnalysis, FinOpsGovernance, CostAnomalyDetector, CostOptimizer, DeploymentCostGate) |
6666
| **Copilot prompts** | 2 (a11y-scan, a11y-fix) | 2 (finops-scan, finops-fix) |
@@ -91,6 +91,25 @@ The demo-app repo owns all scanning logic, Copilot artifacts, and infrastructure
9191
| **Contributing guide** | Yes (lab authoring style guide) | Yes (lab authoring style guide) |
9292
| **License** | MIT | MIT |
9393

94+
The Accessibility workshop includes a workshop-specific Copilot agent that provides guided assistance during lab exercises, along with governance instructions that enforce coding standards within the workshop codebase. The FinOps workshop does not have equivalent Copilot artifacts. Adding a workshop agent and governance instructions to the FinOps workshop would bring AI-assisted lab guidance to parity. See [Gaps Identified](#gaps-identified) for remediation details.
95+
96+
#### Screenshot Script Comparison
97+
98+
Both workshops include automated screenshot capture scripts with comparable capabilities:
99+
100+
| Capability | Accessibility Workshop | FinOps Workshop |
101+
|------------|----------------------|------------------|
102+
| Script size | ~900+ lines | ~710+ lines |
103+
| Target screenshots | 47 PNGs | 46 PNGs |
104+
| Architecture | 3-phase (offline, app-dependent, GitHub web UI) | Multi-phase |
105+
| Charm freeze support | Yes | Yes |
106+
| Playwright helpers | Separate `playwright-helpers.js` with 3 functions | Inline Playwright usage |
107+
| Lab and phase filtering | Yes | Yes |
108+
| Theme and font customization | Yes | Yes |
109+
| Environment modes | local/azure | local/azure |
110+
111+
New domains should follow this pattern and create a `capture-screenshots.ps1` script covering all labs, with phase filtering and both local and Azure environment support.
112+
94113
### Power BI Report (`advsec-pbi-report-ado`)
95114

96115
| Aspect | Current State | Gap |
@@ -114,6 +133,34 @@ Making Azure DevOps a first-class citizen means every GitHub Actions workflow ha
114133
| **FinOps** | `finops-scan.yml`, `finops-cost-gate.yml` | None | Not implemented |
115134
| **APM Security** | `apm-security.yml` | Inline pattern in docs | Not implemented |
116135

136+
## Gaps Identified
137+
138+
Research across both domains reveals four specific parity gaps where FinOps does not match Accessibility capabilities.
139+
140+
### Gap 1: FinOps Workshop Has No Copilot Artifacts
141+
142+
The Accessibility workshop repository includes a workshop-specific agent and governance instructions. The FinOps workshop repository contains zero Copilot artifacts: no agents, prompts, instructions, or skills.
143+
144+
To close this gap, create a FinOps workshop agent in `.github/agents/` and add governance instructions to `.github/instructions/` in the `finops-scan-workshop` repository, following the patterns established in the Accessibility workshop.
145+
146+
### Gap 2: FinOps Demo App Has No ADO Pipelines
147+
148+
The Accessibility demo app repository includes 8 Azure DevOps pipelines covering CI/CD, deployment orchestration, scan variants, and reusable templates. The FinOps demo app repository has none.
149+
150+
To close this gap, create equivalent ADO pipelines in `.azuredevops/pipelines/` in the `finops-scan-demo-app` repository. Reference the Accessibility pipeline templates for structure and task mapping.
151+
152+
### Gap 3: No Domain-Specific Power BI Pages Exist
153+
154+
The Power BI report (`advsec-pbi-report-ado`) contains only 3 security-focused pages (Overview, Alerts by Type, Trend Analysis). Neither Accessibility nor FinOps has domain-specific pages in the report, though FinOps has detailed design specifications for 6 dashboard pages documented in its `docs/` directory.
155+
156+
To close this gap, add domain-specific pages to the Power BI report. Start with FinOps pages because full design specifications already exist, then create equivalent pages for Accessibility.
157+
158+
### Gap 4: SARIF Generation Approach Differs Between Domains
159+
160+
The Accessibility scanner generates SARIF v2.1.0 natively through a built-in TypeScript generator (`src/lib/report/sarif-generator.ts`). The FinOps domain requires 2 Python converter scripts (`custodian-to-sarif.py` and `infracost-to-sarif.py`) because Cloud Custodian and Infracost do not produce SARIF natively. PSRule and Checkov produce SARIF natively without converters.
161+
162+
This is a design difference driven by tool capabilities rather than a gap requiring remediation. New domains should evaluate SARIF capabilities during tool selection (Step 2 of the contribution guide) and document any converters needed.
163+
117164
## Contributing a New Domain
118165

119166
This section provides a step-by-step guide for adding a new scanning domain to the framework. We use **Code Quality** as a concrete example.
@@ -255,9 +302,17 @@ Copy the agent patterns from `agentic-accelerator-framework/agents/`:
255302

256303
* **code-quality-scan/SKILL.md** — Domain knowledge package covering supported tools, coverage formats (lcov, cobertura, JaCoCo XML, Go cover), SARIF mapping rules, and multi-language testing patterns
257304

305+
Skills can exist in two locations: the domain's demo-app repository (for scanner-specific knowledge, as in the FinOps `finops-scan` skill) and the framework repository (for cross-domain reference, as in the `a11y-scan` and `security-scan` skills in `agentic-accelerator-framework/skills/`). Create the skill in the demo-app repo first, then consider adding a corresponding skill to the framework repo if the knowledge benefits users working across domains.
306+
258307
### Step 6: Implement SARIF Integration
259308

260-
All findings must output SARIF v2.1.0 for GitHub Security Overview and ADO Advanced Security:
309+
All findings must output SARIF v2.1.0 for GitHub Security Overview and ADO Advanced Security. Each SARIF file must include:
310+
311+
* `partialFingerprints` for deduplication across runs
312+
* `automationDetails.id` set to the domain category prefix (e.g., `code-quality/coverage/`)
313+
* `runs[].tool.driver.name` populated with the scanner or agent name
314+
* `runs[].tool.driver.rules[]` with unique `ruleId` values per finding type
315+
* `runs[].results[].level` mapped from severity (CRITICAL/HIGH → `error`, MEDIUM → `warning`, LOW → `note`)
261316

262317
```text
263318
SARIF Flow:
@@ -377,6 +432,8 @@ Use this checklist when contributing a new domain:
377432

378433
- [ ] Create repository `{domain}-scan-demo-app`
379434
- [ ] Add 5 sample apps with intentional violations in different languages
435+
- [ ] Add Dockerfile per sample app
436+
- [ ] Add `start-local.ps1` and `stop-local.ps1` per sample app
380437
- [ ] Implement or integrate 3–5 open source scanning tools
381438
- [ ] Write SARIF converters for tools without native SARIF output
382439
- [ ] Create Copilot agents (detector + resolver pattern)
@@ -386,6 +443,7 @@ Use this checklist when contributing a new domain:
386443
- [ ] Create `bootstrap-demo-apps.ps1` (repo creation, OIDC, secrets)
387444
- [ ] Create `setup-oidc.ps1` (Azure AD federation)
388445
- [ ] Create GitHub Actions workflows (ci, scan, gate, deploy-all)
446+
- [ ] Create reusable GitHub Action (composite action) for external CI use
389447
- [ ] Create Azure DevOps pipelines (ci-cd, scan, deploy-all, templates)
390448
- [ ] Document Power BI data model and dashboard design
391449
- [ ] Document Power Query M expressions for data ingestion
@@ -400,6 +458,9 @@ Use this checklist when contributing a new domain:
400458
- [ ] Configure Jekyll for GitHub Pages
401459
- [ ] Write contributing guide with lab authoring style
402460
- [ ] Define half-day and full-day delivery tiers
461+
- [ ] Create workshop Copilot agent for lab guidance
462+
- [ ] Create workshop governance instructions
463+
- [ ] Create Playwright helpers script for browser automation
403464

404465
### Framework Integration
405466

0 commit comments

Comments
 (0)