Pre-flight Checks
Problem Description
During SDD apply/fix iterations, the orchestrator runs the FULL test suite (all unit + all E2E gates) to validate even tiny, well-scoped changes. In a real session, a 6-line shader fix (moving one function from the fragment to the vertex stage) triggered the complete suite (~1300 vitest tests + 22 Playwright WebGL1 gates) multiple times across RED/GREEN/sweep — several minutes of validation for seconds of logic.
The full E2E suite has a large FIXED cost (launch browser + dev server + run all gates) that does NOT scale with change size. So for small fixes the validation overhead dominates and makes the iteration loop disproportionately slow. The fix logic was trivial; the validation was the bottleneck.
Proposed Solution
Make test scope dependency-aware. Each change determines what to test based on what it actually touches:
- During iteration (apply/fix loop): run only contextual tests — the test files for the touched modules + the specific E2E gate(s) affected (e.g.
vitest <file>, playwright -g <gate>). If the change touches SHARED infrastructure (e.g. a shared shader chunk/registry), also run the gates of that infra's known consumers — but still not the whole suite.
- At verify (
sdd-verify): run the FULL suite as the non-regression safety net.
Rule of thumb: the full suite runs once per integration-ready slice (at verify), not once per fix. This keeps the iteration loop fast while preserving full-coverage assurance before a change is declared done.
Affected Area
Other
Alternatives Considered
- Running the full suite every iteration (current behavior — correct but slow; the fixed E2E cost dominates small fixes).
- Running only the touched file's tests with no infra-consumer awareness (fast, but risks missing regressions in consumers of a shared dependency).
The proposed middle ground — dependency-aware scope during iteration + full suite at verify — balances speed and safety.
Additional Context
Surfaced in a real SDD session building a bespoke WebGL1 render engine: small shader fixes repeatedly paid the full-suite cost. Related but DISTINCT from #262 (TDD enforcement) and #869 (StrictWorkflow PR gates) — those govern whether tests/gates run; this governs which tests run when.
Pre-flight Checks
Problem Description
During SDD apply/fix iterations, the orchestrator runs the FULL test suite (all unit + all E2E gates) to validate even tiny, well-scoped changes. In a real session, a 6-line shader fix (moving one function from the fragment to the vertex stage) triggered the complete suite (~1300 vitest tests + 22 Playwright WebGL1 gates) multiple times across RED/GREEN/sweep — several minutes of validation for seconds of logic.
The full E2E suite has a large FIXED cost (launch browser + dev server + run all gates) that does NOT scale with change size. So for small fixes the validation overhead dominates and makes the iteration loop disproportionately slow. The fix logic was trivial; the validation was the bottleneck.
Proposed Solution
Make test scope dependency-aware. Each change determines what to test based on what it actually touches:
vitest <file>,playwright -g <gate>). If the change touches SHARED infrastructure (e.g. a shared shader chunk/registry), also run the gates of that infra's known consumers — but still not the whole suite.sdd-verify): run the FULL suite as the non-regression safety net.Rule of thumb: the full suite runs once per integration-ready slice (at verify), not once per fix. This keeps the iteration loop fast while preserving full-coverage assurance before a change is declared done.
Affected Area
Other
Alternatives Considered
The proposed middle ground — dependency-aware scope during iteration + full suite at verify — balances speed and safety.
Additional Context
Surfaced in a real SDD session building a bespoke WebGL1 render engine: small shader fixes repeatedly paid the full-suite cost. Related but DISTINCT from #262 (TDD enforcement) and #869 (StrictWorkflow PR gates) — those govern whether tests/gates run; this governs which tests run when.