feat(coverage): Add code coverage collection and export

[sohil-kshirsagar]

Add code coverage collection to the Tusk Drift CLI. Collects per-test and aggregate code coverage during test replay, with LCOV/JSON export, config-driven activation for CI, and backend upload.

Coverage activation

Two modes:

• Config-driven (CI): coverage.enabled: true in .tusk/config.yaml → automatically collects during validation runs on the default branch. Silent (no console output). No CI pipeline changes needed.
• Flag-driven (local): --show-coverage displays coverage, --coverage-output writes LCOV/JSON.

Architecture

1. CLI sets TUSK_COVERAGE=true + NODE_V8_COVERAGE=<dir> env vars
2. After service ready, CLI takes baseline snapshot (all coverable lines for denominator)
3. After each test, CLI takes per-test snapshot (delta since last reset)
4. Per-test coverage uploaded to backend via TraceTestCoverageData on TraceTestResult
5. Baseline + startup coverage uploaded via CoverageBaseline on UpdateDriftRunCIStatusRequest
6. Backend computes aggregate from IN_SUITE tests post-promotion

Config options

coverage:
  enabled: true                    # auto-collect during validation runs
  include: ["backend/src/**"]      # restrict to service code (monorepos)
  exclude: ["**/migrations/**"]    # exclude noise
  strip_path_prefix: "/app"        # Docker: strip container mount point

Key features

• Baseline + per-test snapshot architecture with V8 counter auto-reset
• Branch coverage with union semantics (sum covered, clamp to total)
• LCOV and JSON export formats
• Include/exclude glob filtering with doublestar
• In-suite-only filtering for validation runs (draft tests excluded from output)
• TUI integration: baseline capture, aggregate in service logs, sorted per-test display
• Docker Compose support via strip_path_prefix
• Backend upload: per-test coverage piggybacked on test result upload, baseline on CI status update
• Startup coverage included in aggregate (matches industry standard)
• CLI and backend produce identical coverage numbers

TODOs before merge

• Blocked on feat(proto): Add coverage tracking messages tusk-drift-schemas#47 — go.mod replace directive removed, updated to v0.1.34
• Blocked on feat(coverage): Add V8 code coverage collection drift-node-sdk#150 and feat(coverage): Add coverage.py code coverage collection drift-python-sdk#87 — SDKs updated, E2E tested with published schemas
• Add LCOV/JSON export tests

Edge cases / gotchas

• NODE_V8_COVERAGE is inherited by all child processes. SDK quick-scans to skip non-server files.
• Stale dist/ artifacts from tsc cause import errors. Use rm -rf dist before build.
• Docker paths are container-absolute. strip_path_prefix strips the mount point.
• Coverage callback runs BEFORE upload callback so per-test data is available in the proto.
• GetCoverageBaselineForUpload merges baseline + per-test records for complete denominator.

[cubic-dev-ai]

6 issues found across 16 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="internal/runner/server.go">

<violation number="1" location="internal/runner/server.go:1015">
P1: `SendCoverageSnapshot` registers the pending response after sending the request, which can drop fast SDK responses and cause intermittent timeouts.</violation>
</file>

<file name="internal/runner/coverage.go">

<violation number="1" location="internal/runner/coverage.go:201">
P3: `e.coveragePerTest` is accessed without holding `coveragePerTestMu` here (and similarly in `printCoverageSummary` and `FormatCoverageSummaryLines`), while `SetTestCoverageDetail` and `GetTestCoverageDetail` properly lock the mutex. Currently safe because coverage forces concurrency to 1 and these reads occur after all tests complete, but the inconsistent locking would produce a data race under the Go race detector if concurrency assumptions ever change. Lock (or copy under lock) before passing the map to `WriteCoverageJSON`.</violation>

<violation number="2" location="internal/runner/coverage.go:516">
P2: Include/exclude coverage globs won't match on Windows because `doublestar.Match` expects `/`-separated paths.</violation>

<violation number="3" location="internal/runner/coverage.go:541">
P2: Compute the JSON summary after filtering per-test files; `summary.per_test` can currently disagree with the exported aggregate and `per_test` sections.</violation>
</file>

<file name="internal/tui/test_executor.go">

<violation number="1" location="internal/tui/test_executor.go:768">
P2: Only log "Coverage written" when coverage processing succeeds; currently it logs success even after an error.</violation>
</file>

<file name="internal/runner/executor.go">

<violation number="1" location="internal/runner/executor.go:581">
P2: Branch baseline merge ignores higher covered counts when totals are equal, which can undercount branch coverage across environment-group baseline snapshots.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review, or fix all with cubic.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 5821c20. Configure here.}

[jy-tan]

should WriteCoverageJSON(), printCoverageSummary(), and FormatCoverageSummaryLines() call a shared helper for consistency?

something like BuildCoverageReportView(rawSession, reportOptions) that applies suite filtering and include/exclude filtering exactly once.

type CoverageReportView struct {
    Records   []CoverageTestRecord
    Aggregate CoverageSnapshot
    PerTest   map[string]map[string]CoverageFileDiff
    Summary   CoverageSummary
}

• printCoverageSummary() uses view.Summary
• FormatCoverageSummaryLines() uses view.Summary
• WriteCoverageJSON() writes view.Aggregate, view.PerTest, and view.Summary

[cubic-dev-ai]

1 issue found across 2 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="internal/runner/executor.go">

<violation number="1" location="internal/runner/executor.go:531">
P2: This early return drops valid baseline-only coverage. When baseline exists but no per-test records are present, aggregate coverage should still be uploaded from baseline instead of returning nil.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review, or fix all with cubic.}