Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
70cbce0
feat: add coverage snapshot HTTP server for code coverage POC
sohil-kshirsagar Apr 1, 2026
11ab1d4
fix: start coverage server before SDK mode checks
sohil-kshirsagar Apr 1, 2026
0731adf
feat: switch coverage from NYC to V8 native (NODE_V8_COVERAGE + v8.ta…
sohil-kshirsagar Apr 1, 2026
6307d9f
feat: add ?baseline=true parameter for full coverable line map
sohil-kshirsagar Apr 1, 2026
3582ddc
chore: add unref() to coverage server to prevent blocking process exit
sohil-kshirsagar Apr 1, 2026
ced22b3
fix: correct V8 range nesting for accurate uncovered line detection
sohil-kshirsagar Apr 1, 2026
b5bf290
fix: use innermost-range-wins for per-test mode too
sohil-kshirsagar Apr 1, 2026
31afe08
refactor: extract V8 coverage processing into testable module
sohil-kshirsagar Apr 1, 2026
8ff19d4
feat: add branch coverage tracking
sohil-kshirsagar Apr 2, 2026
0aaea01
feat: use v8-to-istanbul for accurate branch coverage
sohil-kshirsagar Apr 2, 2026
ce2a429
feat: switch from v8-to-istanbul to ast-v8-to-istanbul
sohil-kshirsagar Apr 2, 2026
c3f808d
feat: add source map support for TypeScript coverage
sohil-kshirsagar Apr 2, 2026
82afd71
feat: add ts-node support for coverage
sohil-kshirsagar Apr 2, 2026
cb65af5
feat: migrate coverage from HTTP server to protobuf channel
sohil-kshirsagar Apr 2, 2026
060d656
docs: add code coverage documentation
sohil-kshirsagar Apr 3, 2026
69aa774
fix: coverage code quality improvements
sohil-kshirsagar Apr 3, 2026
b270d16
docs: clean up AI writing patterns in coverage doc
sohil-kshirsagar Apr 3, 2026
1e77d1d
fix: address bugbot review feedback
sohil-kshirsagar Apr 3, 2026
53f7539
chore: update @use-tusk/drift-schemas to ^0.1.34
sohil-kshirsagar Apr 7, 2026
4681398
fix: use nullish coalescing for numeric defaults in coverage processor
sohil-kshirsagar Apr 7, 2026
94c30ec
test: add URL-encoded path test for filterScriptUrl
sohil-kshirsagar Apr 7, 2026
998944b
test: add Tusk-generated tests for coverage processor and protobuf ha…
sohil-kshirsagar Apr 7, 2026
28e3e3a
fix: ts-node candidate path should replace extension, not append .js
sohil-kshirsagar Apr 7, 2026
c85da1d
fix: add path boundary check in filterScriptUrl to prevent prefix col…
sohil-kshirsagar Apr 7, 2026
a095168
fix: handle sourceRoot=/ in filterScriptUrl (Docker root directory)
sohil-kshirsagar Apr 7, 2026
031ba83
fix: pass CJS wrapperLength to ast-v8-to-istanbul for correct byte of…
sohil-kshirsagar Apr 7, 2026
4997321
ref: use require('module').wrapper[0].length instead of hardcoded 62
sohil-kshirsagar Apr 7, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 90 additions & 0 deletions docs/coverage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Code Coverage (Node.js)

The Node SDK collects per-test code coverage during Tusk Drift replay using V8's built-in precise coverage. No external dependencies like NYC or c8 are needed.

## How It Works

### V8 Precise Coverage

When coverage is enabled (via `--show-coverage`, `--coverage-output`, or `coverage.enabled: true` in config), the CLI sets `NODE_V8_COVERAGE=<temp-dir>`. This tells V8 to enable precise coverage collection internally:

```
V8 internally calls: Profiler.startPreciseCoverage({ callCount: true, detailed: true })
```

This provides:
- **Real execution counts** (1, 2, 5...) not just binary covered/uncovered
- **Block-level granularity**: branches, loops, expressions
- **Zero external dependencies** — works with any Node.js version that supports `NODE_V8_COVERAGE`
- **Works with CJS, ESM, TypeScript, bundled code** — anything V8 executes

### Snapshot Flow

1. **Baseline**: After the service starts, the CLI sends a `CoverageSnapshotRequest(baseline=true)`. The SDK calls `v8.takeCoverage()`, which writes a JSON file to the `NODE_V8_COVERAGE` directory and **resets all counters**. The baseline captures all coverable lines (including uncovered at count=0) for the coverage denominator.

2. **Per-test**: After each test, the CLI sends `CoverageSnapshotRequest(baseline=false)`. The SDK calls `v8.takeCoverage()` again. Because counters were reset, the result contains **only lines executed by this specific test** — no diffing needed.

3. **Processing**: The SDK processes the V8 JSON using `ast-v8-to-istanbul`, which converts V8 byte ranges into Istanbul-format line/branch coverage. The result is sent back to the CLI via protobuf.

### Why ast-v8-to-istanbul (not v8-to-istanbul)

After `v8.takeCoverage()` resets counters, V8 only reports functions that were called since the reset. Functions that were never called are **absent** from the V8 output.

- **v8-to-istanbul** assumes complete V8 data. Missing functions are treated as "covered by default." This produces 100% coverage for files where only `/health` was hit.
- **ast-v8-to-istanbul** parses the source file's AST independently. It knows about ALL functions from the AST and correctly marks missing ones as uncovered.

This is the key reason we use `ast-v8-to-istanbul`.

### Source Map Support

For TypeScript projects using `tsc`, the SDK automatically:

1. Detects `//# sourceMappingURL=<file>.map` comments in compiled JS
2. Loads the `.map` file
3. Fixes `sourceRoot` if present (TypeScript sets `sourceRoot: "/"` which breaks ast-v8-to-istanbul's internal path resolution — the SDK resolves sources relative to the actual project root)
4. Strips the `sourceMappingURL` comment from code passed to ast-v8-to-istanbul (prevents it from loading the unpatched `.map` file)
5. Passes the fixed source map to ast-v8-to-istanbul, which remaps coverage to original `.ts` files

**Requirements:** `sourceMap: true` in `tsconfig.json`. Source map files (`.js.map`) must be present alongside compiled output.

**Supported setups:**
| Setup | Status |
|-------|--------|
| `tsc` -> `node dist/` | Works (tested) |
| `swc`/`esbuild` (compile, not bundle) -> `node dist/` | Should work (same `.js` + `.js.map` pattern) |
| `ts-node` with `TS_NODE_EMIT=true` | Works (CLI sets this automatically) |
| `ts-node-dev` | Limited — lazy compilation means only executed files have coverage |
| Bundled (webpack/esbuild/Rollup) | Untested — should work if source maps are produced |

### Multi-PID Handling

The start command in `.tusk/config.yaml` often chains processes:

```
rm -rf dist && npm run build && node dist/server.js
```

This creates multiple Node processes (npm, tsc, the server), all inheriting `NODE_V8_COVERAGE`. Each writes its own V8 coverage file. The SDK handles this by **quick-scanning** each file to check for user scripts before running the expensive ast-v8-to-istanbul processing. Files from npm/tsc (which have 0 user scripts) are skipped.

### CJS vs ESM

The SDK tries parsing source code as CJS (`sourceType: "script"`) first, falling back to ESM (`sourceType: "module"`) if that fails. This handles both module formats without configuration.

## Environment Variables

These are set automatically by the CLI when coverage is enabled. You should not set them manually.

| Variable | Description |
|----------|-------------|
| `NODE_V8_COVERAGE` | Directory for V8 to write coverage JSON files. Set by CLI. |
| `TUSK_COVERAGE` | Language-agnostic signal that coverage is enabled. Set by CLI. |
| `TS_NODE_EMIT` | Forces ts-node to write compiled JS to disk (needed for coverage processing). Set by CLI. |

## Limitations

- **acorn parse failures**: `ast-v8-to-istanbul` uses acorn to parse JavaScript. Files using syntax acorn doesn't support (stage 3 proposals, certain decorator patterns) are silently skipped.
- **Stale `dist/` artifacts**: `tsc` doesn't clean old output files. If a source file was renamed or moved, the old compiled file remains in `dist/` and may have broken imports. Use `rm -rf dist` before `tsc` in your start command.
- **Multi-process apps**: If your app uses Node's cluster module or PM2 to fork workers, each worker is a separate process. Only the worker connected to the CLI's protobuf channel handles coverage requests.
- **Dynamic imports**: Modules loaded via dynamic `import()` after startup aren't in the baseline snapshot. Their uncovered functions won't be in the denominator.
- **Large codebases**: Processing 500+ files from a 24MB V8 JSON takes several seconds. The CLI has a 30-second timeout for coverage snapshots.
- **ts-node-dev**: Lazy compilation means only files accessed during the test are compiled and covered. Pre-compiled `tsc` output gives much better coverage.
13 changes: 13 additions & 0 deletions docs/environment-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,20 @@ TUSK_USE_RUST_CORE=1 npm start
TUSK_USE_RUST_CORE=0 npm start
```

## Coverage Variables

These are set automatically by the CLI when `tusk drift run --coverage` is used. You should **not** set them manually.

| Variable | Description |
|----------|-------------|
| `NODE_V8_COVERAGE` | Directory for V8 to write coverage JSON files. Enables V8 precise coverage collection. |
| `TUSK_COVERAGE` | Language-agnostic signal that coverage is enabled. Set to `true`. |
| `TS_NODE_EMIT` | Forces ts-node to write compiled JS to disk (needed for coverage processing). Set to `true`. |

See [Coverage Guide](./coverage.md) for details on how coverage collection works.

## Related Documentation

- [Initialization Guide](./initialization.md) - SDK initialization parameters and config file settings
- [Quick Start Guide](./quickstart.md) - Record and replay your first trace
- [Coverage Guide](./coverage.md) - Code coverage during test replay
49 changes: 42 additions & 7 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@
"@types/supertest": "^6.0.2",
"@typescript-eslint/eslint-plugin": "^6.0.0",
"@typescript-eslint/parser": "^6.0.0",
"@use-tusk/drift-schemas": "^0.1.24",
"@use-tusk/drift-schemas": "^0.1.34",
"ava": "^6.4.1",
"axios": "^1.6.0",
"eslint": "^8.57.1",
Expand All @@ -109,6 +109,8 @@
},
"dependencies": {
"@use-tusk/drift-core-node": "^0.1.8",
"acorn": "^8.14.0",
"ast-v8-to-istanbul": "^1.0.0",
"import-in-the-middle": "^1.14.4",
"js-yaml": "^4.1.0",
"jsonpath": "^1.1.1",
Expand Down
143 changes: 143 additions & 0 deletions src/core/ProtobufCommunicator.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
import test from "ava";
import { ProtobufCommunicator } from "./ProtobufCommunicator";
import {
SDKMessage,
MessageType,
FileCoverageData,
BranchInfo,
} from "@use-tusk/drift-schemas/core/communication";
import net from "net";

// --- handleCoverageSnapshotRequest tests ---

// Helper to create a test ProtobufCommunicator instance
function createTestCommunicator(): {
communicator: ProtobufCommunicator;
sentMessages: SDKMessage[];
mockSocket: net.Socket;
} {
const sentMessages: SDKMessage[] = [];
const mockSocket = new net.Socket();

const communicator = new ProtobufCommunicator();

// Access private method via prototype manipulation for testing
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(communicator as any).sendProtobufMessage = async (message: SDKMessage) => {
sentMessages.push(message);
};

// Set up a mock client connection
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(communicator as any).client = mockSocket;

return { communicator, sentMessages, mockSocket };
}

test("handleCoverageSnapshotRequest: returns error when NODE_V8_COVERAGE not set", async (t) => {
const { communicator, sentMessages } = createTestCommunicator();

// Ensure NODE_V8_COVERAGE is not set
const originalEnv = process.env.NODE_V8_COVERAGE;
delete process.env.NODE_V8_COVERAGE;

try {
// Invoke the private method via message handling
// eslint-disable-next-line @typescript-eslint/no-explicit-any
await (communicator as any).handleCoverageSnapshotRequest("test-req-1", false);

// Check that error response was sent
t.is(sentMessages.length, 1);
t.is(sentMessages[0].type, MessageType.COVERAGE_SNAPSHOT);
t.is(sentMessages[0].payload.oneofKind, "coverageSnapshotResponse");
if (sentMessages[0].payload.oneofKind === "coverageSnapshotResponse") {
t.false(sentMessages[0].payload.coverageSnapshotResponse.success);
t.is(sentMessages[0].payload.coverageSnapshotResponse.error, "NODE_V8_COVERAGE not set");
}
} finally {
if (originalEnv !== undefined) {
process.env.NODE_V8_COVERAGE = originalEnv;
}
}
});

test("handleCoverageSnapshotRequest: handles errors during processing", async (t) => {
const originalEnv = process.env.NODE_V8_COVERAGE;
// Set to a non-existent directory to trigger error
process.env.NODE_V8_COVERAGE = "/nonexistent/coverage/dir";

try {
const { communicator, sentMessages } = createTestCommunicator();

// eslint-disable-next-line @typescript-eslint/no-explicit-any
await (communicator as any).handleCoverageSnapshotRequest("test-req-4", false);

// Should send error response
t.is(sentMessages.length, 1);
if (sentMessages[0].payload.oneofKind === "coverageSnapshotResponse") {
const response = sentMessages[0].payload.coverageSnapshotResponse;
t.false(response.success);
t.truthy(response.error);
t.true(response.error.length > 0);
}
} finally {
if (originalEnv !== undefined) {
process.env.NODE_V8_COVERAGE = originalEnv;
} else {
delete process.env.NODE_V8_COVERAGE;
}
}
});

// --- sendCoverageResponse tests ---

test("sendCoverageResponse: creates correct message structure", async (t) => {
const { communicator, sentMessages } = createTestCommunicator();

const mockCoverage: Record<string, FileCoverageData> = {
"/path/to/file.js": FileCoverageData.create({
lines: { "1": 1, "2": 2 },
totalBranches: 4,
coveredBranches: 2,
branches: {
"3": BranchInfo.create({ total: 2, covered: 1 }),
"5": BranchInfo.create({ total: 2, covered: 1 }),
},
}),
};

// eslint-disable-next-line @typescript-eslint/no-explicit-any
await (communicator as any).sendCoverageResponse("req-123", true, "", mockCoverage);

t.is(sentMessages.length, 1);
const msg = sentMessages[0];

t.is(msg.type, MessageType.COVERAGE_SNAPSHOT);
t.is(msg.requestId, "req-123");
t.is(msg.payload.oneofKind, "coverageSnapshotResponse");

if (msg.payload.oneofKind === "coverageSnapshotResponse") {
const response = msg.payload.coverageSnapshotResponse;
t.true(response.success);
t.is(response.error, "");
t.deepEqual(response.coverage, mockCoverage);
}
});

test("sendCoverageResponse: handles error responses", async (t) => {
const { communicator, sentMessages } = createTestCommunicator();

// eslint-disable-next-line @typescript-eslint/no-explicit-any
await (communicator as any).sendCoverageResponse("req-456", false, "Test error message", {});

t.is(sentMessages.length, 1);
const msg = sentMessages[0];

if (msg.payload.oneofKind === "coverageSnapshotResponse") {
const response = msg.payload.coverageSnapshotResponse;
t.false(response.success);
t.is(response.error, "Test error message");
t.deepEqual(response.coverage, {});
}
});

Loading
Loading