Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 5 additions & 27 deletions crates/tests/fixtures/harness/build.rs
Original file line number Diff line number Diff line change
@@ -1,24 +1,10 @@
// Build script for `test_fixtures`.
//
// Walks `data/<group>/<name>/` and emits one `#[test]` per fixture into
// `$OUT_DIR/generated_tests.rs`. The generated test names encode the group so
// `cargo test <group>::` can target a single category.
//
// Each generated test calls back into `crate::run_fixture(group, name)`,
// which lives in the integration test (`tests/fixtures.rs`); `build.rs` only
// owns discovery + codegen.
//
// The build script also emits `compile_fixtures.rs`, which `src/lib.rs`
// `include!`s. That file contains one module per fixture that has an
// `expected.rs`, each `include!`ing that golden file. The result is that
// every `expected.rs` is fed through `rustc` whenever the harness crate
// compiles — so a malformed golden (bad syntax, references a non-existent
// item, etc.) breaks the build immediately rather than going unnoticed.
// `$OUT_DIR/generated_tests.rs`. Each test calls `crate::run_fixture(group,
// name)` in `tests/fixtures.rs`.
//
// Cargo runs `#[test]` functions in parallel by default, so generating one
// test per fixture preserves the parallelism of the existing roundtrip
// crates (each fixture writes scratch output into a unique `$OUT_DIR`
// subdirectory, so there is no contention on shared files).
// Also emits `$OUT_DIR/compile_fixtures.rs` (included by `src/lib.rs`)
// containing one module per fixture that has an `expected.rs`, each
// `include!`ing that golden so a malformed golden breaks the build.

use std::fmt::Write as _;
use std::path::{Path, PathBuf};
Expand All @@ -33,11 +19,6 @@ fn main() {
let mut tests = String::new();
tests.push_str("// @generated by build.rs — do not edit by hand.\n");

// `compile_fixtures.rs` is `include!`d from `src/lib.rs` so that every
// fixture's `expected.rs` is fed through `rustc` as part of the regular
// build, not just diffed as a string. Each fixture is wrapped in its own
// module so the `pub mod Test { ... }` blocks the various `expected.rs`
// files declare don't collide with each other.
let mut compile = String::new();
compile.push_str("// @generated by build.rs — do not edit by hand.\n");

Expand All @@ -59,8 +40,6 @@ fn main() {

let expected_rs = fixture_dir.join("expected.rs");
if expected_rs.is_file() {
// Use an absolute path so `include!` works regardless of
// where cargo invokes the build from.
let abs = std::fs::canonicalize(&expected_rs).unwrap_or(expected_rs);
writeln!(
compile,
Expand All @@ -70,7 +49,6 @@ fn main() {
.unwrap();
}
}
// Touch the group dir for change tracking even when empty.
println!("cargo:rerun-if-changed={}", group_dir.display());
}

Expand Down
60 changes: 20 additions & 40 deletions crates/tests/fixtures/harness/data/README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,7 @@
# Test fixture format

This directory holds the data-driven fixtures consumed by the
`test_fixtures` harness (`crates/tests/fixtures/harness`). It is the
single living document for how the parser/generator test suite is
organised; older planning notes have been removed now that the
consolidation is complete.
`test_fixtures` harness (`crates/tests/fixtures/harness`).

## Layout

Expand All @@ -17,16 +14,10 @@ crates/tests/fixtures/harness/data/
expected.rdl | expected.rs | expected.err
```

`build.rs` in the harness crate enumerates every `<group>/<name>/`
directory at compile time and emits one `#[test]` per fixture. Each
generated `#[test]` calls back into the harness's `run_fixture(group,
name)` dispatcher; nothing in `data/` is itself Rust code.

The harness writes scratch output under
`$OUT_DIR/scratch/<group>/<name>/`, so concurrent tests never share
filesystem state and Cargo's normal parallel `#[test]` execution is
preserved (this is what keeps the suite as fast as the existing
`roundtrip/{rdl,clang,bindgen}` crates).
`build.rs` enumerates every `<group>/<name>/` directory and emits one
`#[test]` per fixture; each test calls back into `run_fixture(group,
name)`. Scratch output goes under `$OUT_DIR/scratch/<group>/<name>/` so
parallel `#[test]` execution is preserved.

## Groups

Expand All @@ -41,8 +32,8 @@ preserved (this is what keeps the suite as fast as the existing

## `fixture.toml`

A deliberately tiny key/value subset of TOML, parsed without external
dependencies. Supported keys:
A tiny key/value subset of TOML, parsed without external dependencies.
Supported keys:

| key | type | applies to | meaning |
|------------------|------------|------------|-------------------------------------------|
Expand All @@ -53,11 +44,8 @@ dependencies. Supported keys:
| `no_comment` | bool | bindgen | pass `--no-comment` to bindgen |
| `specific_deps` | bool | bindgen | pass `--specific-deps` to bindgen |
| `kind` | string | error | `"reader"` (default), `"reader_no_input"`, or `"writer"` — which stage must fail |
| `arch_inputs` | string[] | merge | per-input arch tagging, e.g. `["input-x86.rdl=X86", "input-x64.rdl=X64"]`. Arches are `X86`/`X64`/`Arm64` or `\|`-joined. When set, the harness uses `Merger::arch_input` so types present in only some arches get a `SupportedArchitecture` attribute. |
| `outputs` | string[] | rdl | run the writer multiple times. Each entry is `"<expected>=<filter[;filter...]>"`; `;` separates multiple `writer.filter(...)` calls in a single invocation. When omitted, the runner falls back to one writer with `filter` (default `"Test"`) and `expected.rdl`. |

The format is a strict subset of real TOML so a fixture written today
will keep parsing if the harness later swaps in a full TOML crate.
| `arch_inputs` | string[] | merge | per-input arch tagging, e.g. `["input-x86.rdl=X86", "input-x64.rdl=X64"]`. Arches are `X86`/`X64`/`Arm64` or `\|`-joined. |
| `outputs` | string[] | rdl | run the writer multiple times. Each entry is `"<expected>=<filter[;filter...]>"`; `;` separates multiple `writer.filter(...)` calls. |

## Adding a fixture

Expand All @@ -73,33 +61,25 @@ that the actual output matches the committed golden.

## Filtering

Each fixture is a normal `#[test]`, so you can use the standard Cargo
filter syntax:
Each fixture is a normal `#[test]`, so standard Cargo filtering works:

```sh
# everything
cargo test -p test_fixtures

# just the bindgen group
cargo test -p test_fixtures bindgen_

# a single fixture
cargo test -p test_fixtures rdl_enum
cargo test -p test_fixtures # everything
cargo test -p test_fixtures bindgen_ # just the bindgen group
cargo test -p test_fixtures rdl_enum # a single fixture
```

## What stays bespoke (and why)
## What stays bespoke

A handful of test files under `crates/tests/` deliberately do **not** use
this fixture format. They exercise things the harness doesn't model
(runtime semantics of generated bindings, direct builder/reader-API
calls, OS-level I/O failures, structural assertions on attribute
values), and forcing them through a byte-stable RDL roundtrip would lose
coverage rather than add it:
A handful of test files under `crates/tests/` deliberately do not use
this fixture format because they exercise things the harness doesn't
model (runtime semantics of generated bindings, direct builder/reader
API calls, OS-level I/O failures, structural attribute assertions):

- `tests/libs/rdl/tests/{assembly_name, const-underlying, const-underlying-rdl, directory, error, exclusive-to, fn_abi, guid-derive, split, struct_fields, struct_values, writer_errors}.rs`
- `tests/libs/metadata/tests/{empty, struct, attribute, class, interface, reader, load_library, assembly_name}.rs`
- `tests/libs/bindgen/tests/{bool, deps, delegate_*, panic, ref_params}.rs`
- `tests/{misc,winrt}/**`

If a future change makes one of these expressible as a roundtrip,
golden-Rust, or error fixture, migrate it; otherwise leave it alone.
Migrate them as future changes make them expressible as a roundtrip,
golden-Rust, or error fixture; otherwise leave them alone.
17 changes: 4 additions & 13 deletions crates/tests/fixtures/harness/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,16 +1,7 @@
//! Compile-time validation of fixture golden Rust files.
//!
//! `build.rs` emits `compile_fixtures.rs` containing one module per fixture
//! that has an `expected.rs`, each `include!`ing the golden file. By
//! `include!`ing that generated file here, every `expected.rs` is compiled
//! as part of building this crate. That guarantees the goldens are not just
//! syntactically valid but also reference real items in the `windows-*`
//! crates they depend on — so a stale golden breaks the build.
//!
//! Gated on `cfg(windows)` for now: some goldens reference items (e.g.
//! `IMarshal`-related glue) that are not available on non-Windows targets.
//!
//! The fixture-execution test logic still lives in `tests/fixtures.rs`.
//! Compile every fixture's `expected.rs` so stale goldens break the build.
//! `build.rs` emits the include file. Test-execution logic lives in
//! `tests/fixtures.rs`. Gated on `cfg(windows)` because some goldens
//! reference Windows-only items.

#[cfg(windows)]
include!(concat!(env!("OUT_DIR"), "/compile_fixtures.rs"));
Loading
Loading