Skip to content

fix(kani): make the crate compile under cfg(kani), and stop swallowing the failure (GH-212) - #221

Merged
noahgift merged 1 commit into
mainfrom
fix/kani-cfg-build
Aug 11, 2026
Merged

fix(kani): make the crate compile under cfg(kani), and stop swallowing the failure (GH-212)#221
noahgift merged 1 commit into
mainfrom
fix/kani-cfg-build

Conversation

@noahgift

Copy link
Copy Markdown
Contributor

Closes #212. Follow-up filed as #220.

cargo +nightly kani -p bashrs --harness <any> failed to build the crate — 19 errors in bashrs's own harness files. So no harness could run, including newly added correct ones, and every contract declaring kani_harnesses: was undischarged. pv validate passing meant the contract was well-formed, not that its obligations held.

It stayed invisible because every kani invocation in the Makefile was suffixed || true.

Three problems, only the first a typo

  1. kani::assert! is not a macro Kani exports. Kani intercepts the standard assert!/assert_eq!.
  2. escape_shell_value, is_valid_rust0, validate_rust0_ast do not exist in this crate, and there's no evidence they ever did.
  3. String/&str are not kani::Arbitrary — and cannot be. They're unbounded heap types; BMC needs a finite state space. Every let s: String = kani::any(); was an E0277. Added crate::kani_bounded: fixed byte array + symbolic length + alphabet constraint.

Two harnesses deleted, not repaired

Compiling them would have produced proofs of nothing:

  • verify_array_bounds_safety — asserted true on one branch; on the other, that a format! literal contains "-lt". No code under test was ever reached.
  • verify_parser_soundness — called the real parser on a symbolic string, checked against the two functions that don't exist. Not tractable under BMC even if they were supplied.

Two more were near-vacuous and now call the real escapers: one asserted properties of a format! literal that hold even if the escaper is the identity function; the other checked a toy re-implementation local to the file against a toy oracle.

A harness that cannot fail is worse than a missing one — it consumes the verification budget and reports success.

verify-kani now separates two different things

outcome behaviour why
build failure hard fail This is the GH-212 defect. Caught by --only-codegen in ~106s, no solver needed.
no convergence UNPROVEN, reported, non-fatal "Didn't finish in budget" ≠ "property is false". Failing here would make make verify permanently red — which is how a gate teaches people to bypass it.

Verified differentially

fixed                → --only-codegen: 0 errors, 106s; target exits 0
E0277 reintroduced   → target exits 1: "does not compile under cfg(kani)"

What this does NOT claim

None of the three surviving harnesses converge. Measured and reported rather than papered over: CBMC spends its time in alloc::raw_vec / Layout / LayoutError — Rust's allocator — because every harness calls an escaper returning a heap String. Bounds of 8, 4 and 2 characters all time out, so string length was never the bottleneck.

Fixing that means giving the escapers an allocation-free core (escape_shell_string_into(&str, &mut [u8])) for harnesses to target. That changes the public signature of functions every emitted script depends on, so it's filed as #220 rather than riding along here.

What this PR buys is that the harnesses now build — the precondition for any of it, and the thing that was silently false for months.

🤖 Generated with Claude Code

… the failure (Refs #212)

`cargo +nightly kani -p bashrs --harness <any>` failed to build the crate: 19
errors in bashrs's OWN harness files (16 in verifier/, 3 in formal/). So no
harness could run, including newly added correct ones, and every contract
declaring `kani_harnesses:` was undischarged. `pv validate` passing meant the
contract was well-formed, not that its obligations held.

It stayed invisible because Makefile:565 ran every kani invocation with
`|| true`.

Three distinct problems, only the first a typo:

1. `kani::assert!` is not a macro Kani exports. Kani intercepts the standard
   `assert!`/`assert_eq!`.
2. `escape_shell_value`, `is_valid_rust0` and `validate_rust0_ast` do not exist
   in this crate, and there is no evidence they ever did.
3. `String`/`&str` are not `kani::Arbitrary` and cannot be — they are unbounded
   heap types and BMC needs a finite state space. Every
   `let s: String = kani::any();` was an E0277. Added crate::kani_bounded with
   a fixed byte array + symbolic length + alphabet constraint.

Two harnesses are DELETED rather than repaired, because compiling them would
have produced proofs of nothing:

  verify_array_bounds_safety  asserted `true` on one branch, and on the other
                              that a format! literal contains "-lt". No code
                              under test was ever reached.
  verify_parser_soundness     called the real parser on a symbolic string and
                              checked it against the two functions that do not
                              exist. Not tractable under BMC even if supplied.

verify_variable_expansion_safety and verify_injection_safety were near-vacuous
too — one asserted properties of a format! literal that hold even if the escaper
is the identity function, the other checked a toy re-implementation local to the
file against a toy oracle. Both now call the real escapers.

A harness that cannot fail is worse than a missing one: it consumes the
verification budget and reports success.

verify-kani now separates two outcomes that are not the same thing:

  BUILD failure  -> HARD FAIL. This is the GH-212 defect itself. Caught by
                    `--only-codegen` in ~106s without invoking the solver.
  No convergence -> UNPROVEN, reported, non-fatal. Failing on it would make
                    `make verify` permanently red, which is how a gate teaches
                    people to bypass it.

Verified differentially:
  fixed             -> --only-codegen: 0 errors, 106s; target exits 0
  E0277 reintroduced -> target exits 1 with the diagnostic above

MEASURED, and reported rather than papered over: none of the three surviving
harnesses converge. CBMC spends its time in alloc::raw_vec / Layout /
LayoutError — Rust's ALLOCATOR — because every harness calls an escaper
returning a heap String. Bounds of 8, 4 and 2 characters all time out, so string
LENGTH was never the bottleneck. Proving these needs the escapers refactored
onto caller-provided &mut [u8] buffers so no allocation is reachable from a
harness; filed separately. What this commit buys is that the harnesses now
BUILD, which is the precondition for any of that.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@noahgift
noahgift force-pushed the fix/kani-cfg-build branch from 42bef85 to aea4c9c Compare August 11, 2026 19:25
@noahgift
noahgift merged commit b7390db into main Aug 11, 2026
8 checks passed
@noahgift
noahgift deleted the fix/kani-cfg-build branch August 11, 2026 19:56
noahgift added a commit that referenced this pull request Aug 12, 2026
Ships the user-facing fixes accumulated since 6.66.2, none of which
reach anyone until this is published:

  - five lint false positives that were driving users to disable
    `bashrs lint` (#219, GH-217, GH-209)
  - CLI stack overflow from an oversized clap frame (#216, #215)
  - RUSTSEC-2026-0204, crossbeam-epoch 0.9.20 (#210)

plus internal repairs: kani harnesses compile under cfg(kani) again
(#221), bashrs-oracle's test module compiles and the workspace is
actually tested (#223), and a workflow template stopped being run as a
workflow (#222).

Cargo.lock regenerated in the same commit -- forjar's 1.12.4 release
tripped its lockfile-preflight by bumping Cargo.toml alone.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

crate does not compile under cfg(kani): 19 errors in verifier/ and formal/ kani_harnesses.rs make all Kani proofs unrunnable

1 participant