Skip to content

tests: land the review round's reproductions, with the refuted claims turned into what is true - #197

Merged
MDA2AV merged 1 commit into
mainfrom
tests/consolidate-review-round
Aug 19, 2026
Merged

tests: land the review round's reproductions, with the refuted claims turned into what is true#197
MDA2AV merged 1 commit into
mainfrom
tests/consolidate-review-round

Conversation

@MDA2AV

@MDA2AV MDA2AV commented Aug 19, 2026

Copy link
Copy Markdown
Owner

A 24-agent review round produced 20 test files that were never committed. 11 of their 31 reproductions had been refuted by an adversarial pass, and shipping those would have put two mutually exclusive assertions in one suite — one asserts the opposite of a test that passes today.

Leaving all 20 out was the wrong trade, because the refuted claims were the smaller half. Those files also hold 77 passing tests: controls, matrices and ordinary coverage written alongside the claims.

             main    this branch
passing       332        419
pending         4         20

The refuted claims are converted, not deleted

Each drove a real scenario that was already wired up. What was wrong was the conclusion, so each now pins the behaviour that was verified correct and records why the claim did not hold.

claim what it pins now
resume a ticket issued before a rotation is refused after it, and the client is still served
truncation the pipe reports both endings alike; the session is what tells them apart
identity a CN the decoder refuses is not named, even where the rendered subject cannot tell it apart
mtls/quic (×2) a DN or CN too long to record is reported as no name, never a prefix
quic/fault a throwing HandshakeCompleted is logged and the connection keeps serving
http3 a body cut mid-frame kills the connection with H3_FRAME_ERROR; the reader signals an ordinary end
parity omitting the host table is refused on QUIC and applied on TCP — a known divergence
lifetime deleted; the file already had a passing test for the only true part

Three of those are worth stating in full, because the refutation is the interesting part.

Resumption. Ticket keys are per SSL_CTX and ReplaceCertificates builds new contexts, so a rotation retires them — which is what nginx, Apache and HAProxy do on reload unless an explicit ticket-key file says otherwise. The test immediately below it depends on that: anchors given as a path are re-read on every rotation, so a ticket that outlived one would carry the old verify verdict past the new anchors. The two could not both pass.

Truncation. TlsDecryptingPipeReader documents the rule it was accused of breaking: close_notify is a clean end, a closed snapshot without one is the peer vanishing, and the difference is left to the caller through TlsSession.Closed. Faulting the cut one would fault every client that merely disposes its SslStream without calling ShutdownAsync — the ordinary polite close. A TLS fault is still kept and reported.

Parity. Neither stack is wrong on its own. TCP's behaviour is documented on the parameter and pinned by a passing test, so raising it to QUIC's stricter contract is a deliberate breaking change that deletes that test — a decision, not a fix. The test now records that the divergence exists, so whichever way it is resolved, nobody resolves it by accident.

Two were better answered in code

TlsService.Start now range-checks MinProtocolVersion and HandshakeTimeoutMs. Both previously resolved to something plausible instead of failing: an undefined version reached the TLS 1.2 floor through a not-Tls13 ternary, and a negative timeout disabled the handshake sweep entirely, because both readers guard on > 0. That silently removes the only bound on a peer that connects and then says nothing, which is the part of a TLS server reachable before anything is authenticated. No config binder validates an enum — Enum.Parse<TlsProtocolVersion>("3") succeeds — so neither value needed a cast to arrive.

Two identity doc comments were corrected to match the code: null from PeerSubject and PeerCommonName is a refusal, not "the peer offered none".

What stays open

The 19 surviving reproductions remain runner.Pending. They report PEND while they still fail and fail the run the moment one starts passing, which is how three defects in this round were caught as fixed rather than being quietly promoted.

All six suites: 419 passed, 0 failed, 20 pending.

… turned into what is true

The 24-agent review round produced 20 test files that were never committed, because 11 of their 31
reproductions had been refuted by an adversarial pass and shipping them would have put two mutually
exclusive assertions in the same suite - one of them asserts the opposite of a test that passes
today. Leaving all 20 out cost the 77 PASSING tests in them, which is the larger number: controls,
matrices and ordinary coverage the agents wrote alongside their claims.

So the refuted 11 are converted rather than deleted. Each drove a real scenario that was already
wired up; what was wrong was the conclusion. Each now pins the behaviour that was verified correct
and records why the claim did not hold:

  resume       a ticket issued before a rotation is REFUSED after it, and the client still served.
               Ticket keys are per SSL_CTX; nginx, Apache and HAProxy retire them on reload too, and
               the test below it depends on that - a ticket outliving a rotation would carry the old
               verify verdict past the new anchors.
  truncation   the pipe reports both endings alike and the SESSION tells them apart. Faulting the
               cut one would fault every client that merely disposes its SslStream.
  identity     a CN the decoder refuses is not named, even where the rendered subject cannot tell it
               apart - PeerSubject is documented for people, PeerCommonName is what authorizes.
  mtls/quic    a DN or CN too long to record is reported as NO name, never a prefix. Fails closed on
               purpose: a prefix can belong to someone else.
  quic/fault   a throwing HandshakeCompleted is logged and the connection keeps serving, which is
               this runtime's policy for user-code faults everywhere else.
  http3        a body cut mid-frame kills the connection with H3_FRAME_ERROR and the reader signals
               an ordinary end - the contract all three body readers document identically.
  parity       omitting the host table is refused on QUIC and applied on TCP, pinned as a KNOWN
               DIVERGENCE. Neither side is a bug; TCP's is documented and pinned green, so resolving
               it is a deliberate breaking change rather than a fix, and this makes sure nobody
               resolves it by accident.
  lifetime     deleted - the file already had a passing test for the only true part.

Two were better answered by fixing the code than by rewriting the test, which both the refuting
agent and the commit auditor independently recommended: TlsService.Start now range-checks
MinProtocolVersion and HandshakeTimeoutMs. Both resolved to something plausible rather than
failing - an undefined version through a not-Tls13 ternary to the TLS 1.2 floor, and a negative
timeout to no handshake sweep at all, silently removing the only bound on a peer that connects and
says nothing. No config binder validates an enum, so neither needed a cast to arrive.

Two identity doc comments were corrected to match: null from PeerSubject/PeerCommonName is a
refusal, not "the peer offered none".

The 19 surviving reproductions stay as Pending - they report PEND and fail the run if they start
passing. 419 passed, 0 failed, 20 pending, across all six suites.
@MDA2AV
MDA2AV merged commit 82ba4ef into main Aug 19, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant