Skip to content

fix(gateway,code): the remaining findings of the independent review of 2.0.0 (D1, D2, D3, A1, A2, N1 to N5) - #473

Merged
HarperZ9 merged 9 commits into
mainfrom
fix/security-round3b-20261114
Nov 14, 2026
Merged

HarperZ9 merged 9 commits into
mainfrom
fix/security-round3b-20261114

Conversation

@HarperZ9

@HarperZ9 HarperZ9 commented Nov 14, 2026 •

Copy link
Copy Markdown
Owner

The remaining findings of the independent review of the 2.0.0 fixes (B1 merged as #472). Each fix has a test that fails on the unfixed tree.

Finding What changed Test
D2 A ledger whose writes fail after start (a full disk) is marked failed at the first failed append. Later model calls answer 503 ledger_unavailable before any upstream call. When writes resume the gateway appends a ledger_gap entry and verify passes; a line left cut short is recorded as a break in a new segment. New module harness/gateway_ledger_health.py. tests/test_gateway_ledger_write_failure.py
A1 verify reads the head record before the segments, so a busy, intact ledger verifies. tests/test_gateway_ledger_live_verify.py, forced order and a writer thread
N2 Files named like a segment that are not one are ignored and named by verify and the start line. same file
A2 The second read of a key's row and its admission are one step under one lock, which a call that ends takes too. tests/test_gateway_admission_lock.py, forced pause, both orders
D1 An unfinished stream's prompt and received text are charged at least a quarter of their characters, rounded up, marked estimated. same file, real sockets
D3 The fallback to the text protocol starts only on an error that says tools or function calling are unsupported. The session prints one notice and records native_tools_fallback. New module harness/endpoints_tools_fallback.py. tests/test_code_session_native_fallback_trigger.py
N1 The Lean scan resolves aliases and covers more spawn APIs and an env= that is not exactly lean_env(...). tests/lean_spawn_scan.py states what it cannot see. tests/test_lean_child_env.py
N3 One deadline for the request line and headers and one for a request body, on every route, before authentication; two seconds for the drain after a refusal. tests/test_gateway_round3_small_fixes.py
N4 KNOWN-LIMITS names /v1/embeddings among the routes outside the ledger. docs
N5 gateway-ledger show cuts a field at 80 characters; --full prints it whole. tests/test_gateway_round3_small_fixes.py

What is stated, not fixed

  • Every model interface appends its ledger entry after the provider answered, because the entry holds the reply's hash. The one call that meets a write failure has reached the provider. A plain reply is held back (500); a stream's text has left.
  • D3: the Claude API errors page, the OpenAI error-codes guide and the Gemini troubleshooting page (read 2026-11-14) give no exact sentence for "this model cannot take tools". The accepted phrases come from two documented Claude API forms and from error texts quoted in public issue reports, which were not checked against a live host. KNOWN-LIMITS has the row, and the module and the test header cite the pages.
  • A2: two calls in flight at the same time keep the documented margin (each reserves its output limit and a quarter of its request bytes). What is closed is admission on a total that another call finished and changed during a pause. The reviewer's forced-interleave probe still prints two served calls and 920 tokens against this branch: its second request now waits for the first one's admission and the two then run together, and the stub's reported input (451 tokens) is larger than the reservation. The test here uses requests whose reservation equals the limit, where the second is refused 429.
  • D2: a gateway that stops while its ledger is still failing, with the disk still full, leaves no ledger_gap entry; the log has the failure.
  • N3: no cap on open connections.

Found by a second read of this branch

Before the merge another agent read this branch with the code and the intended properties in hand, not my conclusions. It is not an independent review: I started it and I judged its findings. Six of them held up against the code and are fixed here, each with a test that fails when the change is reverted:

  • After a refusal that did not read the body (a 401, for one) the gateway discards what the client still sends. That was a buffered read of the promised length, so a client with no credential that sent a byte at a time held its thread for as long as it kept sending. The connection's reader now gives one read(n) a budget: two seconds in that drain, four read timeouts for a body. The body deadline therefore covers every route. gateway_body_drain.py is pinned and unchanged. Checked with plain sockets on Windows and on Linux (WSL): closed at 2.1 and 2.0 s.
  • Whether a reply is held to the native rule was decided when the reply arrived, from the process-wide list of models that moved to the text protocol. Sessions in one process share that list, so a model moved by one session while another's reply was on its way made that reply a text-protocol reply, where a prose TOOL line runs. The decision is now what the request carried.
  • The segment name pattern used \d, which matches digits of other scripts. ASCII digits only.
  • A head record that could not be rewritten after an entry was written did not mark the ledger failed. It does now.
  • The fallback wording matched errors about tool_choice, parallel tool calls and tools with images. It is tighter.
  • The loop runs the calls as parsed from the record, so the executor shares no argument object with the remembered native reply.

Not changed: a resume whose new segment cannot be removed after a failed start stays failed until a restart (it fails closed); a failed debit for an unfinished stream does not suspend the key (as before this branch); the admission lock is held during the key store read.

Corrected wording

CHANGELOG: "could not open or write its ledger" (the earlier fix covered open only), the sentence on the second read of verify, and "the other ways to start a child". KNOWN-LIMITS: the word-estimate row, the read-timeout row, the first row ("a key holder" was narrower than the fact), and the intro, which now says the reviewer's probes were run again and that these fixes have had no further independent review.

Evidence

  • On the unfixed tree (release head 7b7803c6f) the new test files, plus the new scan forms run through the old scan: 81 failed, 6 passed. The six that pass there are controls.
  • Each change to the gateway and the session was reverted alone in a scratch copy of this branch (20 reversions; B1's three parts and the five follow-ups above included) and its tests then failed. The first pass found one gap: the A2 test of that moment did not notice two separate locks. The test for a call that ends inside the pause closes it, and that reversion fails now.
  • Deadlines (N3) were also checked with plain sockets on Windows and on Linux (WSL): a client that drips header lines, or bytes inside one header line, is closed at about 0.6 s with a 0.5 s timeout, and a dripped body gets 408 at 2.0 s.
  • The new gate files pass against a wheel built from this branch, run from a copy of tests/ with no harness/ beside it.
  • One existing control changes with D1: the output estimate of its unfinished stream is 9 (a quarter of the received characters) where the word count is 5.
  • Constraints kept: gateway_ledger.py 295 lines, no file over 300, no function over 50; gateway.py, gateway_body_drain.py and local_loop.py untouched in this PR. An unimportable gateway_ledger_health yields zero upstream requests (tested).

Not run: any hosted provider, a real full disk (the failure is simulated at the segment's file), macOS. Local runs were targeted slices on Windows; the rest is left to CI.

🤖 Generated with Claude Code

HarperZ9 and others added 9 commits November 14, 2026 13:42
…f 2.0.0

D2. A ledger that fails after start stops model calls. The first append that
fails marks the ledger failed and closes the segment's file
(harness/gateway_ledger_health.py). `for_home` then returns None, which every
model interface already answers with 503 before any upstream call. At most
once a second the gateway reads the segment back: when it verifies and
begins with the entries in memory it appends again, starting with a
`ledger_gap` entry; when the tail is cut short a new segment records the
break. The segment is opened unbuffered so no failed line waits in a buffer.
Every model interface appends after the upstream answered, so the call that
meets the failure has reached the provider; the docs say so.

A1. `gateway_ledger.check` reads the head record before the segments, so
entries appended during a check are not reported as removed. `read_segment`
named the line before a cut line and dropped that intact entry; it names the
cut line.

N2. Only names that match segment-NNNNNN.jsonl exactly are segments. Other
`segment-*` files are named as ignored by `verify` and the start line.

A2. `gateway_key_gate.enter` reads the key's row again and takes the slot
under one lock, which `release` takes too.

D1. An unfinished stream's prompt and received text are each charged the
larger of the word count and a quarter of the characters, rounded up.

D3. The fallback from native calling to the text protocol starts only on an
error that says tools or function calling are unsupported
(harness/endpoints_tools_fallback.py), never on another 400 that mentions
them. The session records a `native_tools_fallback` entry and prints one
notice. The provider documentation read on 2026-11-14 gives no exact sentence
for this refusal, so the wording is a stated heuristic (KNOWN-LIMITS).

N1. The Lean scan resolves import aliases and reports more spawn APIs and an
env= that is not exactly lean_env(...) (tests/lean_spawn_scan.py, which also
lists what it cannot see).

N3. The request line and headers have one deadline, on every route, before
authentication; the body of a model call has one (four read timeouts).

N4, N5. KNOWN-LIMITS names /v1/embeddings among the routes outside the
ledger. `gateway-ledger show` cuts a field at 80 characters; `--full`.

CHANGELOG: the three sentences the review flagged are corrected ("cannot
open or write", the verify retry, "the other ways to start a child").

Tests (release_gate): tests/test_gateway_ledger_write_failure.py,
tests/test_gateway_round3_small_fixes.py,
tests/test_code_session_native_fallback_trigger.py, tests/test_lean_child_env.py.
On the unfixed tree 81 of 87 of these cases fail. One existing control
changes: the output estimate of its unfinished stream is 9 (a quarter of the
received characters), where the word count is 5.

Not run: any hosted provider, a real full disk, macOS.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…cord the gap

Seen when the reviewer's probe was run against the branch: the gateway
stopped inside the one-second retry interval, so the ledger held no
`ledger_gap` entry for the append that had failed. `close` now calls
`gateway_ledger_health.settle`, which resumes once if the disk allows it and
logs what stays unrecorded if it does not.

The ledger tests are two files now, each under 300 lines:
tests/test_gateway_ledger_write_failure.py (D2) and
tests/test_gateway_ledger_live_verify.py (A1, N2, and the cut-line count).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…no ledger_gap entry

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…error does not match

The documented Claude API error for forced tool use (`tool_choice: type "tool"
and "any" are not supported for this model`) matched the second phrase. The
phrase now lists its subjects (tools, tool use, tool calling, function calling,
the functions parameter) and the test table holds four documented errors that
must not start the fallback.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The limits page said the gateway resumes within a second. It tries the ledger
again when a call comes in, at most once a second, and once more when it
stops.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…de the pause

Reverting one fix at a time in a scratch copy showed that the A2 test did not
notice when the row read and the slot release used two locks. It covered a
call that starts while another request is paused between its row read and
its admission, and not a call that ends in that pause. The second order is
tested now: the call that ends keeps its slot until the paused request has
been admitted or refused. Both reversions (two locks, no lock) make
tests/test_gateway_admission_lock.py fail. The A2 tests moved to that file.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The admission-lock test let its stub provider answer after 0.6 seconds and
needed the second request to have read the key's row by then. The provider
now answers when the test says so, after the second request has read the row.
The header-drip test allowed two seconds for a close that comes at half a
second; it allows four, and drips for eight. Before the deadline existed the
client was still connected when the dripping ended.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A read-only pass over this branch by another agent, given the code and the
intended properties and not my conclusions, reported these. Each was checked
against the code before it was changed, and each has a test that fails when
the change is reverted in a scratch copy.

- gateway: after a refusal that did not read the body (a 401, for one) the
  drain read the promised length with a buffered read(n), which waits for
  all n bytes. A client with no credential that sent a byte at a time held
  its thread for as long as it kept sending. The connection's reader now
  gives one read(n) a budget: the socket's own timeout while one is set (the
  drain sets two seconds), else four read timeouts. This also puts the body
  deadline on the routes that read their bodies in gateway.py.
  gateway_body_drain.py is pinned and unchanged.
- code: whether a reply is held to the native rule was decided when the
  reply arrived, from the process-wide list of models that moved to the text
  protocol. Sessions in one process share that list, so a model moved by
  one session while another's reply was on its way made that reply a
  text-protocol reply, where a prose TOOL line runs. The decision is now
  what the request carried.
- gateway ledger: the segment name pattern used \d, which matches digits of
  other scripts; such a file sorted last and was taken for the newest
  segment. ASCII digits only.
- gateway ledger: a head record that could not be rewritten after an entry
  was written did not mark the ledger failed, so every later call reached
  the provider and was answered 500. It marks it now, and a resume reopens
  the head record.
- code: the fallback wording matched "does not support tool_choice",
  "parallel tool calls are not supported", "tools are not supported with
  images" and any "tool choice requires". Those are about one way of using
  tools. The phrases are tighter and such errors are excluded.
- code: the loop now runs the calls as parsed from the record (equal to the
  provider's objects by the check before it), so the executor shares no
  argument object with the remembered native reply.

Not changed, and written down in the PR: a resume whose new segment cannot
be removed after a failed start stays failed until a restart; a failed debit
for an unfinished stream does not suspend the key; the admission lock is
held during the key store read.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@HarperZ9
HarperZ9 merged commit 716f549 into main Nov 14, 2026
87 checks passed
HarperZ9 added a commit that referenced this pull request Nov 14, 2026
Main holds the third review's fixes (#472, #473). CHANGELOG.md conflicted:
main's Unreleased section against this branch's reordered 2.0.0 section. The
release side is kept. The ten new bullets are carried with their pull request
numbers, the three #470 sentences that main corrected are carried over, and
the entries under "What changed by default" are updated.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
HarperZ9 added a commit that referenced this pull request Nov 14, 2026
The review paragraph names the third review: one blocker (text inside a native
tool call's arguments could run as a different call) and ten other items,
fixed by #472 and #473. It says the reviewer's probe scripts were run again
against the fixed code, and that #472 and #473 have had no further
independent review.

"What the security review changed" gains: the 503 for a ledger that stops
taking writes while the gateway runs, the quarter-of-characters estimate for
an unfinished stream, the head and body deadlines, the argument rule for
native tool calls, and the notice when a model moves to the text protocol.
Limits follow docs/KNOWN-LIMITS-2.0.0.md on main: connections are bounded by
deadlines and not by a cap, /v1/embeddings is outside the ledger, the call
that meets a ledger write failure has reached the provider, and the fallback
rests on the wording of a provider's error.

The changelog gains one entry for #471 (the Telos Toolkit plugin folder),
which merged to main between #472 and #473 with no changelog entry of its
own. It is in the repository tree and in no release asset; the entry says so.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant