fix(gateway,code): the remaining findings of the independent review of 2.0.0 (D1, D2, D3, A1, A2, N1 to N5)

CHANGELOG.md

@@ -215,14 +215,23 @@ Fixes
 - Fixed: Lean child processes inherited the whole environment of the process that started them, so a candidate proof that evaluates `IO.getEnv` could print a provider key or another variable into `kernel_output`. Every Lean spawn (compile, binding check, replay, second kernel, export check, version probes) now gets an allow-listed environment from `harness/lean_env.py`. See `docs/PROOF-AND-FORMATS.md`, "The environment a Lean process gets". If you relied on a custom variable reaching Lean, it no longer does; `LEAN_PATH`, `LEAN_SYSROOT`, `ELAN_HOME`, `ELAN_TOOLCHAIN`, `LEAN_NUM_THREADS` and `LEAN_STACK_SIZE_KB` still do.
 - Fixed: in a coding session with native function calling, a `TOOL name {json}` line in a reply's own words ran as a call, so a line the model quoted from a file, a tool's output or a web page could run. On a native turn only the provider's tool-call objects run now, in every permission mode; such a line is recorded with `> ` in front of it and leaves no `tool_call` entry. A reply that writes a TOOL line and makes no native call gets one message telling the model to use the function-calling interface (ledger kind `native_tools_notice`), at most once for each request. The text protocol is unchanged where native calling is off or not offered. If you relied on TOOL lines with a native-capable host, set `FLYWHEEL_NATIVE_TOOLS=0`. See `docs/CODE-SESSION.md`, "How tool calls travel".
 - Not fixed: when one native reply holds several calls and a tool's output contains the next call's result header, the results are divided at the wrong place on the way back to the provider. `docs/KNOWN-LIMITS-2.0.0.md` keeps the row.
-- Fixed after a second review of the fixes above: a gateway that could not open or write its ledger still reached the provider. A streamed reply was delivered in full with no ledger entry and no charge, a plain call was billed and then answered 500, and the chat path was served with no entry. The gateway now answers 503 with `retry-after: 5` before any upstream call on `/v1/messages`, `/v1/messages/count_tokens`, `/v1/responses`, `/v1/chat/completions` with a provider endpoint and `/api/route`, for each cause: a second gateway on the same home, another process holding the writer lock, and a ledger folder that cannot be written. The gateway opens the ledger at start and prints whether it did.
+- Fixed after a second review of the fixes above: a gateway that could not open its ledger still reached the provider. A streamed reply was delivered in full with no ledger entry and no charge, a plain call was billed and then answered 500, and the chat path was served with no entry. The gateway now answers 503 with `retry-after: 5` before any upstream call on `/v1/messages`, `/v1/messages/count_tokens`, `/v1/responses`, `/v1/chat/completions` with a provider endpoint and `/api/route`, for each cause: a second gateway on the same home, another process holding the writer lock, and a ledger folder that cannot be written. The gateway opens the ledger at start and prints whether it did. This fix covered a ledger that cannot be opened. A ledger whose writes fail after it opened is covered by the third-review bullets below.
 - Fixed: a client that reset its connection while the upstream was still opening a stream left no ledger entry and was charged nothing, although the provider was called. The response now has an explicit close hook that runs whether or not the stream was read, so the call is recorded as `client_disconnected` with `"abandoned": true` and charged for the prompt.
 - Fixed: a key's limits were checked against the row read when the request headers arrived, so requests that sent headers early and bodies later were all served past a limit. The row is read again after the body, immediately before admission; a key revoked or suspended in between is refused. New: a read timeout of 30 seconds on the requests of every gateway connection (`FLYWHEEL_GATEWAY_READ_TIMEOUT`; `0` switches it off). A body that stops arriving on a model interface gets 408.
-- Fixed: `flywheel gateway-ledger verify` passed when the head record was there and its segment file had been deleted. It now fails, and a gateway that starts on such a home records the break. `verify` reads once more before it reports a problem, so an append during the first read is not reported as a removed entry.
+- Fixed: `flywheel gateway-ledger verify` passed when the head record was there and its segment file had been deleted. It now fails, and a gateway that starts on such a home records the break. `verify` reads once more before it reports a problem. That second read did not settle a ledger that a gateway keeps appending to; the third-review bullets below have the fix.
 - Fixed: an unfinished stream was charged for received text only. It is now charged the provider's output count when one had arrived, and tool-call arguments and thinking count toward the estimate.
-- Changed: no key can be named `owner`, the ledger's name for the gateway token's calls. `flywheel gateway-ledger show` escapes control characters in recorded strings. The static check of the Lean modules also covers `subprocess.call`, `check_call`, `os.system` and the other ways to start a child.
+- Changed: no key can be named `owner`, the ledger's name for the gateway token's calls. `flywheel gateway-ledger show` escapes control characters in recorded strings. The static check of the Lean modules also covers `subprocess.call`, `check_call`, `getoutput`, `getstatusoutput`, `os.system`, `os.popen`, `os.spawn*` and `os.exec*`, matched by the name as written.
 - Tested over real loopback sockets with a stub upstream on Windows and, in CI, Linux: `tests/test_gateway_ledger_unavailable.py`, `tests/test_gateway_abandoned_and_delayed.py`, `tests/test_gateway_confirmation_small_fixes.py`. These and `tests/test_gateway_review_small_fixes.py` and `tests/test_lean_child_env.py` now run against the built wheel before a release. Not run: any hosted provider. Not measured again: the gateway overhead figures, after the second read of the key row (0.16 ms on the test machine).
 - Fixed after an independent review of those fixes (it blocked the release): in a coding session with native function calling, text inside a tool call's arguments could replace the model's call with a different one. A native call is recorded as a `TOOL name {json}` line, and the session's line reader also cuts at U+0085, U+2028 and U+2029, which the record left raw. An argument holding one of them lost the model's call, and a quoted `TOOL run {...}` fragment after the cut ran in `auto` mode or was put to the user in `ask` and `edit`. A call is now one line whatever its arguments hold, and the tool receives the argument unchanged. The session also compares its record with the provider's tool-call objects (count, names, arguments) and runs those objects only; the repair step for malformed TOOL lines never runs on a native turn; a record that does not read back as the objects runs nothing and the model is told once. A side effect is gone too: a native call with one of these characters in an argument used to be dropped without a word. Tested with OpenAI, Anthropic and Gemini replies, plain and streamed, in all four permission modes, and with random arguments drawn from every line and separator character: `tests/test_code_session_native_argument_lines.py`, which runs against the built wheel before a release. Not run: a live model. See `docs/CODE-SESSION.md`, "How tool calls travel".
+- Fixed after the same review: a ledger that opened at start and then stopped taking writes (a full disk) did not stop model calls. Every call still reached the provider for as long as the failure lasted, and when writes came back the segment held repeated sequence numbers and `verify` failed. The first failed append now marks the ledger failed (a head record that cannot be rewritten counts as one), and later calls answer 503 with `ledger_unavailable` before any provider is contacted. When writes resume the gateway reads the segment back and goes on: with a `ledger_gap` entry when the file is whole, so `verify` passes, or with a new segment that records the break when a line was left cut short. One call cannot be refused: every model interface appends after the provider answered, so the call that meets the failure was made. A plain reply is held back and answered 500; a stream's text has left.
+- Fixed: `flywheel gateway-ledger verify` reported "entries were removed" on an intact ledger while a gateway appended to it, on every run for a segment of some thousand entries. It reads the head record before the segments now. It also named the line before a line that was cut short and did not count that entry; it names the cut line.
+- Fixed: a file in the ledger folder named like a segment (`segment-backup.jsonl`, `segment-000001 - Copy.jsonl`) stopped the gateway at start with an uncaught error or failed `verify` on an intact ledger. Only exact segment names, with ASCII digits, are read; `verify` and the start line name the others as ignored.
+- Fixed: a request that was paused between the second read of its key's row and its admission could be admitted on a total that another call had changed in the pause. The read and the admission are one step under one lock now. Calls that run at the same time keep the documented margin.
+- Fixed: an unfinished stream on an OpenAI-format upstream was charged a word count of the prompt, so 80,000 characters with no space cost 6 tokens. An unfinished stream's prompt and received text are each charged at least a quarter of their characters, rounded up, and the entry stays marked `estimated`. `docs/KNOWN-LIMITS-2.0.0.md` said the word count applied to local and compatible hosts only; the row is corrected.
+- Changed: a model moves from native function calling to the text protocol only when its host's error says tools or function calling are unsupported. Any 400 that mentioned "tool" or "function" used to do it, a context-length error included, and a quoted TOOL line in the next reply then ran. The session prints one notice and records a `native_tools_fallback` entry when a model moves. A reply is held to the native rule by what its own request carried, so a model that another session moves meanwhile does not change a reply already on its way. No provider documents the exact sentence, so the accepted wording is a stated heuristic; `docs/KNOWN-LIMITS-2.0.0.md` has the row.
+- New: the request line and headers of a gateway request must arrive within one read timeout and a request body within four, on every route, with or without a credential. A late body gets 408 on a model interface; other routes close the connection. After a refusal that did not read the body (a 401, for one), what the client still sends is discarded for two seconds at most. A client that dripped header lines, or bytes after a refusal, used to hold a thread for as long as it liked. There is still no cap on open connections.
+- Changed: `flywheel gateway-ledger show` cuts a field longer than 80 characters and says how much it left out; `--full` prints it whole. The static check of the Lean modules resolves import aliases and also reports `asyncio.create_subprocess_*`, `os.posix_spawn*`, `os.startfile`, `os.fork`, `pty`, `multiprocessing`, `getattr` on those modules and an `env=` that is not exactly `lean_env(...)`; `tests/lean_spawn_scan.py` lists what it cannot see. `docs/KNOWN-LIMITS-2.0.0.md` names `/v1/embeddings` among the routes outside the ledger.
+- Tested over real loopback sockets with a stub upstream, and with recorded provider replies: `tests/test_gateway_ledger_write_failure.py`, `tests/test_gateway_ledger_live_verify.py`, `tests/test_gateway_admission_lock.py`, `tests/test_gateway_round3_small_fixes.py`, `tests/test_code_session_native_fallback_trigger.py`, which run against the built wheel before a release with `tests/test_lean_child_env.py`. Each change to the gateway and the session was also reverted alone in a scratch copy, and its tests then failed. The reviewer's probe scripts were run again against the fixed code. These last fixes have had no further independent review. Not run: any hosted provider, a real full disk (the failure is simulated at the segment's file), macOS.
 
 ## 1.9.0 (2026-10-15)
 

docs/CODE-SESSION.md

@@ -531,10 +531,18 @@ everywhere. The text protocol is used whenever native calling is not:
 
 - the switch is off, or the host is not on the list above and the switch is
   not `1`;
-- the host answers a request that carries tool schemas with an error that
-  names tools or functions. The session logs one warning, sends the request
-  again in the text protocol, and stays in it for that model until the
-  session ends;
+- the host answers a request that carries tool schemas with a 400, 404 or
+  422 that says the model cannot take tools or function calling ("does not
+  support tools", "function calling is not enabled", an OpenAI-shape error
+  with `param` `tools` and the code `unsupported_parameter`). The session
+  sends the request again in the text protocol and stays in it for that
+  model until the program ends. It prints one line that says so
+  (`Notice: ...`) and records it as a `native_tools_fallback` entry, because
+  from then on a `TOOL name {json}` line in that model's replies is a call.
+  Any other error, such as a context-length error that mentions functions,
+  fails the request and changes nothing. No provider documents the exact
+  sentence for this refusal, so a host that words it differently is not
+  recognized: set `FLYWHEEL_NATIVE_TOOLS=0` for it;
 - an earlier turn cannot be matched to the native reply it came from, which
   happens after `--continue`, `--resume` or a compaction. That turn is sent
   as text; new turns are native again.