Commit fd6a3e6
authored
http2: the write queue, a pure-C# client, streaming both ways, and nghttp2 retired (#176)
* feat(http2): streamed response bodies - WIP, do not merge yet
The response half only, as agreed: an h2 handler can now push body bytes as it
produces them, each flush becoming a DATA frame, instead of returning a
finished Http2Response. That is what unblocks a streaming IResponseContent,
and what makes an endless response expressible at all.
The part HTTP/3 did not have is flow control. Every stream here rides one TCP
connection and a write needs credit in BOTH the stream's window and the
connection's, so FlushAsync waits for a WINDOW_UPDATE rather than resetting
the stream the way the buffered path does - a whole body was always in hand
there, so running out of credit could only mean the peer had stopped reading.
Verified correct: 8192 bytes exact on an 8 x 1KiB response, and 1 MiB
delivered whole - sixteen times the default 65535 window, which is what proves
the wait works rather than the stream dying at the first window.
One leak found and fixed on the way. The streamed dispatch skips the buffered
path's tail, and that tail is what retires the stream, so every response was
leaking its PendingRequest arena: 20 GB of RSS over eight seconds of load.
Retiring the stream in the writer's finally brings that to 97 MB, and doubles
throughput as a side effect.
NOT READY. Streamed still runs 93990 req/s against the buffered path's
1773623 on the same 8 KiB - a 19x gap I have not diagnosed. Eight flushes per
response against one is part of it, but not obviously all of it, and shipping
a streaming path that slow would mislead anyone who reached for it. No
playground pane, no registry entry, no version bump until that is understood.
Unit 29, E2E 46, Http 35 pass.
* perf(http2): coalesce streamed writes into the pass flush - 6.1x
A streamed response measured the same whether it carried 2 bytes or 8 KiB -
408k against 355k req/s - and that flatness is the shape of a syscall, not of
work. Each response was forcing its own transport write, where the buffered
path spreads ONE write across every stream in flight on the connection. That
alone was the whole gap, and at 2 bytes it was 12.8x.
So a streamed writer no longer writes for itself while the read loop still
owes a flush: it stages, and the pass carries it out alongside every other
response. Coalescing is the reason buffered h2 is fast and there is no reason
a streamed response cannot share it.
It has to stay bounded, though not for memory. A producer that loops without
awaiting anything of its own has no yield point except that write, so skipping
it unconditionally spins the reactor and the response never moves at all -
which is exactly how two earlier attempts at this broke the endless case, both
times silently, because a benchmark only measures responses that END. Past
16 KiB staged the write happens for real and hands the thread back. A paced
producer never reaches the limit: it parks, the pass flush takes its chunk out
immediately, and it resumes outside the pass where the write is unconditional.
before after buffered
2 bytes 408,483 3,873,462 5,235,162
1 x 8192 355,209 585,830 1,801,105
8 x 1024 93,006 569,832 1,801,105
Chunk count stopped mattering - 8x1024 and 1x8192 now sit within 3% where they
differed 3.8x, because writes follow bytes rather than how often the handler
asks for a flush.
What remains is per-byte, not per-write: raising the limit to 128 KiB changed
nothing, and streamed still falls off with body size faster than buffered.
That points at the staging copy - the handler fills an ArrayPool block that is
then copied into the pipe, where the buffered path writes the body once.
Framing the DATA header directly into the pipe's span would remove it.
Verified unchanged: 8192 exact, 1 MiB through a 65535 window, endless /feed
still trickling, RSS flat at 98 MB. Unit 29, E2E 46, Http 35 pass.
* fix(http2): a slow handler no longer blocks every other stream
DispatchReadyAsync awaited each handler in turn, so one request that parked -
a database, an upstream, a disk - held up every other stream on that TCP
connection, including responses already produced and staged with nowhere to
go. Two requests on one connection, one sleeping a second:
/slow /fast
before 1.02s 1.02s <- waited for /slow
after 1.02s 20.99ms
That is the whole point of multiplexing, and h2 is the worst place to lose it:
QUIC streams are independent, so an h3 handler that parks inconveniences only
itself, but on h2 everything shares one connection and one dispatch loop. Both
h3 modules already dispatch this way, buffered and streamed alike. h2 was the
outlier.
A handler that answers synchronously - nearly all of them - stays inline, so
its response is still staged in time for the pass flush and still leaves with
every other one in a single write, and there is no Task to allocate. Only a
handler that actually parks is detached, and it writes its own bytes when it
finishes, since the pass flush has gone by.
Detaching means nothing is awaiting the tail, so the tail has to do the work
the loop used to do in its finally: retire the request, and flush. Skipping
exactly that is what leaked 20 GB in the streamed path. An escaping exception
would also vanish silently and leave the peer waiting on a stream that never
comes, so it is caught, logged and answered with a 500 - the buffered path
previously let it kill the connection.
No cost to the fast path: 2 bytes 5353304 req/s (was 5235162), 8 KiB 1783354
(was 1801105) - both inside noise. Unit 29, E2E 46, Http 35 pass.
nghttp2 has the same defect at RunBuffered.cs:94 and is deliberately left
alone; the managed stack is where the effort goes.
* fix(http2): async handlers can write again - staging and flushing take turns
Non-blocking dispatch made handlers complete outside the pass, and the write
path could not accept them: a PipeWriter permits no Write while a flush is
outstanding, so every asynchronous handler faulted with "Cannot write while
flush is in progress" and served nothing at all.
Frames produced during a flush now land in a queue; when the flush completes
the whole queue moves into the pipe and leaves as the next one. So responses
that completed during one transport write share the single write after it,
which extends the pass coalescing to handlers that finish outside the pass -
which is every handler that touches a database or an upstream. Callers who
queued await that turn, keeping backpressure and the yield a real flush gave
them. Kestrel's Http2FrameWriter and Go's net/http2 writer take this shape.
h2load -t4 -c32 -m16 -D8, 2 reactors, 3 reps:
async handler, Task.Yield 0 bytes (faulted) -> 1928617 req/s
async handler, Task.Delay(1) 0 bytes (faulted) -> 352000 req/s
streamed 8x1KiB 551000 -> 1346000 (2.44x)
buffered 2 B 5180000 -> 5110000 (-1%)
buffered 8 KiB 1754280 -> 1768103
The coalesce limit is now per writer rather than per pass: many responses
coalescing into one large write is the point, and the per-pass version split
that write and cost a third of streamed throughput.
First tests for any of this - the fake transport enforces the real contract,
that a write during a flush throws and so does a second flush, so the case
that used to break is the case under test.
* feat(httpclient): the HTTP/2 client is pure C# now
The h2 client was the last thing holding the nghttp2 binding in the tree. It
is the same framing, HPACK and flow control ioxide.http2 already runs for the
server, pointed the other way round: the preface, odd stream ids, requests
instead of responses, and the retry rules that decide whether a failed
exchange may be sent again.
Those types stay internal to ioxide.http2 and reach the client through
InternalsVisibleTo. Duplicating them was the alternative, and two copies of an
HPACK encoder is how the two drift; promoting them to public API would have
made the package's surface bigger than its job.
Kept from the binding, because the hazards are the client's own rather than
nghttp2's: completions are recorded during a parse and resumed after it
unwinds, so a resumed caller that submits again - or retries through the pool,
which may dispose this connection - never re-enters the parser from inside
itself.
Verified against nginx over h2c and over TLS with ALPN, not just against our
own server: GET, a POST body that reaches the origin, 25 requests multiplexed
onto one connection, MaxResponseBytes, and a trailered response. Those five
were skipping for want of a sidecar; they run now. Two more cover what a 1 KiB
GET cannot reach - a 1 MiB body that has to park on the flow-control window and
resume on each WINDOW_UPDATE, and a header block that has to leave as HEADERS
plus CONTINUATION.
Http 37 pass, 0 skipped.
* chore(nghttp2): retire the binding into dropped/
ioxide.http2 started as the drop-in that needed no native library and ended as
the only HTTP/2 here. It measured level with the binding (0.98x-1.09x on a
small body, the ordering depending on the connection-to-reactor ratio rather
than the codec), then grew past it: streamed responses, streamed request
bodies and non-blocking dispatch all landed on the managed side, while the
binding kept the blocking dispatch loop where one slow handler held up every
other stream on the connection.
Two implementations of one protocol is a tax paid in samples, docs, tests and
benchmark fixtures, and the second one had stopped buying coverage of the
protocol's darker corners - it was buying a native build step.
dropped/ is where retired code goes: out of ioxide.slnx, out of CI, off NuGet,
kept readable because the reasoning is easier to follow with the thing itself
still there. Its README says so.
The five samples that used it move to ioxide.http2. Four are the three-name
swap the packages were designed for - Proxy/H2ToH1, H2ToH2, H2ToH3 and
Http2/SslStream. The fifth, Http2/Tls, was already ported: Http2/ManagedTls is
that same server on the managed stack, so keeping both would have been one
sample twice.
Also corrects comments the drop made false - the pure-C# module described
itself as an alternative to nghttp2, its response type claimed bytes were
copied into nghttp2 at submit, and the h2c sample said its read loop fed it.
Unit 33, Http 37, E2E 46 pass. Solution builds with no reference to the
binding left outside dropped/.
* feat(http2): request bodies can be streamed, not only responses
h2 could stream a response but never a request: Http2Request carried only
Body, so the whole upload was assembled before the handler saw any of it, and
MaxRequestBytes was the only thing standing between a hostile peer and the
arena. h3 has had Http3Request.BodyReader all along; this is its counterpart.
Http2Options.StreamRequestBodies dispatches at the HEADERS and hands the
handler an Http2BodyReader. The stream then stays in _streams while it runs -
DATA frames still have somewhere to go - and is retired when the handler is
done rather than when the request ends.
What makes it worth having is where the credit goes. A chunk opens the peer's
window only as the handler READS it, so a slow consumer stops replenishing and
the peer stops sending: memory is bound by one window instead of by the body.
Crediting on arrival, which is what the buffered path does and should, would
leave the bound off. Unlike h3 the credit is shared - every stream is on one
TCP connection - so a handler that never reads holds down the connection window
for every other stream too, and the comment says so.
Wakes are deferred to after the parser unwinds, the same discipline the h3
reader and the write queue already use, so a resumed handler cannot re-enter
the parser mid-frame.
Tests: 1 MiB uploaded through the pure-C# client into a streaming origin, and
three unit tests on the part end-to-end cannot see - that no WINDOW_UPDATE is
emitted while a handler holds the body unread, that a bodyless request reads
empty instead of parking forever, and that buffered dispatch still assembles
and still credits on arrival.
Unit 36, Http 38, both 0 failed.
* feat(playground): name the samples by what streams, and cover h2 both ways
"Managed" stopped meaning anything when nghttp2 left - all HTTP/2 here is pure
C# now - and "streamed" never said WHICH direction, which is the thing a reader
actually needs to know. Samples are named for what they do, and the library
appears only where two of them still exist:
Http2/Managed -> Http2/Buffered
Http2/ManagedStreamed-> Http2/StreamedResponse
Http2/ManagedTls -> Http2/Tls
Http3/Managed -> Http3/Buffered
Http3/ManagedStreamed-> Http3/StreamedBoth
Http3/Nghttp3 -> Http3/Nghttp3Request
Http3/Buffered -> Http3/Nghttp3Buffered
Http3/Streamed -> Http3/Nghttp3Response
Two new h2 samples close the ladder against h3's: StreamedRequest, where the
body arrives a chunk at a time and reading it is what credits the peer, and
StreamedBoth, whose /echo reads and writes at once - the shape a proxy needs,
and the reason the two directions are separate switches rather than one.
Writing them found two real bugs, which is the argument for samples that run:
ReleaseAllCreditWaiters enumerated _creditWaiters while waking writers, and
those wake INLINE and re-register at once - "collection was modified", thrown
from the teardown finally, so it escaped the catch that exists to stop a
malformed peer looking like a server fault. The stream-0 path had the mirror of
it, clearing after releasing and so discarding the waiter a resumed writer had
just added, parking it forever. Both take the waiters out before waking any.
And PendingRequest.SendWindow opened at the RFC default of 65535 instead of what
the peer's SETTINGS advertised. Streams are created long after those SETTINGS
arrive, so a response longer than 65535 bytes stalled waiting for a WINDOW_UPDATE
the peer had no reason to send - it believed we still held its whole window.
Streamed responses had been dodging it by dropping the stream from _streams,
which also dropped the per-stream window from the credit calculation; keeping the
stream for a streamed request is what exposed it. /feed went from 64 KB in three
seconds to 926 MB.
Verified by running them: 8 MiB through /echo exactly, 50 MiB uploaded to
/upload, /feed endless. Unit 36, Http 38, E2E 46, Chaos 37.
* docs(site): the example tabs say which direction streams
The http/3 menu had TWO tabs reading "nghttp3 · streamed" - one was request
streaming, one was response streaming, and nothing on the page distinguished
them. The rest were named after a library ("pure c#"), which says what a sample
is built from and not what it does.
Every tab now names the direction, and the library appears only in http/3 where
two implementations still exist:
h2c · buffered h3 · buffered
h2c · response streamed h3 · request + response streamed
h2c · request streamed h3 · buffered (nghttp3)
h2c · both streamed h3 · request streamed (nghttp3)
h2 · tls & alpn h3 · response streamed (nghttp3)
h2 · over sslstream
Three of those h2 tabs are new: response streaming was never on the site at all,
and request streaming and both-directions did not exist until this branch. Their
notes are about the trade rather than the API - buffered bounds nothing but
MaxRequestBytes, streamed bounds one flow-control window because a chunk credits
the peer only as the handler reads it.
The two nghttp2 panes are gone with the package. The proxy panes said they
needed ioxide.nghttp2, which would now fail to restore; they take ioxide.http2.
The h2-over-TLS pane no longer explains itself as a diff against a tab that does
not exist, and the learn pages stop offering a choice between two h2 packages.
Panes regenerate idempotently from the samples, no reference to a removed tab is
left in the page or the stylesheet, and no mention of nghttp2 survives anywhere
under docs/.
* chore(release): 0.4.176
All ten packable projects move together, as they always have. The gate on
bumping was the async-workload numbers, and those exist now: an asynchronous
handler went from serving 0 bytes to serving, streamed responses measure 2.45x,
and buffered 8 KiB is unchanged.
ioxide.nghttp2 is not among them any more - it stays at 0.4.169 in dropped/,
which is genuinely its last published version. That also means 0.4.176 is the
first release where ioxide.httpclient does not pull it in: its dependencies are
ioxide, ioxide.http2, ioxide.nghttp3 and ioxide.ngtcp2, so the HTTP/2 half of
the client no longer ships a native library. The h3 half still does.
Verified by packing: every inter-package dependency resolves to 0.4.176, and
CI's pack steps match the ten projects exactly.
* fix(http2): close the two published h2 DoS vectors
Both were found by asking what nghttp2 was buying us beyond speed. The answer
was hardening, and these are the two most-published HTTP/2 denial-of-service
vectors of recent years - neither of which the managed server defended against.
MaxConcurrentStreams was advertised in SETTINGS and never enforced: the option
appeared exactly three times, none of which compared it to _streams.Count. So a
peer could open unbounded streams, each costing a PendingRequest and a pooled
arena, and "open a stream, reset it, repeat" (CVE-2023-44487) cost the peer
nothing. Streams past the limit are now refused with REFUSED_STREAM, which
RFC 9113 8.7 makes safe for the peer to retry elsewhere.
The header block was unbounded. MaxFrameSize caps one frame at 16 KiB, but
nothing capped how many CONTINUATION frames follow a HEADERS that never sets
END_HEADERS, so the accumulated block grew until the process died - the
CONTINUATION flood. MaxHeaderListSize bounds it, is advertised, and exceeding
it is a CONNECTION error rather than a stream one, because a block that stops
being decoded desynchronises HPACK for everything after it.
The subtlety in both: a refused or over-long block still has to be DECODED.
HPACK is one stream across the whole connection, so skipping a block would
desynchronise the table for every later request. A refused stream's block
decodes into a shared scratch and is thrown away, and that scratch is bounded
too - otherwise refusing a stream would itself be the way in.
The tests fail without the fix, which is the only reason to trust them: the
flood test wedges for the full 120s timeout, and the stream test sees no
RST_STREAM at all. Chaos 39 pass, 0 failed.
* revert(nghttp2): bring the binding back as a supported alternative
Retiring it traded away something the benchmark could not see. It is the
reference implementation: continuously fuzzed, patched by people whose job it
is when the next HTTP/2 CVE lands, and carrying a decade of interop against
every other stack. The two DoS vectors closed in the previous commit are
exactly the class of thing that buys - both were vectors nghttp2 had defended
against for years and the managed server never had.
So it is back in src/, in the solution, in CI's pack list and at 0.4.176, and
dropped/ is gone with it - an empty folder documenting a decision that was
reversed is worse than no folder.
What is NOT restored is the client. ioxide.httpclient stays on the managed
stack, which measured 1.35x-1.39x the binding as a client and is where the
features are. So nothing depends on ioxide.nghttp2 now: it is a standalone
server-side option a user opts into, not something pulled in transitively.
The sample comes back as Playground/Http2/Nghttp2Buffered - the naming scheme
puts the library back on the tab now that there are two h2 implementations
again, and Buffered is the honest suffix because buffered is all it does. Its
pane says so, and says what that costs: no streamed response, no streamed
request, and the dispatch loop still waits for each handler in turn.
Unit 36, Chaos 39, Http 38 pass. The sample serves.
* fix(nghttp2): a slow handler no longer blocks every other stream
The binding kept the dispatch loop the managed stack had already been fixed
out of: DispatchReadyAsync awaited each handler in turn, so one request that
parked held up every other stream on that TCP connection - including responses
already submitted with nowhere to go. Two requests on one connection, one
sleeping a second, over nghttp:
/slow /fast
before 1.02s 1.02s
after 1.01s 17.01ms
Handlers that answer synchronously stay inline, so they still submit in time
for the pass drain and allocate no Task.
The drain needed the guard first, and for the reason the managed stack needed a
whole write queue: a handler finishing late submits and drains from outside the
read loop, and two drains interleaving would write out of the single _egress
buffer while a flush was outstanding - which a PipeWriter refuses outright.
Here one flag is enough, because nghttp2 holds the queued frames itself: a
caller that arrives mid-drain sets _drainAgain and the in-flight drain loops
once more to pull what was just submitted.
The detached tail also has to do what the loop would have: submit, answer 500
and log if the handler threw, retire the request, and drain - nothing observes
that Task, so an escaping exception would leave the peer waiting on a stream
that never comes.
This is the prerequisite for streaming. Streaming on a dispatch loop that
blocks would mean a streamed response parking the whole connection.
Unit 36, Chaos 39, Http 38, E2E 46 pass.
* test(nghttp2): the binding is supported again, so it is tested again
Chaos covered only the managed Http2Connection, and the Http suite's h2 tests
moved to the managed client - so a package we just committed to supporting had
essentially no automated coverage, and the dispatch fix in the previous commit
had none at all.
The wire is identical, so H2cClient is shared and the same assaults point at
Nghttp2Connection: bad preface, oversize frame, unknown frame types, a frame
truncated mid-payload, and a CONTINUATION flood.
That last one is worth stating, because it answers what "battle-tested" buys in
something other than adjectives: nghttp2 refuses the flood with NO configuring,
where the managed server had to be taught MaxHeaderListSize this morning. The
same test, the same client, two implementations, one of which had the defence
already.
The head-of-line test asserts on ORDER rather than elapsed time - the first
response to come back must be /fast, not the /slow stream dispatched before it.
That is deterministic where a stopwatch is not, and it fails against the loop it
replaced: made blocking again, it reports "expected [3], got [1]".
Chaos 46, Unit 36, Http 38, E2E 46.1 parent bf7375e commit fd6a3e6
90 files changed
Lines changed: 4242 additions & 1186 deletions
File tree
- .github/workflows
- Playground
- Clients/Http
- Http2
- Buffered
- ManagedTls
- Nghttp2Buffered
- SslStream
- StreamedBoth
- StreamedRequest
- StreamedResponse
- Tls
- Http3
- Buffered
- Managed
- Nghttp3Buffered
- Nghttp3Request
- Nghttp3Response
- StreamedBoth
- Proxy
- H1ToH3
- H2ToH1
- H2ToH2
- H2ToH3
- H3ToH3
- Quic/Alpn
- bench
- results
- docs
- assets
- learn
- scripts
- src
- clients
- ioxide.file
- ioxide.httpclient
- Http2
- ioxide.pg
- ioxide.redis
- ioxide
- protocols
- ioxide.http2
- ioxide.http3
- ioxide.nghttp2
- Connection
- ioxide.nghttp3
- ioxide.ngtcp2
- serving/ioxide.Kestrel
- tests
- Ioxide.Tests.Chaos
- Ioxide.Tests.Http
- Ioxide.Tests.Unit
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
85 | 85 | | |
86 | 86 | | |
87 | 87 | | |
88 | | - | |
89 | | - | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
90 | 91 | | |
91 | 92 | | |
92 | 93 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
17 | | - | |
| 17 | + | |
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | | - | |
| 6 | + | |
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
10 | | - | |
| 10 | + | |
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| |||
Lines changed: 2 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | | - | |
10 | | - | |
| 9 | + | |
| 10 | + | |
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| |||
Lines changed: 4 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
10 | | - | |
| 10 | + | |
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| |||
71 | 71 | | |
72 | 72 | | |
73 | 73 | | |
74 | | - | |
75 | | - | |
| 74 | + | |
| 75 | + | |
76 | 76 | | |
77 | 77 | | |
78 | 78 | | |
| |||
89 | 89 | | |
90 | 90 | | |
91 | 91 | | |
92 | | - | |
| 92 | + | |
93 | 93 | | |
94 | 94 | | |
95 | 95 | | |
| |||
This file was deleted.
Lines changed: 2 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | | - | |
10 | | - | |
| 9 | + | |
| 10 | + | |
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| |||
File renamed without changes.
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
| 16 | + | |
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | | - | |
| 6 | + | |
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
| 16 | + | |
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
21 | | - | |
| 21 | + | |
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| |||
90 | 90 | | |
91 | 91 | | |
92 | 92 | | |
93 | | - | |
94 | | - | |
| 93 | + | |
| 94 | + | |
95 | 95 | | |
96 | 96 | | |
97 | 97 | | |
| |||
0 commit comments