Triage for dotnet/runtime#115728 by KirillKurdyukov

[MihuBot]

Triage for dotnet/runtime#115728.
Last updated: 2025-05-19_16-13-30

EnableMultipleHttp2Connections=true doesn't create additional TCP connections under high load (1000+ RPS) by KirillKurdyukov
[Tool] Searching for EnableMultipleHttp2Connections not creating multiple TCP connections, SocketsHttpHandler multiple HTTP/2 connections high load, gRPC SocketsHttpHandler only one TCP connection, EnableMultipleHttp2Connections true not working, SocketsHttpHandler high throughput multiple connections
[Tool] Found 29 issues, 37 comments, 32 returned results (6257 ms)
[Tool] Obtained 91 comments for issue #40249: SocketsHttpHandler.EnableMultipleHttp2Connections exceeds server stream limit
[Tool] Obtained 26 comments for issue #35088: HTTP2: Create additional connections when maximum active streams is reached
[Tool] Obtained 15 comments for issue #66944: HttpConnectionPool should aggressively open multiple HTTP/2 connections under load if EnableMultipleHttp2Connections is set

Here are the most relevant related issues and discussions for triaging #115728 ("EnableMultipleHttp2Connections=true doesn't create additional TCP connections under high load (1000+ RPS)"):

---

1. #40249 - SocketsHttpHandler.EnableMultipleHttp2Connections exceeds server stream limit (92 comments)

Summary:

• This issue and its discussion are highly relevant. It details how, even with EnableMultipleHttp2Connections=true, SocketsHttpHandler may not open additional connections as expected under high load, especially when a burst of requests is sent before the server's SETTINGS frame (with MaxConcurrentStreams) is received.
• The root cause is a race condition: the client initially assumes an unlimited stream limit, so a burst of requests can all be assigned to a single connection before the real limit is known. When the SETTINGS frame arrives, the limit is enforced, but the excess streams are already in flight, leading to server-side REFUSED_STREAM errors and sometimes connection termination.
• The discussion covers:
  • The HTTP/2 spec's ambiguity and practical server/client behaviors.
  • The need for a "slow start" or artificial initial stream limit (e.g., 100) on the client until the server's SETTINGS are received.
  • The fact that not all servers behave the same (e.g., Kestrel vs. HTTP.SYS).
  • The conclusion that the current behavior is a protocol limitation, but improvements (like an initial artificial stream limit) were proposed and implemented.
  • The importance of retrying REFUSED_STREAM errors and the need for SocketsHttpHandler to handle them gracefully.
• Key takeaways for your issue:
  • The observed behavior is not unique and is a known limitation/race in the protocol and implementation.
  • SocketsHttpHandler should now (in recent versions) use an initial stream limit of 100, which should help, but there may still be edge cases.
  • Additional connections are only created once the stream limit is reached and enforced, but a burst of requests can still overload a single connection if they all start before the SETTINGS frame is processed.

---

2. #35088 - HTTP2: Create additional connections when maximum active streams is reached (27 comments)

Summary:

• This is the original feature proposal for allowing SocketsHttpHandler to open multiple HTTP/2 connections per server, to address the exact scenario you describe (high concurrency, gRPC, server-to-server).
• The discussion led to the introduction of the EnableMultipleHttp2Connections property.
• The implementation is such that new connections are only created when all existing connections have reached the server's SETTINGS_MAX_CONCURRENT_STREAMS limit.
• There is no load balancing across connections; the handler tries to minimize the number of open connections.
• Key takeaways:
  • The property is intended for high-throughput scenarios, but its effect is only seen when the stream limit is actually reached and enforced.
  • If your requests are not exceeding the server's stream limit (or if the initial burst all lands on one connection before the SETTINGS frame), you may not see multiple connections.

---

3. #66944 - HttpConnectionPool should aggressively open multiple HTTP/2 connections under load if EnableMultipleHttp2Connections is set (16 comments)

Summary:

• This issue discusses the fact that, even with EnableMultipleHttp2Connections set, SocketsHttpHandler is conservative and only opens one connection at a time, even under high load.
• There is a suggestion to open more connections more aggressively (e.g., based on the number of pending requests and the initial stream limit).
• The discussion acknowledges that the current behavior is by design, but that more aggressive connection creation could be considered for future improvements.
• Key takeaways:
  • The current implementation does not proactively open multiple connections just because there is high load; it waits until the stream limit is hit.
  • This can lead to underutilization of available throughput in bursty or high-concurrency scenarios.

---

4. #40391 - Advanced multiple HTTP/2 connections support in SocketsHttpHandler (2 comments)

Summary:

• This is a tracking issue for finalizing the multiple connections feature.
• It references the proof-of-concept and the need to handle the stream limit logic robustly.

---

5. #28591 - HTTP2: SocketHttpHandler does not respect Max concurrent streams (3 comments)

Summary:

• Early report that SocketsHttpHandler ignored server-side stream limits, leading to RST_STREAM errors.
• This was part of the motivation for the later improvements.

---

Answers to Your Questions

1. Is this behavior intentional for SocketsHttpHandler?

• Yes, the current behavior is intentional and is a result of both the HTTP/2 spec and the implementation. SocketsHttpHandler only opens additional connections when the server's stream limit is reached and enforced. However, a burst of requests before the SETTINGS frame can still all land on a single connection.

2. Are there additional configurations required?

• No additional configuration beyond EnableMultipleHttp2Connections = true is required for the handler to open multiple connections. However, you may need to ensure that your load pattern actually causes the stream limit to be reached (i.e., more concurrent requests than the server's SETTINGS_MAX_CONCURRENT_STREAMS).
• There is no way to force more aggressive connection creation or to pre-warm multiple connections.

3. What's the recommended approach for high-throughput scenarios?

• Use EnableMultipleHttp2Connections = true as you are.
• Ensure your server's SETTINGS_MAX_CONCURRENT_STREAMS is not set too high (which would prevent additional connections from being created).
• If you need more aggressive connection creation, consider splitting your load across multiple HttpClient or SocketsHttpHandler instances, or track future improvements in the runtime for more aggressive connection pooling.
• Be aware that the current implementation does not balance load across connections; it fills up one before opening another.

---

Additional Notes

• The behavior you are seeing is a known limitation and is being discussed for future improvements (see #66944).
• There is no setting to control the number of connections or to pre-warm connections; the handler is reactive, not proactive.
• If you are not seeing multiple connections under high load, it may be because the initial burst of requests is not exceeding the stream limit on the first connection before the SETTINGS frame is processed.

---

References:

• #40249
• #35088
• #66944
• #40391
• #28591

Let me know if you need a more detailed summary of any specific discussion!