fix(balancer): keep least_conn load state across upstream scaling

[AlinsRan]

Fixes #12217

The problem

Scale a least_conn upstream out and the new node is not preferred — the load stays on the original nodes. With WebSocket it never recovers, because those connections outlive the scale out.

Reproduced by t/node/least_conn3.t TEST 9 (real WebSocket sessions through APISIX). Two nodes, four live sessions, add a third node, open two more:

sessions before the scale out    0.0.0.0    127.0.0.1  127.0.0.1  0.0.0.0     <- two each
the two sessions after it        0.0.0.0    127.0.0.2
                                 ^^^^^^^                ^^^^^^^^^
                    already carrying two sessions       the new, empty node

Round-robin. The empty node gets half, an already-loaded node gets the other half.

Why

The per-server score heap lives inside the picker, and the picker is cached by upstream version. Scaling bumps the version, the heap is rebuilt, every score resets to 1/weight — the four live sessions are forgotten.

Seeding the new heap from a side count doesn't fix it either. In-flight requests keep the picker they were routed with and release their server on that picker in the log phase, so those releases land on a heap nobody reads. A heap that is only correct at rebuild time drifts: once the old sessions drain, the surviving nodes stay penalized for load they no longer carry — the same imbalance, mirrored.

The fix

Move the heap and the in-flight counts out of the picker, into per-worker state keyed by (resource_key, priority) — stable across scaling and health flips, unlike the picker version. The picker now reconciles that heap with the current node set instead of rebuilding it:

• surviving nodes keep their load
• a new node starts empty and is preferred right away
• a node that leaves keeps its count, so its score is restored if it comes back
• every picker generation shares one view of the load, so a release always lands on the heap in use

Single-node upstreams now go through the balancer. pick_server used to skip it when #nodes == 1, so anything routed while the upstream was alone was never counted — and a 1→N scale out then saw the surviving node as empty. One node is where a k8s deployment or a discovery service starts, so this was most of the bug, not an edge case.

The state is held weakly (__mode = "v"), which is exactly the lifetime it needs: a picker holds its state, an in-flight request holds its picker, the picker cache holds the current one. So a state lives as long as anything can still release into it — and when nothing can, there is no connection left to count. Nothing to size, nothing to evict. (An LRU would rank states by how recently they were rebuilt; a stable upstream full of long-lived connections is precisely the one that is never rebuilt.)

Scores are derived from the count, (count + 1) / weight, instead of accumulated with ± effect_weight. Same semantics, exact over time.

Release discipline

Because the state now outlives the picker, releasing a server the request no longer holds is no longer self-healing — it used to be washed away by the next rebuild, now it is written into shared state for good.

So after_balance owns the invariant: ctx.balancer_server names only the server the request currently holds, and releasing it clears the field. Three release sites relied on the caller to remember, and didn't:

where	what happened
pick_server, on entering a retry	ran out of servers → log phase released the same one again
pick_server, skipping an unhealthy node	same
ai-proxy-multi.retry_on_error	releases before it knows whether it will pick again

The first needs nothing unusual to hit: retries is derived from all nodes while only the healthy ones are picked from, so one node failing its health check is enough. ai-proxy-multi is fixed by construction — the plugin isn't touched.

Two more, in the same vein:

• ctx.server_picker is published as soon as a server is held, not at the end of pick_server, so a request that bails out after picking one still releases it.
• The retry entry only marks a server tried when there is another one to move to. Otherwise a single-node upstream would mark its only node tried, get() would find nothing left, and a configured retries would turn a retryable blip into a hard 502.

Stream

stream_log_phase() never called after_balance. On L4 the count only ever went up, so least_conn could not balance TCP long connections at all, and ctx.balancer_tried_servers leaked.

Scope and known limits

State is shared per (resource_key, priority) per worker. An upstream with no stable resource key falls back to state private to its picker — today's behaviour, so no regression, but no fix either:

• traffic-split inline upstreams — up_conf is synthesised in the plugin, and its own picker cache key already embeds the nodes table address, so there is nothing stable to key on
• ai-proxy-multi instances (ups_tab = {})
• conf_server in the EE port (picker.new(nodes, {}))

Two things routing single-node least_conn through the balancer changes:

1. Cost. Such upstreams now pay the picker path (lrucache_server_picker + lrucache_addr + a size-1 heap peek/update per request) — the same cost every multi-node upstream already pays, but a real one on the hottest path for the most common topology.
2. Unbracketed IPv6 node hosts. A single node used to bypass parse_addr; it no longer does, so "::1:1980" now fails the way it already fails everywhere else. Pre-existing bug, not a new one — on origin/master, a two-node upstream with unbracketed IPv6 hosts crashes identically (balancer.lua:238: attempt to concatenate field 'port' (a nil value)) for every balancer type. The Admin API rejects such nodes; only the DP-side check skips that validation. Worth fixing, but on its own: the addr string is built in transform_node and shared with chash and the health checker.

No new config, no shared dict, on by default. Counting stays per-worker, matching the existing per-worker heap. Short requests return their count on completion, so steady-state behaviour is unchanged.

Tests

These three fail on master:

test	what it covers
least_conn3.t TEST 9	the issue as reported — real WebSocket sessions, scaled out underneath them
least_conn3.t TEST 10	the 1→N case
least_conn3.t TEST 7	the same scale-out with long-running HTTP requests

The rest pin the internals: count kept across a rebuild, drained node returns to the pool, scale-down, releases on the old picker seen by the new one, priority isolation, weight change, keyless-upstream fallback, and no double release (TEST 6 drives the real balancer.pick_server).

Also least_conn.t TEST 5 (more retries than nodes — that path had no coverage) and TEST 6 (a single node is retried, because it is the only one there is), and t/stream-node/least_conn.t for the L4 release.

Checklist

• I have explained the need for this PR and the problem it solves
• I have explained the changes or the new features added to this PR
• I have added tests corresponding to this change
• I have updated the documentation to reflect this change
• I have verified that this change is backward compatible