Skip to content

fix(redis): pin dedicated connection in Conn for connection-stateful commands - #4864

Open
yangfei1024 wants to merge 3 commits into
gogf:masterfrom
yangfei1024:fix/redis-conn-pin-connection
Open

fix(redis): pin dedicated connection in Conn for connection-stateful commands#4864
yangfei1024 wants to merge 3 commits into
gogf:masterfrom
yangfei1024:fix/redis-conn-pin-connection

Conversation

@yangfei1024

@yangfei1024 yangfei1024 commented Aug 22, 2026

Copy link
Copy Markdown

Description

Some Redis commands are connection-stateful: they change the state of the
connection itself on the server side, and all subsequent commands on that same
connection are then interpreted in that state. The most common examples are
MULTI/EXEC (a transaction opened with MULTI belongs to the connection —
every command sent on it is queued and answered with +QUEUED until EXEC or
DISCARD arrives), plus WATCH, SELECT, etc.

For such commands, a client must guarantee they all run on the same physical
connection
. Native go-redis does this: client.Conn() returns a connection
backed by a sticky pool that holds one dedicated connection until Close.

The redis adapter's Conn() does not. It returns a thin wrapper that forwards
every command to the shared client, so each Do independently borrows some
connection from the pool and returns it immediately (connPool.Get → execute →
releaseConn). The consequences for connection-stateful commands:

  1. conn.Do(ctx, "MULTI") runs on some pooled connection A, and A goes back to
    the pool while still in MULTI state on the server side.
  2. Any concurrent request that borrows A gets its commands queued inside the open
    transaction. It receives the literal string "QUEUED" instead of the real
    reply — a GET returns "QUEUED", a TTL check returns "QUEUED". Whatever
    the application does with that "value" is now wrong.
  3. Under load, the EXEC may be routed to a different, clean connection and
    fail with EXEC without MULTI, while A stays stuck in MULTI state in the
    pool, poisoning every request that borrows it — potentially for a long time
    (default ConnMaxIdleTime is 30 minutes).

In short: code written as conn, _ := g.Redis().Conn(ctx); conn.Do("MULTI"); …; conn.Do("EXEC") looks correct, matches the documented purpose of Conn
("a connection object for continuous operations … call Close function manually
if you do not use this connection"
), and works under native go-redis — but with
this adapter the transaction is silently broken and other innocent requests get
corrupted replies. It is a leaky abstraction: the API copies the shape of
go-redis Conn but not its semantics.

问题描述

Redis 有一部分命令是连接态命令:它们改变的是连接本身在服务端的状态,
之后发往同一条连接的所有命令都会在这个状态下被解释。最常见的例子是
MULTI/EXECMULTI 开启的事务属于这条连接——其后这条连接上的每条命令都
被排队、回复 +QUEUED,直到收到 EXECDISCARD),此外还有 WATCH
SELECT 等。

对这类命令,客户端必须保证它们全部落在同一条物理连接上。原生 go-redis
正是这么做的:client.Conn() 返回的连接由 sticky pool 支撑,独占一条物理连接
直到 Close

redis 适配层的 Conn() 没有做到。它返回的只是一个薄包装,把每条命令
转发给共享的 client,因此每次 Do 都会独立地从池里借出某条连接、执行完
立即归还(connPool.Get → 执行 → releaseConn)。对连接态命令的后果是:

  1. conn.Do(ctx, "MULTI") 在某条池化连接 A 上执行,A 归还回池时在服务端
    仍处于 MULTI 状态
  2. 任何并发请求借到 A,它发出的命令都会被排进这个未提交的事务,收到的不是
    真实回复而是字面量字符串 "QUEUED"——GET 返回 "QUEUED",TTL 查询返回
    "QUEUED"。应用程序拿这个"值"做的任何处理都随之出错。
  3. 高并发下,EXEC 还可能被路由到另一条干净连接上,以 EXEC without MULTI 报错;而 A 带着 MULTI 状态滞留在池中,持续污染每个借到它的请求
    ——持续时间可能很长(默认 ConnMaxIdleTime 为 30 分钟)。

简言之:conn, _ := g.Redis().Conn(ctx); conn.Do("MULTI"); …; conn.Do("EXEC")
这样的代码看起来正确、符合 Conn 的文档定位("a connection object for
continuous operations … call Close function manually if you do not use this
connection"
)、在原生 go-redis 下也确实正确——但在这层适配器下事务被静默破坏,
其他无关请求收到被污染的回复。这是一个典型的泄漏抽象:API 复制了 go-redis
Conn 的形状,却没有复制它的语义。

Reproduce

conn, _ := g.Redis().Conn(ctx)
defer conn.Close(ctx)

conn.Do(ctx, "MULTI")
// While the transaction is open, a concurrent command through the shared pool
// can borrow the MULTI-state connection and receive "QUEUED" instead of the
// stored value:
v, _ := g.Redis().Do(ctx, "GET", "anykey")   // v.String() == "QUEUED"
conn.Do(ctx, "EXEC")                          // may fail: "EXEC without MULTI"

复现方式

conn, _ := g.Redis().Conn(ctx)
defer conn.Close(ctx)

conn.Do(ctx, "MULTI")
// 事务打开期间,走共享池的并发命令可能借到处于 MULTI 状态的连接,
// 收到的不是存储的值而是 "QUEUED":
v, _ := g.Redis().Do(ctx, "GET", "anykey")   // v.String() == "QUEUED"
conn.Do(ctx, "EXEC")                          // 可能失败: "EXEC without MULTI"

Real-world example & timeline

A typical scenario where this bites — no exotic setup required:

  • Goroutine 1 (e.g. a background job) writes a batch of members into a set using
    Conn + MULTI … SADD … EXEC, because it looks like the idiomatic way to do a
    transactional batch write.
  • Goroutine 2 is any ordinary request reading some unrelated key through
    g.Redis().Do(ctx, "GET", key) on the same default group.

They share one connection pool. The interleaving looks like this:

EXPECTED (what the Conn API implies, and what native go-redis does):

  Goroutine 1 (owns a pinned connection A)
    ├─ MULTI ──────────→ A: enters transaction state
    ├─ SADD m1 ────────→ A: +QUEUED
    ├─ SADD m2 ────────→ A: +QUEUED
    ├─ EXEC ───────────→ A: committed, A is clean again
    └─ Close ──────────→ A returns to pool (only now)

ACTUAL (adapter implementation — every Do borrows/returns independently):

  Time ─────────────────────────────────────────────────────────────────────→

  Goroutine 1                     pool [A | B | C]              Goroutine 2
  ───────────────────────────────────────────────────────────────────────────
  conn.Do("MULTI")
    borrows A → +OK
    A RETURNS TO POOL ─────→  [A(MULTI!) | B | C]
                                             ←────────────── Do("GET", key)
                                               borrows A (LIFO reuse)
                                               server queues GET into the
                                                 open transaction on A
                                               receives "+QUEUED"  ❌
                                               app now holds "QUEUED"
                                               as the value of `key`
  conn.Do("SADD", m1)
    borrows A → +QUEUED
    A returns to pool
  ...
  conn.Do("EXEC")
    borrows ? ── under load this may be B, not A
    if B: "-ERR EXEC without MULTI"  ❌
      and A stays in MULTI state in the pool,
      poisoning every later borrower
      until idle eviction (default 30 min)

Two properties make this bug particularly nasty in production:

  • It is intermittent. On an idle pool, LIFO reuse usually hands the same
    connection back to the transaction user, so everything appears to work. It
    only breaks when another request happens to borrow the connection inside the
    millisecond-scale MULTI→EXEC window.
  • It self-heals — sometimes. Once an EXEC does land on A, the queued
    commands of the victims are submitted together with the transaction and A
    becomes clean again, leaving no obvious trace. Only the unlucky branch
    (EXEC without MULTI) leaves a poisoned connection behind.

Typical symptoms reported by application logs: an unrelated key's value is
occasionally the 6-character string "QUEUED"; intermittent
EXEC without MULTI errors from the transaction user; hard to reproduce because
each individual piece looks correct.

真实案例与时序图

一个很容易踩中的典型场景,不需要任何特殊配置:

  • 协程 1(比如一个后台任务)用 Conn + MULTI … SADD … EXEC 批量写一个 set
    ——这看起来正是事务批量写入的地道写法。
  • 协程 2 是一个再普通不过的请求,通过同一默认分组的
    g.Redis().Do(ctx, "GET", key) 读一个毫不相关的 key。

两者共享同一个连接池。交错时序如下:

预期行为(Conn 的 API 注释所暗示的、原生 go-redis 的实际行为):

  协程 1(独占钉住的连接 A)
    ├─ MULTI ──────────→ A:进入事务状态
    ├─ SADD m1 ────────→ A:+QUEUED
    ├─ SADD m2 ────────→ A:+QUEUED
    ├─ EXEC ───────────→ A:提交,A 恢复干净
    └─ Close ──────────→ 此刻 A 才归还连接池

实际行为(适配层实现——每次 Do 都独立借还连接):

  时间 ────────────────────────────────────────────────────────────────────→

  协程 1                        连接池 [A | B | C]              协程 2
  ─────────────────────────────────────────────────────────────────────────
  conn.Do("MULTI")
    借出 A → +OK
    A 立即归还 ──────────→  [A(MULTI态!) | B | C]
                                             ←────────────── Do("GET", key)
                                               借到 A(LIFO 复用)
                                               服务器把这条 GET 排进
                                                 A 上未提交的事务
                                               收到 "+QUEUED"  ❌
                                               应用拿到 "QUEUED"
                                               作为 key 的值
  conn.Do("SADD", m1)
    借出 A → +QUEUED
    A 归还
  ...
  conn.Do("EXEC")
    借出 ?── 并发下可能是 B 而不是 A
    若是 B:"-ERR EXEC without MULTI"  ❌
      且 A 永远滞留在池中、保持 MULTI 状态,
      持续毒化后续借到它的每个请求
      直到空闲回收(默认 30 分钟)

两个特性使这个 bug 在生产环境中尤其难缠:

  • 偶发。 池空闲时,LIFO 复用通常会把同一条连接还给事务使用者,一切看起来
    正常。只有当别的请求恰好在这个毫秒级的 MULTI→EXEC 窗口内借走该连接时才会
    出问题。
  • 部分自愈。 一旦某个 EXEC 恰好落回 A 上,受害请求被排队的命令会随事务
    一起提交、A 恢复干净,不留明显痕迹。只有不走运的分支(EXEC without MULTI)才会留下一条被毒化的连接。

应用日志里的典型症状:某个无关 key 的值偶发变成 6 个字符的字符串
"QUEUED";事务使用者偶发 EXEC without MULTI 报错;每段代码单独看都正确,
因此极难复现定位。

Solution

  • Redis.Conn() now pins a dedicated underlying connection for non-cluster
    clients via go-redis client.Conn() (backed by a StickyConnPool), so all
    commands issued through the same Conn run on the same physical connection,
    and Conn.Close actually returns it to the shared pool.
  • Subscribe/PubSub behavior is unchanged (PubSub already manages its own
    connection).
  • Cluster clients are not supported by go-redis Conn(); they keep the previous
    shared-pool behavior (Conn.conn == nil fallback).

Note: Redis.Do internally uses Conn() + Close() per call, which now
borrows/returns the pinned connection once per command — the same pool Get/Put
cost as before, so no extra overhead for stateless usage.

修复方案

  • Redis.Conn() 对非集群客户端,通过 go-redis 的 client.Conn()(底层为
    StickyConnPool)钉住一条专用物理连接,使同一个 Conn 上发出的所有命令
    都落在同一条物理连接上,Conn.Close 真正把连接归还共享池。
  • Subscribe/PubSub 行为不变(PubSub 本就自管连接)。
  • go-redis 的 Conn() 不支持集群客户端,集群保持原有共享池行为
    Conn.conn == nil 时回退)。

说明:Redis.Do 内部每次调用都会走一遍 Conn() + Close(),现在相当于每条
命令借还一次钉住连接——池的 Get/Put 开销与之前相同,无状态用法没有额外损耗。

Example (after the fix)

conn, _ := g.Redis().Conn(ctx)
defer conn.Close(ctx)

conn.Do(ctx, "MULTI")
conn.Do(ctx, "SADD", "k", "m1")   // +QUEUED, same physical connection guaranteed
conn.Do(ctx, "SADD", "k", "m2")
v, _ := conn.Do(ctx, "EXEC")      // correct transaction results

示例(修复后)

conn, _ := g.Redis().Conn(ctx)
defer conn.Close(ctx)

conn.Do(ctx, "MULTI")
conn.Do(ctx, "SADD", "k", "m1")   // +QUEUED,保证在同一条物理连接上
conn.Do(ctx, "SADD", "k", "m2")
v, _ := conn.Do(ctx, "EXEC")      // 返回正确的事务结果

Tests

Added TestConn_Transaction in redis_z_unit_conn_test.go, covering:

  • commands issued through the shared client during an open transaction are NOT
    affected by the MULTI state (previously they could receive QUEUED);
  • EXEC returns the correct transaction results;
  • the pinned connection is clean after EXEC.

Verified locally by actually running go test -count=1 ./... in
contrib/nosql/redis (Go 1.26.4 / Redis 8.10.1): all tests pass, including the
new TestConn_Transaction. The only failures are the sentinel tests, which
require a local Sentinel environment and fail identically on master
(connection refused, unrelated to this change).

测试

redis_z_unit_conn_test.go 中新增 TestConn_Transaction,覆盖:

  • 事务打开期间,走共享客户端发出的命令不受 MULTI 状态影响(此前可能收到
    QUEUED);
  • EXEC 返回正确的事务结果;
  • EXEC 之后钉住的连接是干净的。

已在本地实际执行 go test -count=1 ./...(Go 1.26.4 / Redis 8.10.1)验证:
contrib/nosql/redis 全部通过,包括新增的 TestConn_Transaction。唯一的
失败项是 sentinel 相关测试,需要本地 Sentinel 环境,在 master 上同样失败
(连接被拒,与本次改动无关)。

yangfei and others added 3 commits August 20, 2026 01:06
…commands

The redis adapter's Conn() returned a thin wrapper that only forwarded
each command to the shared client, so every command borrowed and
returned a pooled connection independently. For connection-stateful
commands like MULTI/EXEC this breaks transaction semantics: the
connection returns to the pool still in MULTI state, and concurrent
requests that borrow it receive QUEUED replies instead of command
results (e.g. an auth GET reading the literal string "QUEUED").
EXEC may also land on a different clean connection and fail with
"EXEC without MULTI", leaving the polluted connection stuck in the
pool until idle eviction.

Now Conn() pins a dedicated underlying connection via go-redis
StickyConnPool for non-cluster clients, so all commands issued
through the same Conn run on the same physical connection, and
Close() returns it to the pool. Cluster clients are not supported
by go-redis Conn() and keep the previous shared-pool behavior.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant