Demonstrate deadlock involving transactions and prepared statements

[wchargin]

Gorm can deadlock in the following scenario:

1. There is a connection pool of size 2
2. Caller A and Caller B each acquire a connection and begin a transaction
3. Caller C tries to run SELECT 1
4. Caller A and Caller B each try to run SELECT 1

In step (3), Caller C will try to prepare the statement SELECT 1, since no one else is currently preparing that statement. In order to do that, Caller C needs to acquire a database connection. But they are all currently in use, so Caller C waits on the pool.

In step (4), the holders of all the connections in the pool will wait for the statement SELECT 1 to be prepared.

This creates a deadlock: Caller C is waiting on a connection, but all connections are waiting on Caller C.

This can happen in normal Gorm user code just by using db.Transaction when the underlying database uses a connection pool. No trickery is needed to reproduce this bug. When it occurs, the entire application ceases to make progress forever.

I'm observing this in prod on Gorm v1.25.6, and can reproduce it in latest Gorm as demonstrated in this playground.

Here is an alternate single-file example, which can emit a Goroutine call stack to confirm that the situation is well and truly deadlocked:
https://gist.github.com/wchargin/9b48fd98d95299640a14fff2882d2b80

[jinzhu]

Seems it works for me in the latest version, no dead lock happen?

[wchargin]

Seems it works for me in the latest version, no dead lock happen?

Did you make sure to run with GORM_DIALECT=postgres? It doesn't repro with the default sqlite (because I didn't configure pooling).

Here's what I ran:

$ docker compose up -d --build postgres
 ✔ Container gorm-playground-prepare-deadlock-postgres-1  Started                                                                                         0.1s

$ GORM_DIALECT=postgres ./test.sh
git clone --depth 1 -b master https://github.com/go-gorm/gorm.git
Cloning into 'gorm'...
remote: Enumerating objects: 191, done.
remote: Counting objects: 100% (191/191), done.
remote: Compressing objects: 100% (183/183), done.
remote: Total 191 (delta 10), reused 49 (delta 4), pack-reused 0 (from 0)
Receiving objects: 100% (191/191), 256.46 KiB | 1.51 MiB/s, done.
Resolving deltas: 100% (10/10), done.
testing postgres...
# gorm.io/playground.test
ld: warning: '/private/var/folders/kk/lltn85xd5fqg2b3df6sr7fq00000gp/T/go-link-4171751067/000023.o' has malformed LC_DYSYMTAB, expected 98 undefined symbols to start at index 1626, found 95 undefined symbols starting at index 1626
2025/05/29 14:45:37 testing postgres...
=== RUN   TestGORM
    main_test.go:29: Bob has a connection and is thinking...
    main_test.go:29: Alice has a connection and is thinking...
    main_test.go:43: Camille wants to add two numbers...
    main_test.go:43: Bob wants to add two numbers...
    main_test.go:43: Alice wants to add two numbers...

It blocks here, and after ten minutes the tests fail with this combined call stack: https://gist.github.com/wchargin/330800fee39443dbf340c7b41377ba50

I tested on macOS Sonoma 14.5 and also Ubuntu 20.04.6 LTS, with go1.23 and go1.24, with Gorm master and Gorm v1.26.1.

[wchargin]

Did you make sure to run with GORM_DIALECT=postgres?

I've pushed a patch to reproduce the issue even if you forget to do this and just run ./test.sh.