On Ubuntu (docker up): backend failed to connect to MySQL after 30 tentatives.

[AchrafB2015]

While running "sudo docker-compose -f docker-compose.prod.yml up", I had the following error :

mysql_1 | 2023-09-08T08:54:15.042298Z 0 [Note] mysqld: ready for connections.
mysql_1 | Version: '5.7.43' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server (GPL)
backend_1 | Couldn't connect to MySQL: 0/30
backend_1 | Couldn't connect to MySQL: 1/30
backend_1 | Couldn't connect to MySQL: 2/30
backend_1 | Couldn't connect to MySQL: 3/30
backend_1 | Couldn't connect to MySQL: 4/30
backend_1 | Couldn't connect to MySQL: 5/30
backend_1 | Couldn't connect to MySQL: 6/30
backend_1 | Couldn't connect to MySQL: 7/30
backend_1 | Couldn't connect to MySQL: 8/30
backend_1 | Couldn't connect to MySQL: 9/30
backend_1 | Couldn't connect to MySQL: 10/30
backend_1 | Couldn't connect to MySQL: 11/30
backend_1 | Couldn't connect to MySQL: 12/30
backend_1 | Couldn't connect to MySQL: 13/30
backend_1 | Couldn't connect to MySQL: 14/30
backend_1 | Couldn't connect to MySQL: 15/30
backend_1 | Couldn't connect to MySQL: 16/30
backend_1 | Couldn't connect to MySQL: 17/30
backend_1 | Couldn't connect to MySQL: 18/30
backend_1 | Couldn't connect to MySQL: 19/30
backend_1 | Couldn't connect to MySQL: 20/30
backend_1 | Couldn't connect to MySQL: 21/30
backend_1 | Couldn't connect to MySQL: 22/30
backend_1 | Couldn't connect to MySQL: 23/30
backend_1 | Couldn't connect to MySQL: 24/30
backend_1 | Couldn't connect to MySQL: 25/30
backend_1 | Couldn't connect to MySQL: 26/30
backend_1 | Couldn't connect to MySQL: 27/30
backend_1 | Couldn't connect to MySQL: 28/30
backend_1 | Couldn't connect to MySQL: 29/30
backend_1 | Error connecting to database
audino_backend_1 exited with code 1

Anyone had this issue, how could I solve this? Thanks !

[manrajgrover]

@AchrafB2015 The backend cannot connect to the MySQL server that docker spawned. If you could share the docker-compose file used and the output of docker ps -a or docker-compose ps -a, it would help me debug the issue.

[turian]

@manrajgrover I have the same issue. I just used the docker-compose.prod.yml from main, but changed the passwords from the defaults.

Here is my output:

joseph@audino:~/audino$ docker ps -a
CONTAINER ID   IMAGE             COMMAND                  CREATED         STATUS                     PORTS                               NAMES
0d8b1d03db08   audino_nginx      "nginx -g 'daemon of…"   2 minutes ago   Up 2 minutes               0.0.0.0:80->80/tcp, :::80->80/tcp   audino-nginx-1
499bce86b6e7   audino_backend    "sh /app/backend/scr…"   2 minutes ago   Exited (1) 5 seconds ago                                       audino-backend-1
aae638d1b9ff   audino_frontend   "docker-entrypoint.s…"   2 minutes ago   Exited (0) 2 minutes ago                                       audino-frontend-1
0e583e825d31   mysql:5.7         "docker-entrypoint.s…"   2 minutes ago   Up 2 minutes               3306/tcp, 33060/tcp                 audino-mysql-1
c812218e22ef   audino_redis      "docker-entrypoint.s…"   2 minutes ago   Up 2 minutes               6379/tcp                            audino-redis-1

Here are my precise logs from docker compose up:

Attaching to audino-backend-1, audino-frontend-1, audino-mysql-1, audino-nginx-1, audino-redis-1
audino-redis-1     | 1:C 07 Jan 2024 01:14:00.353 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
audino-redis-1     | 1:C 07 Jan 2024 01:14:00.353 # Redis version=5.0.8, bits=64, commit=00000000, modified=0, pid=1, just started
audino-redis-1     | 1:C 07 Jan 2024 01:14:00.353 # Configuration loaded
audino-redis-1     | 1:M 07 Jan 2024 01:14:00.356 * Running mode=standalone, port=6379.
audino-redis-1     | 1:M 07 Jan 2024 01:14:00.356 # Server initialized
audino-redis-1     | 1:M 07 Jan 2024 01:14:00.356 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
audino-redis-1     | 1:M 07 Jan 2024 01:14:00.356 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
audino-redis-1     | 1:M 07 Jan 2024 01:14:00.357 * Ready to accept connections
audino-mysql-1     | 2024-01-07 01:14:00+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.44-1.el7 started.
audino-frontend-1 exited with code 0
audino-mysql-1     | 2024-01-07 01:14:00+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
audino-mysql-1     | 2024-01-07 01:14:00+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.44-1.el7 started.
audino-backend-1   | Connecting to database
audino-mysql-1     | '/var/lib/mysql/mysql.sock' -> '/var/run/mysqld/mysqld.sock'
audino-backend-1   | Couldn't connect to MySQL: 0/30
audino-mysql-1     | 2024-01-07T01:14:01.356110Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
audino-mysql-1     | 2024-01-07T01:14:01.357846Z 0 [Note] mysqld (mysqld 5.7.44) starting as process 1 ...
audino-mysql-1     | 2024-01-07T01:14:01.365136Z 0 [Note] InnoDB: PUNCH HOLE support available
audino-mysql-1     | 2024-01-07T01:14:01.365175Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
audino-mysql-1     | 2024-01-07T01:14:01.365180Z 0 [Note] InnoDB: Uses event mutexes
audino-mysql-1     | 2024-01-07T01:14:01.365187Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
audino-mysql-1     | 2024-01-07T01:14:01.365191Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.13
audino-mysql-1     | 2024-01-07T01:14:01.365195Z 0 [Note] InnoDB: Using Linux native AIO
audino-mysql-1     | 2024-01-07T01:14:01.366350Z 0 [Note] InnoDB: Number of pools: 1
audino-mysql-1     | 2024-01-07T01:14:01.366920Z 0 [Note] InnoDB: Using CPU crc32 instructions
audino-mysql-1     | 2024-01-07T01:14:01.371168Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
audino-mysql-1     | 2024-01-07T01:14:01.383086Z 0 [Note] InnoDB: Completed initialization of buffer pool
audino-mysql-1     | 2024-01-07T01:14:01.385645Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
audino-mysql-1     | 2024-01-07T01:14:01.397769Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
audino-mysql-1     | 2024-01-07T01:14:01.409140Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
audino-mysql-1     | 2024-01-07T01:14:01.409238Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
audino-mysql-1     | 2024-01-07T01:14:01.463027Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
audino-mysql-1     | 2024-01-07T01:14:01.464560Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
audino-mysql-1     | 2024-01-07T01:14:01.464583Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
audino-mysql-1     | 2024-01-07T01:14:01.465073Z 0 [Note] InnoDB: Waiting for purge to start
audino-mysql-1     | 2024-01-07T01:14:01.515375Z 0 [Note] InnoDB: 5.7.44 started; log sequence number 12219281
audino-mysql-1     | 2024-01-07T01:14:01.516049Z 0 [Note] Plugin 'FEDERATED' is disabled.
audino-mysql-1     | 2024-01-07T01:14:01.516903Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
audino-mysql-1     | 2024-01-07T01:14:01.520064Z 0 [Note] InnoDB: Buffer pool(s) load completed at 240107  1:14:01
audino-mysql-1     | 2024-01-07T01:14:01.523478Z 0 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them.
audino-mysql-1     | 2024-01-07T01:14:01.523530Z 0 [Note] Skipping generation of SSL certificates as certificate files are present in data directory.
audino-mysql-1     | 2024-01-07T01:14:01.523537Z 0 [Warning] A deprecated TLS version TLSv1 is enabled. Please use TLSv1.2 or higher.
audino-mysql-1     | 2024-01-07T01:14:01.523540Z 0 [Warning] A deprecated TLS version TLSv1.1 is enabled. Please use TLSv1.2 or higher.
audino-mysql-1     | 2024-01-07T01:14:01.524637Z 0 [Warning] CA certificate ca.pem is self signed.
audino-mysql-1     | 2024-01-07T01:14:01.524716Z 0 [Note] Skipping generation of RSA key pair as key files are present in data directory.
audino-mysql-1     | 2024-01-07T01:14:01.525245Z 0 [Note] Server hostname (bind-address): '*'; port: 3306
audino-mysql-1     | 2024-01-07T01:14:01.525323Z 0 [Note] IPv6 is available.
audino-mysql-1     | 2024-01-07T01:14:01.525343Z 0 [Note]   - '::' resolves to '::';
audino-mysql-1     | 2024-01-07T01:14:01.525381Z 0 [Note] Server socket created on IP: '::'.
audino-mysql-1     | 2024-01-07T01:14:01.529947Z 0 [Warning] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
audino-mysql-1     | 2024-01-07T01:14:01.541827Z 0 [Note] Event Scheduler: Loaded 0 events
audino-mysql-1     | 2024-01-07T01:14:01.542048Z 0 [Note] Execution of init_file '/mysql/create_database.sql' started.
audino-mysql-1     | 2024-01-07T01:14:01.543036Z 0 [Note] Execution of init_file '/mysql/create_database.sql' ended.
audino-mysql-1     | 2024-01-07T01:14:01.543215Z 0 [Note] mysqld: ready for connections.
audino-mysql-1     | Version: '5.7.44'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  MySQL Community Server (GPL)
audino-mysql-1     | 2024-01-07T01:14:06.316654Z 3 [Note] Access denied for user 'audino'@'172.18.0.4' (using password: YES)
audino-backend-1   | Couldn't connect to MySQL: 1/30
audino-nginx-1     | 24.144.100.225 - - [07/Jan/2024:01:14:06 +0000] "\x16\x03\x01\x00\xA2\x01\x00\x00\x9E\x03\x03L6z\x83\xECt>\x83r\x1A\x1B\xE5} \x90\x8C\xD1\xD8Z\xE1N\xCE\xF0i" 400 150 "-" "-"
audino-nginx-1     | 24.144.100.225 - - [07/Jan/2024:01:14:06 +0000] "\x16\x03\x01\x00\xA2\x01\x00\x00\x9E\x03\x03\x09\xAB<\x91R\xDD\xD3\x16\xF7\xA5s\x96)\xCE2\x05\xD4<\x17\x96\x97\xC3E\xE9\x80\xE2J5\xBC\xF0\xBC\x9A\x00\x00*\xC0+\xC0/\x00\x9E\xCC\xA9\xCC\xA8\xCC\xAA\xC0" 400 150 "-" "-"
audino-mysql-1     | 2024-01-07T01:14:11.323396Z 4 [Note] Access denied for user 'audino'@'172.18.0.4' (using password: YES)
audino-backend-1   | Couldn't connect to MySQL: 2/30
audino-mysql-1     | 2024-01-07T01:14:16.327175Z 5 [Note] Access denied for user 'audino'@'172.18.0.4' (using password: YES)
audino-backend-1   | Couldn't connect to MySQL: 3/30
audino-mysql-1     | 2024-01-07T01:14:21.334869Z 6 [Note] Access denied for user 'audino'@'172.18.0.4' (using password: YES)

[turian]

@manrajgrover I also tried again using the default passwords (i.e. not changing the docker-compose.prod.yml from main) and am still getting this issue.

[turian]

My base system is ubuntu-22.04

[turian]

It does appear to work with the dev docker, I can at least get to the landing page.

[manrajgrover]

@turian Hi, can you try connecting to MySQL through docker container created?

Using credentials mentioned here

$ docker exec -it audino-mysql-1 /bin/bash
$ mysql -u audino -p -A -P 3306

[turian]

@manrajgrover So trying those commands using the default credentials, I cannot connect:

joseph@audino:/root$ docker exec -it audino-mysql-1 /bin/bash
bash-4.2# mysql -u audino -p -A -P 3306
Enter password:
ERROR 1045 (28000): Access denied for user 'audino'@'localhost' (using password: YES)