Skip to content

[do-not-merge] mysql ci debug #14468

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

shmsr
Copy link
Member

@shmsr shmsr commented Jul 9, 2025

Proposed commit message

See title.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

@shmsr
Copy link
Member Author

shmsr commented Jul 9, 2025

Changes made to fix this error:

ERROR 1410 (42000) at line 1: You are not allowed to create a user with GRANT
MySQL slave is not up yet...
MySQL slave is not up yet...
MySQL slave is not up yet...

Got hints from BK logs:

_bk;t=17515512786772025/07/03 14:01:18 DEBUG Container mysql (bb6e892b01d166a6a3f5863b28b0329a7a1c266216f0d3b8e6e1e0b64b8889c8) status: unhealthy

_bk;t=17515512786772025/07/03 14:01:18 DEBUG Container mysql_is_ready (4a2511a56d9dac3031a989a9a5a0b471c71d6103f172f987d85672dd84c68562) status: exited (exit code: 0)

_bk;t=17515512786772025/07/03 14:01:18 DEBUG Container mysql_master (54b0b6da0a4b96fd5ad2b7f2a76265bcf836e98d4325410b6013225645775e94) status: running (no health status)

_bk;t=17515512786772025/07/03 14:01:18 DEBUG Container mysql_replica (1873b2536fdb2bac511c70115e46391d2d462dab1d35418b2bbd130f832b7880) status: exited (exit code: 1)

@elastic-vault-github-plugin-prod

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@shmsr shmsr added the bugfix Pull request that fixes a bug issue label Jul 9, 2025
@shmsr shmsr self-assigned this Jul 9, 2025

# Verify slave status
echo "Verifying slave status..."
mysql -uroot -p"$MYSQL_ROOT_PASSWORD" -e "GRANT REPLICATION CLIENT ON *.* TO 'mydb_replica_user'@'%'; FLUSH PRIVILEGES;"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is redundant as it is already there in grant-permissions.sql. Also the placement is wrong as GRANTs should be done before starting the slave.

Comment on lines 63 to 90
mysql -uroot -p"$MYSQL_ROOT_PASSWORD" -e "STOP SLAVE; RESET SLAVE ALL;" 2>/dev/null || echo "Slave was not running"

# Use here-doc to avoid command line escaping issues
mysql -uroot -p"$MYSQL_ROOT_PASSWORD" <<EOF
CHANGE MASTER TO
MASTER_HOST='$MYSQL_MASTER_HOST',
MASTER_USER='mydb_replica_user',
MASTER_PASSWORD='mydb_replica_pwd',
MASTER_LOG_FILE='$CURRENT_LOG',
MASTER_LOG_POS=$CURRENT_POS;
EOF

# Prepare the CHANGE MASTER TO command
start_slave_stmt="CHANGE MASTER TO MASTER_HOST='$MYSQL_MASTER_HOST', MASTER_USER='mydb_replica_user', MASTER_PASSWORD='mydb_replica_pwd', MASTER_LOG_FILE='$CURRENT_LOG', MASTER_LOG_POS=$CURRENT_POS; START SLAVE;"
if [ $? -eq 0 ]; then
echo "CHANGE MASTER command executed successfully"

# Display the command that will be executed
echo "Running the following command to start replication: $start_slave_stmt"
# Start the slave
mysql -uroot -p"$MYSQL_ROOT_PASSWORD" -e "START SLAVE;"

# Run the CHANGE MASTER TO command
mysql -uroot -p"$MYSQL_ROOT_PASSWORD" -e "$start_slave_stmt" || {
echo "Failed to start replication. Entering sleep mode for debugging."
tail -f /dev/null # This will keep the container running indefinitely
}
if [ $? -eq 0 ]; then
echo "Slave started successfully"
else
echo "Failed to start slave"
exit 1
fi
else
echo "Failed to execute CHANGE MASTER command"
exit 1
fi
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same thing but better error handling.

Comment on lines +23 to +33
echo "Attempt $i: Checking master status..."
MS_STATUS=$(mysql -h "$MYSQL_MASTER_HOST" -P 3306 -u root -p"$MYSQL_ROOT_PASSWORD" -e "SHOW MASTER STATUS;" 2>/dev/null || echo "FAILED")

if [ "$MS_STATUS" = "FAILED" ]; then
echo "Failed to connect to master, retrying..."
sleep 2
continue
fi

echo "Master status response: $MS_STATUS"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again better handling but with 2s delay before retrying.

Comment on lines +2 to 3
GRANT REPLICATION SLAVE ON *.* TO 'mydb_replica_user'@'%';
GRANT REPLICATION CLIENT ON *.* TO 'mydb_replica_user'@'%';
Copy link
Member Author

@shmsr shmsr Jul 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REPLICATION CLIENT - This allows the user to execute SHOW MASTER STATUS and SHOW SLAVE STATUS commands

GRANT REPLICATION SLAVE - This allows the user to connect as a replica (START SLAVE)

@shmsr
Copy link
Member Author

shmsr commented Jul 9, 2025

Changes made to fix this error:

ERROR 1410 (42000) at line 1: You are not allowed to create a user with GRANT
MySQL slave is not up yet...
MySQL slave is not up yet...
MySQL slave is not up yet...

Got hints from BK logs:

_bk;t=17515512786772025/07/03 14:01:18 DEBUG Container mysql (bb6e892b01d166a6a3f5863b28b0329a7a1c266216f0d3b8e6e1e0b64b8889c8) status: unhealthy

_bk;t=17515512786772025/07/03 14:01:18 DEBUG Container mysql_is_ready (4a2511a56d9dac3031a989a9a5a0b471c71d6103f172f987d85672dd84c68562) status: exited (exit code: 0)

_bk;t=17515512786772025/07/03 14:01:18 DEBUG Container mysql_master (54b0b6da0a4b96fd5ad2b7f2a76265bcf836e98d4325410b6013225645775e94) status: running (no health status)

_bk;t=17515512786772025/07/03 14:01:18 DEBUG Container mysql_replica (1873b2536fdb2bac511c70115e46391d2d462dab1d35418b2bbd130f832b7880) status: exited (exit code: 1)

After I fixed, there was one more issue. The previous script was working as of now somehow but it had issues. Now it is much robust than before.

Copy link

Quality Gate failed Quality Gate failed

Failed conditions
1 New issue

See analysis details on SonarQube

Catch issues before they fail your Quality Gate with our IDE extension SonarLint SonarLint

@elasticmachine
Copy link

💚 Build Succeeded

History

cc @shmsr

@shmsr
Copy link
Member Author

shmsr commented Jul 9, 2025

CI passed 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix Pull request that fixes a bug issue Integration:mysql MySQL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants