I've got a new MySQL replication setup with one master and one slave.

In all of the following scenarios, the slave automatically reconnects to the master and resumes replication where it left off:

  • mysql service is restarted on the slave host
  • Slave host is rebooted
  • mysql service is restarted on the master host (reconnection happens after the default 60-second slave I/O reconnect timeout)

However, if I try rebooting the master host, the mysql service on the slave host neither detects that the master went down nor resumes replication when the master becomes available again. In this scenario:

  • I see no entries in the mysql error log on the slave host
  • Running show slave hosts; on master (once it comes back up) shows no hosts connected

UPDATE: Looking more closely at the mysql errors logs on the master, I suspect that the master is not completing a clean shutdown because when stopping the service I see the logs:

2013-07-29 17:43:45 5281 [Note] /usr/sbin/mysqld: Normal shutdown

2013-07-29 17:43:45 5281 [Note] Giving 1 client threads a chance to die gracefully
2013-07-29 17:43:48 5281 [Note] InnoDB: Shutdown completed; log sequence number 146430781066
2013-07-29 17:43:48 5281 [Note] Shutting down plugin 'binlog'
2013-07-29 17:43:48 5281 [Note] /usr/sbin/mysqld: Shutdown complete

130729 17:43:48 mysqld_safe mysqld from pid file /opt/mysql/run/mysqld.pid ended

...while after rebooting, the mysql error log on master ends with:

2013-07-29 17:46:58 6441 [Note] InnoDB: Starting shutdown...

...and is missing the Shutdown complete message.

  • 1
    What versions of MySQL are the master and slave running? Are the two machines on the same subnet or are they separated by any kind of firewall or other stateful packet inspection device? Have you ever allowed this to go on for over half an hour or have you always fixed it before then? What OS? – Michael - sqlbot Jul 30 '13 at 1:52
  • Might also want to post SHOW SLAVE STATUS\G output with confidential information removed. I'd be interested in the following settings: Master_Retry_Count, Connect_Retry. Also output of SHOW GLOBAL VARIABLES LIKE 'slave_net_timeout';. – James Lupolt Jul 31 '13 at 19:53
  • Both master and slave machines are RHEL 6.4, running MySQL 5.6.12. My guess is that after issuing the shutdown command, the OS is not allowing enough time for mysqld to cleanly shutdown before it forcefully kills the process. Does the master send some sort of signal to the slaves when mysqld is cleanly shutdown? – Gary Aug 15 '13 at 16:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.