We have Percona MySQL Cluster running with three nodes on CentOS.
Server : Percona-XtraDB-Cluster-server-56-5.6.19-25.6.824.el6.x86_64
One node is crashing repeatedly.

Thread pointer: 0x18910ba30
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7f354c30fd38 thread_stack 0x40000
/usr/sbin/mysqld(_ZN13MYSQL_BIN_LOG19raw_get_current_logEP11 st_log_info+0x2e)[0x89b1be]
/usr/sbin/mysqld(_Z24plugin_foreach_with_maskP3THDPFcS0_P13s t_plugin_intPvEijS3_+0x226)[0x6f7746]
/usr/sbin/mysqld(_Z28ha_start_consistent_snapshotP3THD+0x281 )[0x5a8d51]
/usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x65 8)[0x6ed768]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3 THDPcj+0x1362)[0x6ef412]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7effb4e03890): is an invalid pointer
Connection ID (thread ID): 202009

I am unable to understand why it is crashing and what should I look at.

I have already tried following but server still crashes with same output.
1. Removed Percona Server.
2. Removed all data under /var/lib/mysql
3. Rebooted server
4. Install Percona Server
5. Sync with cluster (full SST) - Copies all data from running node.

It still crashes with same error.
It does not crash immediately but randomly after about 18-22 hours.

  • Might be a bug, is 5.6.19 the latest version? because community is on 5.6.20. Is there a possibility to over-commit memory?
    – Antonios
    Aug 4, 2014 at 15:37
  • Yes, it is latest version. Server has sufficient free memory.
    – Jai
    Aug 5, 2014 at 0:51
  • I would suggest to turn on general query log to catch what was executed on crash time. This line smells like a bug: Query (7effb4e03890): is an invalid pointer, Did you open a case on Percona about it?
    – Antonios
    Aug 5, 2014 at 8:12
  • It's probably worth your time to thoroughly test the server's RAM, and if that's good, check the disks for bad blocks. Aug 6, 2014 at 1:51
  • You are absolutely right. It does seems like drive errors. Server is on LSI RAID 10 with 8 drives. MegaCLI is showing raid array optimal but one of the drive is showing errors, I believe that drive is faulty and needs to be replaced. Thanks.
    – Jai
    Aug 6, 2014 at 6:02


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.