0

I have 4 node and database has InnoDB tables. my key_buffer_size is 128M. Should i increase it in my system? My innodb_buffer_pool_size is 75G and innodb_log_buffer_size = 256M.

Mem:         96688      92580       4107          0        116       8501
-/+ buffers/cache:      83962      12725
Swap:        10239       5104       5135

MariaDB [mydata]> SHOW STATUS LIKE "key%";
+------------------------+--------+
| Variable_name          | Value  |
+------------------------+--------+
| Key_blocks_not_flushed | 0      |
| Key_blocks_unused      | 107171 |
| Key_blocks_used        | 4      |
| Key_blocks_warm        | 0      |
| Key_read_requests      | 25     |
| Key_reads              | 4      |
| Key_write_requests     | 14     |
| Key_writes             | 11     |
+------------------------+--------+
8 rows in set (0.00 sec)

I got that error, one of my node closed so I have that question. I use 5.5.46-MariaDB-1~trusty wsrep_25.12.r4f8102 Thanks

RECORD LOCKS space id 513 page no 16 n bits 296 index `GEN_CLUST_INDEX` of table `mydata`.`user_counter` trx id 65CFC8177 lock_mode X locks rec but not gap
161230 19:08:36 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see http://kb.askmonty.org/en/reporting-bugs

We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.

Server version: 5.5.46-MariaDB-1~trusty-wsrep-log
key_buffer_size=134217728
read_buffer_size=2097152
max_used_connections=809
max_threads=2002
thread_count=224
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 12467846 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0x7feb728c6000
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7ff378886a00 thread_stack 0x48000
(my_addr_resolve failure: fork)
/usr/sbin/mysqld(my_print_stacktrace+0x2e) [0x7ff37c2db1ae]
/usr/sbin/mysqld(handle_fatal_signal+0x457) [0x7ff37bebffc7]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x10340) [0x7ff37a90e340]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x39) [0x7ff379f65cc9]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x148) [0x7ff379f690d8]
/usr/sbin/mysqld(+0x2ded4e) [0x7ff37bcafd4e]
/usr/sbin/mysqld(+0x835033) [0x7ff37c206033]
/usr/sbin/mysqld(+0x83b811) [0x7ff37c20c811]
/usr/sbin/mysqld(+0x83c35b) [0x7ff37c20d35b]
/usr/sbin/mysqld(+0x75a2b2) [0x7ff37c12b2b2]
/usr/sbin/mysqld(+0x75e7ad) [0x7ff37c12f7ad]
/usr/sbin/mysqld(+0x729202) [0x7ff37c0fa202]
/usr/sbin/mysqld(Rows_log_event::find_row(Relay_log_info const*)+0x665) [0x7ff37bfa0a45]
/usr/sbin/mysqld(Update_rows_log_event::do_exec_row(Relay_log_info const*)+0x9c) [0x7ff37bfa0e8c]
/usr/sbin/mysqld(Rows_log_event::do_apply_event(Relay_log_info const*)+0x25c) [0x7ff37bf944ac]
/usr/sbin/mysqld(wsrep_apply_cb(void*, void const*, unsigned long, unsigned int, wsrep_trx_meta const*)+0x7ba) [0x7ff37be70afa]
/usr/lib/galera/libgalera_smm.so(galera::TrxHandle::apply(void*, wsrep_cb_status (*)(void*, void const*, unsigned long, unsigned int, wsrep_trx_meta const*), wsrep_trx_meta const&) const+0xd8) [0x7ff377b188f8]
/usr/lib/galera/libgalera_smm.so(+0x1df27d) [0x7ff377b4f27d]
/usr/lib/galera/libgalera_smm.so(galera::ReplicatorSMM::apply_trx(void*, galera::TrxHandle*)+0xd2) [0x7ff377b51b32]
/usr/lib/galera/libgalera_smm.so(galera::ReplicatorSMM::process_trx(void*, galera::TrxHandle*)+0x10e) [0x7ff377b5498e]
/usr/lib/galera/libgalera_smm.so(galera::GcsActionSource::dispatch(void*, gcs_action const&, bool&)+0x1b8) [0x7ff377b33668]
/usr/lib/galera/libgalera_smm.so(galera::GcsActionSource::process(void*, bool&)+0x58) [0x7ff377b33ef8]
/usr/lib/galera/libgalera_smm.so(galera::ReplicatorSMM::async_recv(void*)+0x73) [0x7ff377b54ef3]
/usr/lib/galera/libgalera_smm.so(galera_recv+0x18) [0x7ff377b634e8]
/usr/sbin/mysqld(+0x4a0744) [0x7ff37be71744]
/usr/sbin/mysqld(start_wsrep_THD+0x48e) [0x7ff37bcccc0e]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ff37a906182]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff37a02947d]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x0): is an invalid pointer
Connection ID (thread ID): 10
Status: NOT_KILLED

Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=off

The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
161230 19:08:40 mysqld_safe Number of processes running now: 0
161230 19:08:40 mysqld_safe WSREP: not restarting wsrep node automatically
161230 19:08:40 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
  • 1
    The STATUS values imply that key_buffer_size is much bigger than necessary. Furthermore, Galera cannot replicate MyISAM, so there is virtually no use for your tables to be MyISAM. – Rick James Jan 6 '17 at 18:04
2

The answer is no. Why ???

key_buffer_size is a buffer used for MyISAM table index caching. Since you have a 75G buffer pool, you can leave all MyISAM related settings in their default settings.

You also seem to have 4G of RAM free.

Let's look into this section of your Stack Trace

/usr/sbin/mysqld(Rows_log_event::find_row(Relay_log_info const*)+0x665) [0x7ff37bfa0a45]
/usr/sbin/mysqld(Update_rows_log_event::do_exec_row(Relay_log_info const*)+0x9c) [0x7ff37bfa0e8c]
/usr/sbin/mysqld(Rows_log_event::do_apply_event(Relay_log_info const*)+0x25c) [0x7ff37bf944ac]
/usr/sbin/mysqld(wsrep_apply_cb(void*, void const*, unsigned long, unsigned int, wsrep_trx_meta const*)+0x7ba) [0x7ff37be70afa]
/usr/lib/galera/libgalera_smm.so(galera::TrxHandle::apply(void*, wsrep_cb_status (*)(void*, void const*, unsigned long, unsigned int, wsrep_trx_meta const*), wsrep_trx_meta const&) const+0xd8) [0x7ff377b188f8]
/usr/lib/galera/libgalera_smm.so(+0x1df27d) [0x7ff377b4f27d]
/usr/lib/galera/libgalera_smm.so(galera::ReplicatorSMM::apply_trx(void*, galera::TrxHandle*)+0xd2) [0x7ff377b51b32]
/usr/lib/galera/libgalera_smm.so(galera::ReplicatorSMM::process_trx(void*, galera::TrxHandle*)+0x10e) [0x7ff377b5498e]
/usr/lib/galera/libgalera_smm.so(galera::GcsActionSource::dispatch(void*, gcs_action const&, bool&)+0x1b8) [0x7ff377b33668]
/usr/lib/galera/libgalera_smm.so(galera::GcsActionSource::process(void*, bool&)+0x58) [0x7ff377b33ef8]
/usr/lib/galera/libgalera_smm.so(galera::ReplicatorSMM::async_recv(void*)+0x73) [0x7ff377b54ef3]
/usr/lib/galera/libgalera_smm.so(galera_recv+0x18) [0x7ff377b634e8]
/usr/sbin/mysqld(+0x4a0744) [0x7ff37be71744]
/usr/sbin/mysqld(start_wsrep_THD+0x48e) [0x7ff37bcccc0e]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ff37a906182]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff37a02947d]

I can see you are using Galera wsrep Replication but the message also show that you are using MySQL Replication. The Galera node you are using appears to be a Slave trying to extract a binlog event from a relay log.

What would make extracting a binlog event from a relay log trigger a problem like this ? Only when the Master is a major release older than a Slave.

About 2 years ago, I answered the post mysql replication master 5.5 slave 5.1 error on create database. where I discuss how DATETIME in MySQL 5.6 now has microseconds in its binlog layout and would break replication when seeing a DATETIME value from a MySQL 5.5 Master. I also mentioned this issue in another past post (mysql master slave replication scenario)

Replicating from older versions of MySQL could introduce problems of this nature.

I would suggested upgrading the Master to the same version of MariaDB using the wsrep. It is evident from the trace that wsrep's process of reading from a relay log encountered a binlog event it does not understand.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.