I have a fresh install on Mariadb 10.0.20 on three servers. I'm using the same my.cnf from another running cluster, just changed the cluster name and the IPs. I bootstrap the first node, and I get a cluster size of 1. When starting the next node, the cluster size goes to 2 for a bit, syslog shows data being transfered, but then it says that there are zero mysql processes running, and shuts down.
my.cnf file on first node:
[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock
[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0
[mysqld]
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /run/mysqld
lc_messages_dir = /usr/share/mysql
lc_messages = en_US
skip-external-locking
skip_name_resolve
bind-address = 0.0.0.0
max_connections = 1000
connect_timeout = 5
wait_timeout = 600
max_allowed_packet = 16M
thread_cache_size = 1280
sort_buffer_size = 4M
bulk_insert_buffer_size = 16M
tmp_table_size = 32M
max_heap_table_size = 32M
myisam_recover = BACKUP
key_buffer_size = 18M
#open-files-limit = 2000
table_open_cache = 400
myisam_sort_buffer_size = 52M
concurrent_insert = 2
read_buffer_size = 2M
read_rnd_buffer_size = 1M
query_cache_limit = 128K
query_cache_size = 0
/etc/mysql/conf.d/mysqld_safe_syslog.cnf.
log_warnings = 2
slow_query_log_file = /var/log/mysql/mariadb-slow.log
long_query_time = 10
log_slow_verbosity = query_plan
log_bin = /var/log/mysql/mariadb-bin
log_bin_index = /var/log/mysql/mariadb-bin.index
expire_logs_days = 10
max_binlog_size = 100M
binlog_format = ROW
default_storage_engine = InnoDB
innodb_log_file_size = 512M
innodb_buffer_pool_size = 10G
innodb_log_buffer_size = 8M
innodb_file_per_table = 1
innodb_open_files = 400
innodb_io_capacity = 400
innodb_flush_method = O_DIRECT
innodb_thread_concurrency = 32
innodb_io_capacity = 2000
innodb_doublewrite = 0
innodb_use_fallocate = 1
innodb_buffer_pool_instances = 10
innodb_write_io_threads = 32
innodb_read_io_threads = 32
innodb_file_format = Barracuda
innodb_lru_scan_depth = 2000
innodb_log_group_home_dir = /mysql_logdir
innodb_data_home_dir = /mysql_datadir
innodb_autoinc_lock_mode = 2
innodb_flush_log_at_trx_commit = 0
wsrep_provider = /usr/lib/libgalera_smm.so
wsrep_provider_options = "gcache.size=300M; gcache.page_size=1G"
wsrep_cluster_name = "device_cluster"
wsrep_cluster_address = "gcomm://10.150.28.19, 10.150.28.20"
wsrep_sst_method = rsync
wsrep_node_address = "10.150.28.18"
[mysqldump]
quick
quote-names
max_allowed_packet = 16M
syslog from 2nd node:
20:47:29 localhost mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql
20:47:29 localhost mysqld_safe: WSREP: Running position recovery with --log_error='/var/lib/mysql/wsrep_recovery.20E6uy' --pid-file='/var/lib/mysql/avc2-wus1-db05-recover.pid'
20:47:29 localhost mysqld: 150813 20:47:29 [Note] /usr/sbin/mysqld (mysqld 10.0.20-MariaDB-1~trusty-wsrep-log) starting as process 5443 ...
20:47:34 localhost mysqld_safe: WSREP: Recovered position 00000000-0000-0000-0000-000000000000:2
20:47:34 localhost mysqld: 150813 20:47:34 [Note] /usr/sbin/mysqld (mysqld 10.0.20-MariaDB-1~trusty-wsrep-log) starting as process 5558 ...
20:47:34 localhost mysqld: 150813 20:47:34 [Note] WSREP: Read nil XID from storage engines, skipping position init
20:47:34 localhost mysqld: 150813 20:47:34 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/libgalera_smm.so'
20:47:34 localhost mysqld: 150813 20:47:34 [Note] WSREP: wsrep_load(): Galera 3.9(rXXXX) by Codership Oy <[email protected]> loaded successfully.
20:47:34 localhost mysqld: 150813 20:47:34 [Note] WSREP: CRC-32C: using hardware acceleration.
20:47:34 localhost mysqld: 150813 20:47:34 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:2
20:47:34 localhost mysqld: 150813 20:47:34 [Note] WSREP: Passing config to GCS: base_host = 10.150.28.19; base_port = 4567; cert.log_conflicts = no; debug = no; evs.auto
_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_
install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /
var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 1G; gcache.size = 300M; gcs.fc_debug = 0; gcs.fc
_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_l
imit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; pc.npv
20:47:34 localhost mysqld: o = false; pc.recovery
20:47:34 localhost mysqld: 150813 20:47:34 [Note] WSREP: Service thread queue flushed.
20:47:34 localhost mysqld: 150813 20:47:34 [Note] WSREP: Assign initial position for certification: 2, protocol version: -1
20:47:34 localhost mysqld: 150813 20:47:34 [Note] WSREP: wsrep_sst_grab()
20:47:34 localhost mysqld: 150813 20:47:34 [Note] WSREP: Start replication
20:47:34 localhost mysqld: 150813 20:47:34 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:2
20:47:34 localhost mysqld: 150813 20:47:34 [ERROR] WSREP: gcs/src/gcs_group.cpp:int gcs_group_init_history(gcs_group_t*, gcs_seqno_t, const gu_uuid_t*)():80: Non-negativ
e state seqno requires non-nil history UUID.
20:47:34 localhost mysqld: 150813 20:47:34 [ERROR] WSREP: gcs init failed:Invalid argument
20:47:34 localhost mysqld: 150813 20:47:34 [ERROR] WSREP: wsrep::connect() failed: 7
20:47:34 localhost mysqld: 150813 20:47:34 [ERROR] Aborting
20:47:34 localhost mysqld:
20:47:34 localhost mysqld: 150813 20:47:34 [Note] WSREP: Service disconnected.
20:47:35 localhost mysqld: 150813 20:47:35 [Note] WSREP: Some threads may fail to exit.
20:47:35 localhost mysqld: 150813 20:47:35 [Note] /usr/sbin/mysqld: Shutdown complete
20:47:35 localhost mysqld:
20:47:35 localhost mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended
20:47:43 localhost mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql
20:47:43 localhost mysqld_safe: WSREP: Running position recovery with --log_error='/var/lib/mysql/wsrep_recovery.zgu0Hd' --pid-file='/var/lib/mysql/avc2-wus1-db05-recove
r.pid'
20:47:43 localhost mysqld: 150813 20:47:43 [Note] /usr/sbin/mysqld (mysqld 10.0.20-MariaDB-1~trusty-wsrep-log) starting as process 6001 ...
20:47:48 localhost mysqld_safe: WSREP: Recovered position 00000000-0000-0000-0000-000000000000:2
20:47:48 localhost mysqld: 150813 20:47:48 [Note] /usr/sbin/mysqld (mysqld 10.0.20-MariaDB-1~trusty-wsrep-log) starting as process 6106 ...
20:47:48 localhost mysqld: 150813 20:47:48 [Note] WSREP: Read nil XID from storage engines, skipping position init
20:47:48 localhost mysqld: 150813 20:47:48 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/libgalera_smm.so'
20:47:48 localhost mysqld: 150813 20:47:48 [Note] WSREP: wsrep_load(): Galera 3.9(rXXXX) by Codership Oy <[email protected]> loaded successfully.
20:47:48 localhost mysqld: 150813 20:47:48 [Note] WSREP: CRC-32C: using hardware acceleration.
20:47:48 localhost mysqld: 150813 20:47:48 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1
20:47:48 localhost mysqld: 150813 20:47:48 [Note] WSREP: Passing config to GCS: base_host = 10.150.28.19; base_port = 4567; cert.log_conflicts = no; debug = no; evs.auto
_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_
install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /
var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 1G; gcache.size = 300M; gcs.fc_debug = 0; gcs.fc
_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_l
imit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; pc.np
v
20:47:48 localhost mysqld: o = false; pc.recovery
20:47:48 localhost mysqld: 150813 20:47:48 [Note] WSREP: Service thread queue flushed.
20:47:48 localhost mysqld: 150813 20:47:48 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
20:47:48 localhost mysqld: 150813 20:47:48 [Note] WSREP: wsrep_sst_grab()
20:47:48 localhost mysqld: 150813 20:47:48 [Note] WSREP: Start replication
20:47:48 localhost mysqld: 150813 20:47:48 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
20:47:48 localhost mysqld: 150813 20:47:48 [Note] WSREP: protonet asio version 0
20:47:48 localhost mysqld: 150813 20:47:48 [Note] WSREP: Using CRC-32C for message checksums.
20:47:48 localhost mysqld: 150813 20:47:48 [Note] WSREP: backend: asio
20:47:48 localhost mysqld: 150813 20:47:48 [Warning] WSREP: access file(gvwstate.dat) failed(No such file or directory)
20:47:48 localhost mysqld: 150813 20:47:48 [Note] WSREP: restore pc from disk failed
20:47:48 localhost mysqld: 150813 20:47:48 [Note] WSREP: GMCast version 0
20:48:03 localhost mysqld: 150813 20:48:03 [Warning] WSREP: Failed to resolve tcp:// 10.150.28.20:4567
20:48:03 localhost mysqld: 150813 20:48:03 [Note] WSREP: (8e8c1fdb, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
20:48:03 localhost mysqld: 150813 20:48:03 [Note] WSREP: (8e8c1fdb, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
20:48:03 localhost mysqld: 150813 20:48:03 [Note] WSREP: EVS version 0
20:48:03 localhost mysqld: 150813 20:48:03 [Note] WSREP: gcomm: connecting to group 'device_cluster', peer '10.150.28.18:, 10.150.28.20:'
20:48:03 localhost mysqld: 150813 20:48:03 [Note] WSREP: (8e8c1fdb, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers:
20:48:03 localhost mysqld: 150813 20:48:03 [Note] WSREP: declaring 620d451d at tcp://10.150.28.18:4567 stable
20:48:03 localhost mysqld: 150813 20:48:03 [Note] WSREP: Node 620d451d state prim
20:48:03 localhost mysqld: 150813 20:48:03 [Note] WSREP: view(view_id(PRIM,620d451d,4) memb {
20:48:03 localhost mysqld: #011620d451d,0
20:48:03 localhost mysqld: #0118e8c1fdb,0
20:48:03 localhost mysqld: } joined {
20:48:03 localhost mysqld: } left {
20:48:03 localhost mysqld: } partitioned {
20:48:03 localhost mysqld: })
20:48:03 localhost mysqld: 150813 20:48:03 [Note] WSREP: save pc into disk
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: gcomm: connected
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: Opened channel 'device_cluster'
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 2
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: Waiting for SST to complete.
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: STATE EXCHANGE: sent state msg: 97aed7ec-41fc-11e5-8545-635104629780
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: STATE EXCHANGE: got state msg: 97aed7ec-41fc-11e5-8545-635104629780 from 0 (avc2-wus1-db04)
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: STATE EXCHANGE: got state msg: 97aed7ec-41fc-11e5-8545-635104629780 from 1 (avc2-wus1-db05)
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: Quorum results:
20:48:04 localhost mysqld: #011version = 3,
20:48:04 localhost mysqld: #011component = PRIMARY,
20:48:04 localhost mysqld: #011conf_id = 3,
20:48:04 localhost mysqld: #011members = 1/2 (joined/total),
20:48:04 localhost mysqld: #011act_id = 2,
20:48:04 localhost mysqld: #011last_appl. = -1,
20:48:04 localhost mysqld: #011protocols = 0/7/3 (gcs/repl/appl),
20:48:04 localhost mysqld: #011group UUID = 76a27c1d-41f1-11e5-846a-73bafbb95904
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: Flow-control interval: [23, 23]
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 2)
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: State transfer required:
20:48:04 localhost mysqld: #011Group state: 76a27c1d-41f1-11e5-846a-73bafbb95904:2
20:48:04 localhost mysqld: #011Local state: 00000000-0000-0000-0000-000000000000:-1
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: New cluster view: global state: 76a27c1d-41f1-11e5-846a-73bafbb95904:2, view# 4: Primary, number of nodes: 2, my index: 1, protocol version 3
20:48:04 localhost mysqld: 150813 20:48:04 [Warning] WSREP: Gap in state sequence. Need state transfer.
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: Running: 'wsrep_sst_rsync --role 'joiner' --address '10.150.28.19' --auth '' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --defaults-group-suffix '' --parent '6106' --binlog '/var/log/mysql/mariadb-bin' '
20:48:04 localhost rsyncd[6293]: rsyncd version 3.1.0 starting, listening on port 4444
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: Prepared SST request: rsync|10.150.28.19:4444/rsync_sst
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: REPL Protocols: 7 (3, 2)
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: Service thread queue flushed.
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: Assign initial position for certification: 2, protocol version: 3
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: Service thread queue flushed.
20:48:04 localhost mysqld: 150813 20:48:04 [Warning] WSREP: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (76a27c1d-41f1-11e5-846a-73bafbb95904): 1 (Operation not permitted)
20:48:04 localhost mysqld: #011 at galera/src/replicator_str.cpp:prepare_for_IST():456. IST will be unavailable.
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: Member 1.0 (avc2-wus1-db05) requested state transfer from '*any*'. Selected 0.0 (avc2-wus1-db04)(SYNCED) as donor.
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 2)
20:48:04 localhost mysqld: 150813 20:48:04 [Note] WSREP: Requesting state transfer: success, donor: 0
20:48:04 localhost rsyncd[6308]: connect from bb-db-device-01 (10.150.28.18)
20:48:04 localhost rsyncd[6308]: rsync to rsync_sst/ from bb-db-device-01 (10.150.28.18)
20:48:04 localhost rsyncd[6308]: receiving file list
20:48:04 localhost rsyncd[6310]: connect from bb-db-device-01 (10.150.28.18)
20:48:04 localhost rsyncd[6308]: sent 63 bytes received 12,596,622 bytes total size 12,593,152
20:48:05 localhost rsyncd[6310]: rsync to rsync_sst-log_dir/ from bb-db-device-01 (10.150.28.18)
20:48:05 localhost rsyncd[6310]: receiving file list
20:48:06 localhost mysqld: 150813 20:48:06 [Note] WSREP: (8e8c1fdb, 'tcp://0.0.0.0:4567') turning message relay requesting off
20:48:12 localhost rsyncd[6403]: connect from bb-db-device-01 (10.150.28.18)
20:48:12 localhost rsyncd[6404]: connect from bb-db-device-01 (10.150.28.18)
20:48:12 localhost rsyncd[6310]: sent 63 bytes received 1,074,004,192 bytes total size 1,073,741,824
20:48:12 localhost rsyncd[6404]: rsync to rsync_sst/./performance_schema from bb-db-device-01 (10.150.28.18)
20:48:12 localhost rsyncd[6403]: rsync to rsync_sst/./mysql from bb-db-device-01 (10.150.28.18)
20:48:12 localhost rsyncd[6404]: receiving file list
20:48:12 localhost rsyncd[6403]: receiving file list
20:48:12 localhost rsyncd[6407]: connect from bb-db-device-01 (10.150.28.18)
20:48:12 localhost rsyncd[6404]: sent 1,036 bytes received 100,851 bytes total size 96,357
20:48:12 localhost rsyncd[6403]: sent 1,682 bytes received 1,036,830 bytes total size 1,030,397
20:48:12 localhost rsyncd[6407]: rsync to rsync_sst/ from bb-db-device-01 (10.150.28.18)
20:48:12 localhost rsyncd[6407]: receiving file list
20:48:12 localhost mysqld: 150813 20:48:12 [Note] WSREP: 0.0 (avc2-wus1-db04): State transfer to 1.0 (avc2-wus1-db05) complete.
20:48:12 localhost mysqld: 150813 20:48:12 [Note] WSREP: Member 0.0 (avc2-wus1-db04) synced with group.
20:48:12 localhost rsyncd[6407]: sent 48 bytes received 183 bytes total size 39
20:48:12 localhost mysqld: WSREP_SST: [INFO] Extracting binlog files: (20150813 20:48:12.683)
20:48:12 localhost mysqld: mariadb-bin.000014
20:48:12 localhost mysqld: WSREP_SST: [INFO] Joiner cleanup. (20150813 20:48:12.693)
20:48:13 localhost rsyncd[6293]: sent 0 bytes received 0 bytes total size 0
20:48:13 localhost mysqld: WSREP_SST: [INFO] Joiner cleanup done. (20150813 20:48:13.201)
20:48:13 localhost mysqld: 150813 20:48:13 [Note] WSREP: SST complete, seqno: 2
20:48:13 localhost mysqld: 150813 20:48:13 [Note] InnoDB: Using mutexes to ref count buffer pool pages
20:48:13 localhost mysqld: 150813 20:48:13 [Note] InnoDB: The InnoDB memory heap is disabled
20:48:13 localhost mysqld: 150813 20:48:13 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
20:48:13 localhost mysqld: 150813 20:48:13 [Note] InnoDB: Memory barrier is not used
20:48:13 localhost mysqld: 150813 20:48:13 [Note] InnoDB: Compressed tables use zlib 1.2.8
20:48:13 localhost mysqld: 150813 20:48:13 [Note] InnoDB: Using Linux native AIO
20:48:13 localhost mysqld: 150813 20:48:13 [Note] InnoDB: Using CPU crc32 instructions
20:48:13 localhost mysqld: 150813 20:48:13 [Note] InnoDB: Initializing buffer pool, size = 10.0G
20:48:13 localhost mysqld: 150813 20:48:13 [Note] InnoDB: Completed initialization of buffer pool
20:48:13 localhost mysqld: 150813 20:48:13 [Note] InnoDB: Highest supported file format is Barracuda.
20:48:13 localhost mysqld: 150813 20:48:13 [Warning] InnoDB: The log sequence number in the ibdata files is higher than the log sequence number in the ib_logfiles! Are you sure you are using the right ib_logfiles to start up the database. Log sequence number in the ib_logfiles is 1600939, logsequence numbers stamped to ibdata file headers are between 1601220 and 1601220.
20:48:13 localhost mysqld: 150813 20:48:13 [Note] InnoDB: The log sequence numbers 1601220 and 1601220 in ibdata files do not match the log sequence number 1600939 in the ib_logfiles!
20:48:13 localhost mysqld: 150813 20:48:13 [Note] InnoDB: Database was not shutdown normally!
20:48:13 localhost mysqld: 150813 20:48:13 [Note] InnoDB: Starting crash recovery.
20:48:13 localhost mysqld: 150813 20:48:13 [Note] InnoDB: Reading tablespace information from the .ibd files...
20:48:13 localhost mysqld: 150813 20:48:13 [Note] InnoDB: Restoring possible half-written data pages
20:48:13 localhost mysqld: 150813 20:48:13 [Note] InnoDB: from the doublewrite buffer...
20:48:16 localhost /etc/init.d/mysql[6553]: 0 processes alive and '/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in
20:48:16 localhost /etc/init.d/mysql[6553]: #007/usr/bin/mysqladmin: connect to server at 'localhost' failed
20:48:16 localhost /etc/init.d/mysql[6553]: error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111 "Connection refused")'
20:48:16 localhost /etc/init.d/mysql[6553]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
20:48:16 localhost /etc/init.d/mysql[6553]:
20:48:27 localhost mysqld: 2015-08-13 20:48:27 7f7ad3fcb7c0 InnoDB: Error: page 5 log sequence number 1601220
20:48:27 localhost mysqld: InnoDB: is in the future! Current system log sequence number 1600939.
20:48:27 localhost mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
20:48:27 localhost mysqld: InnoDB: tablespace but not the InnoDB log files. See
20:48:27 localhost mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
20:48:27 localhost mysqld: InnoDB: for more information.
20:48:28 localhost mysqld: 150813 20:48:28 [Warning] InnoDB: Resizing redo log from 2*32768 to 2*131008 pages, LSN=1600939
20:48:28 localhost mysqld: 150813 20:48:28 [Warning] InnoDB: Starting to delete and rewrite log files.
20:48:29 localhost mysqld: 150813 20:48:29 [Note] InnoDB: Setting log file /mysql_logdir/ib_logfile101 size to 2047 MB
20:48:29 localhost mysqld: 150813 20:48:29 [Note] InnoDB: Setting log file /mysql_logdir/ib_logfile1 size to 2047 MB
20:48:29 localhost mysqld: 150813 20:48:29 [Note] InnoDB: Renaming log file /mysql_logdir/ib_logfile101 to /mysql_logdir/ib_logfile0
20:48:29 localhost mysqld: 150813 20:48:29 [Warning] InnoDB: New log files created, LSN=1601036
20:48:29 localhost mysqld: 150813 20:48:29 [Note] InnoDB: 128 rollback segment(s) are active.
20:48:29 localhost mysqld: 150813 20:48:29 [Note] InnoDB: Waiting for purge to start
20:48:29 localhost mysqld: 150813 20:48:29 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.24-72.2 started; log sequence number 1600939
20:48:29 localhost mysqld: 150813 20:48:29 [Note] Plugin 'FEEDBACK' is disabled.
20:48:29 localhost mysqld: 150813 20:48:29 [Note] Recovering after a crash using /var/log/mysql/mariadb-bin
20:48:29 localhost mysqld: 150813 20:48:29 [Note] Starting crash recovery...
20:48:29 localhost mysqld: 150813 20:48:29 [Note] Crash recovery finished.
20:48:29 localhost mysqld: 150813 20:48:29 [Warning] InnoDB: Cannot open table mysql/gtid_slave_pos from the internal data dictionary of InnoDB though the .frm file for the table exists. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting.html for how you can resolve the problem.
20:48:29 localhost mysqld: 150813 20:48:29 [Warning] Failed to load slave replication state from table mysql.gtid_slave_pos: 1932: Table 'mysql.gtid_slave_pos' doesn't exist in engine
20:48:29 localhost mysqld: 150813 20:48:29 [Note] Server socket created on IP: '0.0.0.0'.
20:48:29 localhost mysqld: 150813 20:48:29 [Warning] 'user' entry 'root@avc2-wus1-db04' ignored in --skip-name-resolve mode.
20:48:29 localhost mysqld: 150813 20:48:29 [Warning] 'proxies_priv' entry '@% root@avc2-wus1-db04' ignored in --skip-name-resolve mode.
20:48:29 localhost mysqld: 150813 20:48:29 [Note] Event Scheduler: Loaded 0 events
20:48:29 localhost mysqld: 150813 20:48:29 [Note] WSREP: Signalling provider to continue.
20:48:29 localhost mysqld: 150813 20:48:29 [Note] WSREP: SST received: 00000000-0000-0000-0000-000000000000:2
20:48:29 localhost mysqld: 150813 20:48:29 [ERROR] WSREP: Application received wrong state:
20:48:29 localhost mysqld: #011Received: 00000000-0000-0000-0000-000000000000
20:48:29 localhost mysqld: #011Required: 76a27c1d-41f1-11e5-846a-73bafbb95904
20:48:29 localhost mysqld: 150813 20:48:29 [ERROR] WSREP: Application state transfer failed. This is unrecoverable condition, restart required.
20:48:29 localhost mysqld: 150813 20:48:29 [Note] WSREP: Closing send monitor...
20:48:29 localhost mysqld: 150813 20:48:29 [Note] WSREP: Closed send monitor.
20:48:29 localhost mysqld: 150813 20:48:29 [Note] WSREP: gcomm: terminating thread
20:48:29 localhost mysqld: 150813 20:48:29 [Note] WSREP: gcomm: joining thread
20:48:29 localhost mysqld: 150813 20:48:29 [Warning] InnoDB: Cannot open table mysql/gtid_slave_pos from the internal data dictionary of InnoDB though the .frm file for the table exists. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting.html for how you can resolve the problem.
20:48:29 localhost mysqld: 150813 20:48:29 [Warning] Failed to load slave replication state from table mysql.gtid_slave_pos: 1932: Table 'mysql.gtid_slave_pos' doesn't exist in engine
20:48:29 localhost mysqld: 150813 20:48:29 [Note] WSREP: gcomm: closing backend
20:48:29 localhost mysqld: 150813 20:48:29 [Note] Reading of all Master_info entries succeded
20:48:29 localhost mysqld: 150813 20:48:29 [Note] Added new Master_info '' to hash table
20:48:29 localhost mysqld: 150813 20:48:29 [Note] /usr/sbin/mysqld: ready for connections.
20:48:29 localhost mysqld: Version: '10.0.20-MariaDB-1~trusty-wsrep-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution, wsrep_25.10.r4144
20:48:30 localhost mysqld: 150813 20:48:30 [Note] WSREP: view(view_id(NON_PRIM,620d451d,4) memb {
20:48:30 localhost mysqld: #0118e8c1fdb,0
20:48:30 localhost mysqld: } joined {
20:48:30 localhost mysqld: } left {
20:48:30 localhost mysqld: } partitioned {
20:48:30 localhost mysqld: #011620d451d,0
20:48:30 localhost mysqld: })
20:48:30 localhost mysqld: 150813 20:48:30 [Note] WSREP: view((empty))
20:48:30 localhost mysqld: 150813 20:48:30 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
20:48:30 localhost mysqld: 150813 20:48:30 [Note] WSREP: gcomm: closed
20:48:30 localhost mysqld: 150813 20:48:30 [Note] WSREP: Flow-control interval: [16, 16]
20:48:30 localhost mysqld: 150813 20:48:30 [Note] WSREP: Received NON-PRIMARY.
20:48:30 localhost mysqld: 150813 20:48:30 [Note] WSREP: Shifting JOINER -> OPEN (TO: 2)
20:48:30 localhost mysqld: 150813 20:48:30 [Note] WSREP: Received self-leave message.
20:48:30 localhost mysqld: 150813 20:48:30 [Note] WSREP: Flow-control interval: [0, 0]
20:48:30 localhost mysqld: 150813 20:48:30 [Note] WSREP: Received SELF-LEAVE. Closing connection.
20:48:30 localhost mysqld: 150813 20:48:30 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 2)
20:48:30 localhost mysqld: 150813 20:48:30 [Note] WSREP: RECV thread exiting 0: Success
20:48:30 localhost mysqld: 150813 20:48:30 [Note] WSREP: recv_thread() joined.
20:48:30 localhost mysqld: 150813 20:48:30 [Note] WSREP: Closing replication queue.
20:48:30 localhost mysqld: 150813 20:48:30 [Note] WSREP: Closing slave action queue.
20:48:30 localhost mysqld: 150813 20:48:30 [Note] WSREP: /usr/sbin/mysqld: Terminated.
20:48:30 localhost mysqld_safe: Number of processes running now: 0
20:48:30 localhost mysqld_safe: WSREP: not restarting wsrep node automatically
20:48:30 localhost mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended
If you have any ideas I would really appreciate it.
Thanks, Aaron
