MariaDB Galera cluster node won't start

Question

I have configured a MariaDB Galera cluster with three nodes. The master node and one slave-master node is up and running, but the third node just won't run. The system specs and cluster config are the same on all three nodes (except changing the wsrep_node_address and the wsrep_node_name on each node).

Here is the config for the nodes (wsrep_node_address and the wsrep_node_name are unique on each node):

[galera]
binlog_format=ROW
default-storage-engine=innodb
innodb_autoinc_lock_mode=2
innodb_locks_unsafe_for_binlog=1
query_cache_size=0
query_cache_type=0
bind-address=0.0.0.0

datadir=/var/lib/mysql
innodb_log_file_size=100M
innodb_file_per_table
innodb_flush_log_at_trx_commit=2

wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_cluster_address="gcomm://192.168.1.111,192.168.1.112,192.168.1.113"
wsrep_cluster_name='galera_cluster'
wsrep_node_address='192.168.9.113'
wsrep_node_name='george-db4'
wsrep_sst_method=xtrabackup
wsrep_sst_auth=username:password

It seems like MariaDb does not like the config

wsrep_cluster_address="gcomm://192.168.1.111,192.168.1.112,192.168.1.113"

When I comment it out MariaDB restarts just fine. When I uncomment it, it just displays this message:

Starting MySQL................................... ERROR! 
ERROR! Failed to restart server.

When I change the wsrep_cluster_address to just

wsrep_cluster_address="gcomm://

MariaDb runs with no problem. So I am not sure why it is having problems with assigning the IP address (even if it works fine on the other two nodes). I have tried emptying the galera.cache file, I have tried re-installing the cluster, I have tried rebooting the machine etc. and nothing works so far. I even truncated the log file /var/lib/mysql/db1.err with the command truncate logfile --size 0 (now I cannot figure out how to get my log file back with space as I realized I would need the log file).

Answer 1

Please post the tail end of your MySQL error log. Usually, it will be in /var/log/syslog or /var/log/mysql/mysql.log (depending on your setup).

Without having some details, it is hard to say, but it almost sounds like you have network connectivity problems. Please ensure that the necessary ports are open on the third node to properly allow for all Galera traffic.

Answer 2

SELinux and I think AppArmour on Debian-based Linux distros will prevent MariaDB Galera Cluster from running correctly since MariaDB doesn't yet provide policies for these. So make sure to put SELinux into permissive mode or disable it completely. (And similar if you have AppArmour.)