Can't add a node to PXC due to SST error

Question

I've successfully bootstrap a Cluster (mysqld Ver 5.6.30-76.3-56 for debian-linux-gnu on x86_64 (Percona XtraDB Cluster (GPL), Release rel76.3, Revision aa929cb, WSREP version 25.16, wsrep_25.16)).

When starting a second node to join this cluster there is the following error:

2016-07-29 09:31:58 12359 [Note] WSREP: New cluster view: global state: 3f3843a3-54aa-11e6-af38-5b145f899741:195614, view# 4: Primary, number of nodes: 2, my index: 1, protocol version 3
2016-07-29 09:31:58 12359 [Warning] WSREP: Gap in state sequence. Need state transfer.
2016-07-29 09:31:58 12359 [Note] WSREP: Running: 'wsrep_sst_xtrabackup-v2 --role 'joiner' --address 'XYZ.40.39.13' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --defaults-group-suffix '' --parent '12359'  '' '
2016-07-29 09:31:58 12359 [ERROR] execlp() failed: Permission denied
2016-07-29 09:31:58 12359 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_xtrabackup-v2 --role 'joiner' --address 'XYZ.40.39.13' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --defaults-group-suffix '' --parent '12359'  '' 
    Read: '(null)'
2016-07-29 09:31:58 12359 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address 'XYZ.40.39.13' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --defaults-group-suffix '' --parent '12359'  '' : 1 (Operation not permitted)
2016-07-29 09:31:58 12359 [ERROR] WSREP: Failed to prepare for 'xtrabackup-v2' SST. Unrecoverable.
2016-07-29 09:31:58 12359 [ERROR] Aborting

There is no special config, just:

wsrep_provider                  = /usr/lib/libgalera_smm.so
wsrep_cluster_name              = "legacy_cluster"
wsrep_cluster_address           = "gcomm://ZYX.4.123.242,XYZ.40.39.13"
wsrep_node_name                 = legacy6
wsrep_node_address              = XYZ.40.39.13

wsrep_sst_method                = xtrabackup-v2
wsrep_sst_auth                  = sst:secret

I've also tried rsync, but got the same error. I'm not sure how to further debug this error!?

Answer 1

Finally I figured out that AppArmor was blocking the SST. The most resources about setting up the Cluster I read were about SE Linux.

I put the mysql profile into the complain mode: https://blogs.oracle.com/jsmyth/entry/apparmor_and_mysql