If I understand this correctly, you have 6 nodes. 3 in DC1 (3 data nodes) and 3 in DC2 (2 data and 1 arbiter). All are connected together forming a 6 node cluster. If that assumption is correct, the cluster behaves as expected. Galera requires a majority of the cluster to be available in order to operate. In the case of 6 nodes, that would have to be 4 nodes (4 data or 3 data + 1 garbd) to be connected. If you have 3 and 3, while cutting the network in between, you have exactly 50% - 50% split and both partitioned parts will switch to non-Primary. There’s no other way in this scenario.

The solution for an automated failure handling is to have 3DC. Then you can build topology like: DC1: 3 data nodes, DC2: 3 data nodes, DC3 1 garbd (arbitrator) node. It that case, if there’s a failure of a single datacenter, at the minimum you have 4 out of 7 nodes, keeping it a majority. In that case 4 nodes will stay in Primary state, serving the traffic.

If I understand this correctly, you have 6 nodes. 3 in DC1 (3 data nodes) and 3 in DC2 (2 data and 1 arbiter). All are connected together forming a 6 node cluster. If that assumption is correct, the cluster behaves as expected. Galera requires a majority of the cluster to be available in order to operate. In the case of 6 nodes, that would have to be 4 nodes (4 data or 3 data + 1 garbd) to be connected. If you have 3 and 3, while cutting the network in between, you have exactly 50% - 50% split and both partitioned parts will switch to non-Primary. There’s no other way in this scenario.

The solution for an automated failure handling is to have 3DC. Then you can build topology like: DC1: 3 data nodes, DC2: 3 data nodes, DC3 1 garbd (arbitrator) node. It that case, if there’s a failure of a single datacenter, at the minimum you have 4 out of 7 nodes, keeping it a majority. In that case 4 nodes will stay in Primary state, serving the traffic.

EDIT/ADDITION/CLARIFICATION

I think you are missing the point as to why bootstrapping the cluster in DC2 does not make any sense. Let’s take a look at it step by step visually.

This is a normal scenario - transactions are written to both DC1 and DC2 and are distributed across all nodes in the cluster. Every node will have transactions T1, T2, T3, Z1, Z2 and Z3.

Now, let’s say you don’t have a connection between DC1 and DC2. If you bootstrap DC2, you have a separate 2 node cluster there. You can execute writes which otherwise would not be possible. So, we have transactions T11, T22 and T33 in DC1 and Z11, Z22 and Z33 in DC2

There is no syncing of data, those are two separate entities. Cluster in DC1 stores transactions T11, T22 and T33. Cluster in DC2 stores transactions Z11, Z22 and Z33.

Now, if you want to resync DC2 with DC1 what will happen?

Data on nodes in DC2 will be removed and data from donor in DC1 will be transferred.

As a result, transactions Z11, Z22 and Z33 are gone forever, all nodes only have T11, T22 and T33. As you can imagine, this is bad. In this scenario it does not make any sense to even use DC2 as whatever you’ll write there will be gone anyway. You can as well stick to just a single datacenter.

The whole point of having 3 DC is that if DC1 is unavailable, it goes non-Primary and you do not touch that. It does not accept writes. Other datacenters do. Your application can still work based on the remaining datacenters. It can read the data, it can write the data. Once the DC1 becomes available again, it gets synced from other nodes, which were receiving traffic. As DC1 was not writeable, you don’t have to worry that some data will be lost - if you could not write to DC1, you can’t loose anything on SST. You do not have to touch anything, you only have to configure your application (or load balancer) to track the status of the nodes and if node does not respond or respond that it is in non-Primary state, you just connect to another node that’s available. In the case of 3DC, as long as nodes in DC2 and DC3 are up and running, there will always be nodes available to write to.

If I understand this correctly, you have 6 nodes. 3 in DC1 (3 data nodes) and 3 in DC2 (2 data and 1 arbiter). All are connected together forming a 6 node cluster. If that assumption is correct, the cluster behaves as expected. Galera requires a majority of the cluster to be available in order to operate. In the case of 6 nodes, that would have to be 4 nodes (4 data or 3 data + 1 garbd) to be connected. If you have 3 and 3, while cutting the network in between, you have exactly 50% - 50% split and both partitioned parts will switch to non-Primary. There’s no other way in this scenario.

The solution for an automated failure handling is to have 3DC. Then you can build topology like: DC1: 3 data nodes, DC2: 3 data nodes, DC3 1 garbd (arbitrator) node. It that case, if there’s a failure of a single datacenter, at the minimum you have 4 out of 7 nodes, keeping it a majority. In that case 4 nodes will stay in Primary state, serving the traffic.

If I understand this correctly, you have 6 nodes. 3 in DC1 (3 data nodes) and 3 in DC2 (2 data and 1 arbiter). All are connected together forming a 6 node cluster. If that assumption is correct, the cluster behaves as expected. Galera requires a majority of the cluster to be available in order to operate. In the case of 6 nodes, that would have to be 4 nodes (4 data or 3 data + 1 garbd) to be connected. If you have 3 and 3, while cutting the network in between, you have exactly 50% - 50% split and both partitioned parts will switch to non-Primary. There’s no other way in this scenario.

The solution for an automated failure handling is to have 3DC. Then you can build topology like: DC1: 3 data nodes, DC2: 3 data nodes, DC3 1 garbd (arbitrator) node. It that case, if there’s a failure of a single datacenter, at the minimum you have 4 out of 7 nodes, keeping it a majority. In that case 4 nodes will stay in Primary state, serving the traffic.