4

We have a galera cluster with 3 nodes, on 3 different physical machines but all located in the same datacenter.

From what I understood, the reason they deployed this in the past was to increase availability and reliability, DC failures are not a concern. Each node is installed on a VM using 12 cores and 4Gb RAM. (from the monitoring I did we can decrease the amount of cores used to 4)

We are asked to migrate this to Google Cloup Platform in order to get rid of the ops tasks. I could create 3 compute engine instances and deploy the galera cluster, GCP even have a Percona XtraDB Cluster one click deployment service, but I have difficulties to see the added value compared to Cloud SQL instances with replication and backup. I am not very familiar in scaling heavy load systems.

The db hosted in these nodes is kind of critical and should ensure maximum availability and reliability.

What strategy should I adopt in order to migrate this architecture to GCP ?

Improve this question

1 Answer 1

Reset to default
6

but I have difficulties to see the added value compared to Cloud SQL instances with replication and backup. I am not very familiar in scaling heavy load systems.

Galera-based clusters like Percona XtraDB Cluster support true active/active multi-master, so failover is seamless because you can actually be writing on any node at any time (tho it's recommended to write only on a single node to avoid performance issues due to lock-conflicts that can happen due to the optimistic locking strategy used by Galera).

This type of cluster can also do distributed fully-synchronous reads so you can scale heavy read loads without problems; One shining example is Magento, which is read-heavy and has many reads that are critical -think money- and traditionally needed to be done on master to guarantee consistency, and here they can be distributed.

Keep in mind that as with any distributed CP system, Galera-cluster will add latency to the writes as it has to verify the write is valid on the other nodes, and hence it's not designed to scale writes; That said: it can usually withstand same write load than a regular master, as long as you keep transactions small (in terms of amounts of rows affected) and short (in regards to time spent by active transaction holding locks).

Also make sure to read list of limitations here: https://www.percona.com/doc/percona-xtradb-cluster/LATEST/limitation.html

What strategy should I adopt in order to migrate this architecture to GCP ?

To decrease downtime you can migrate by setting up the GCP cluster as an asynchronous slave of the current cluster (yes, Galera cluster nodes can act as a master and/or slave in traditional, binlog based, asynchronous replication). To prime the slave cluster you can use XtraBackup, which can take fully lock-less backups from current cluster (make sure to use latest 2.3 XtraBackup) which you can then restore on one of the nodes and allow the other two nodes to perform Snapshot State Transfer. Then simply designate one of the nodes to become the slave, and start replication using binlog coordinates from xtrabackup_binlog_info.

Overview of steps would be:

  1. Enable log-bin, log-slave-updates, server_id on one of the nodes from the current cluster

  2. Take a backup of that node using xtrabackup (make sure to desync the node before: https://www.percona.com/blog/2013/10/08/taking-backups-percona-xtradb-cluster-without-stalls-flow-control/).

  3. Bring up one of the nodes in GCP cluster and restore the backup into it, and bootstrap the cluster with this node. Most likely use cloud storage to feed the backup to the cluster.

  4. Bring up the other two nodes in GCP, one at a time and allow them to perform SST

  5. Setup one of the GCP cluster nodes as async slave to the current cluster (this node also requires log-bin, log-slave-updates, server-id); this node will feed the other nodes with updates

  6. Failover to new cluster once the slave node is caught up

Hope that helps! :)

Full disclosure: am a member of Percona Support team.

Improve this answer
2
  • Thanks a lot for this detailed answer. If I don't need to scale my writes, only my reads, and therefore no need for master replicas, only slaves, then a cloud sql replica is a as safe option as the cluster, right?
    – Vincent Teyssier
    Jan 29, 2017 at 7:38
  • 1
    Well, traditional async replication (Cloud SQL replica) would certainly work for scaling reads, as long as you don't have critical reads that require to run on master to guarantee data consistency. If you have many of those then the cluster will scale better, as those can also be distributed and performed fully-syncrhonous with writes.
    – MarcosAlbe
    Jan 30, 2017 at 16:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.