2

We have a 2 node (plus an arbiter) Percona XtraDB cluster. It is used for data indexing/archiving. It has it's own database called archive but it is also the replication slave for a database called primary. This comes from a standard master->slave Percona Server pair.

One of the clustered servers segfaulted (reason still being investigated...). This node was, specifically, the one that was doing replication. On restarting it, it deleted the contents of the MySQL data directory (/var/lib/mysql/) and did an SST from the other node. This went fine, but it now has a partially incomplete dataset on the primary database and no master binlog position information.

How should replication be restored?

  • It is not feasible to export the primary database as a mysql dump since it is circa 1TB in size when looking at the raw innodb files
  • If we export the primary DB with xtrabackup and prepare it as a whole on the clustered node, it is assumed it will get deleted as soon as the node starts SST
  • It is difficult but not infeasible to allow the node to do SST and then restore an xtrabackup of primary DB table by table... but it will take a while given 500+ tables

So, the question is... What is the correct way to set up replication into a multi-master cluster?

Improve this question

1 Answer 1

Reset to default
0

I suspect you are 'out of luck'.

Was there some reason not to have 'primary' part of the cluster?

Look into pt-table-checksum and pt-table-sync. These may be a way to incrementally 'fix' the Slave to get it synced with Primary.

Improve this answer
1
  • Rebuilding replicated into a cluster seemed like it'd have a lot of downtime so we are waiting for budget for new hardware before we cluster those servers. Thoug, even then, we will probably still replicate between 2 clusters as the archive database isn't required by services which access the primary db.
    – phil-lavin
    Commented May 26, 2016 at 19:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.