4

I managed to install and configure MySQL Fabric that manages multiple MySQL Server nodes. It works very well since I tested it by connecting to the Master node directly and played with the data (insert, delete). The slave nodes become identical to the master.

Now, the fabric server should manage everything. Hence, the application should connect to the Fabric and not to the master node. This way, when the Master fails, Fabric will nominate a new Slave.

Q1: How can I avoid Fabric server itself from becoming a single point of failure?

Q2: Anyone has experience implementing Fabric? Is what I am planning to do is the best practice?

Regards,

Improve this question

5 Answers 5

Reset to default
4

The MySQL Fabric node itself is not currently redundant but as stated in the MySQL Fabric FAQ:

There is currently only a single instance of the MySQL Fabric node. If that process should fail then it can be restarted on that or another machine and the state and routing information read from the existing state store (a MySQL database) or a replicated copy of the state store.

While the MySQL Fabric node is unavailable, Fabric-aware connectors continue to route queries and transactions to the correct MySQL Servers based on their cached copies of the routing data. However, should a Primary fail, automated failover will not happen until the MySQL Fabric node is returned to service and so it's important to recover the process as quickly as possible.

In terms of best practices, this MySQL Fabric blog post provides an end-to-end worked example of using MySQL Fabric and then the MySQL Fabric white paper adds more of an overview and context information.

Finally, you can watch the MySQL Fabric webinar replay and read Q&A transcript - there are some good, technical questions + responses there.

If you're looking for more discussion on the topic then there's also a dedicated forum for MySQL Fabric, Sharding, HA & Utilities.

Improve this answer
1

I know this is a bit old, but I was looking into setting up some clusters with MySQL Fabric and ran into the same question.

The best answer I have found is to use Pacemaker and Corosync. After deciding on that I - luckily - found that the MySQL Utilities Manual already has a section on how to set up exactly that.

http://dev.mysql.com/doc/mysql-utilities/1.5/en/fabric-pacemaker.html

Improve this answer
1

Just found this question while looking for answers on a related topic. What I've done is to set up the MySQL Fabric controller as a pair of servers with multi-master replication and a daemon that looks for a heartbeat on the first server's mysqlfabric process and restarts it, starting the second server's mysqlfabric in case the first refuses to restart.

Why it works:

  • All Fabric information is stored in the Fabric controller database, so replicating the database provides redundancy for Fabric.
  • Active shard information is cached in the connector, so when the controller fails you've basically lost management, failure detection, and the ability to add new sessions, but not the ability of running sessions to continue so long as no configuration changes happen.
  • Restarting or firing up the 2nd instance of the Fabric controller (on the 2nd machine) takes just a few seconds, so the hiccup can be almost imperceptible to a user, so long as the application knows what to try.

What does not work:

  • You'll lose the transactions while it's failing if a server fails also. That is, connectors cache the sharding and primary server information, but they won't know if that information changes. This may sound a little over-cautious, but think about it: the fabric controller is not the most stable process, but there are many situations where it might fail because of a network switch problem or something like that which might also take out a data server in your pool.

What's the cost:

  • Manual intervention required. (You don't want the first instance restarting after the 2nd instance has taken over - It's not immediately catastrophic, but the potential for disaster if either mysqlfabric instance initiates any management such as promoting a server while both are running, for example, is high.)
  • Double the permissions masks to manage - at the mysql server level (mysqlfabric) this is transparent, but, for example, if you run hard servers with the firewall only allowing traffic from specific ip:port combinations, you'll need to match access on the Fabric servers.
Improve this answer
1

Use DRBD Replication to enable active-passive MySQL Fabric. I've been using 2 MySQL Fabric to manage my database farms. In any event that the Active Fabric fails, it will automatically switch to the other Fabric Instance using the replicated data in my logical drive.

I'm using the following for this setup:

  • DRBD
  • Pacemaker
  • Corosync
Improve this answer
0

DISCLAIMER : MySQL Fabric is new to me so I cannot say much.

Notwithstanding, the FAQ for MySQL Fabric says the following:

8.7.8: Is the MySQL Fabric node itself fault tolerant? What happens when the MySQL Fabric node is not available?

There is currently only a single instance of the MySQL Fabric node. If that process should fail then it can be restarted on that or another machine and the state and routing information read from the existing state store (a MySQL database) or a replicated copy of the state store.

While the MySQL Fabric node is unavailable, Fabric-aware connectors continue to route queries and transactions to the correct MySQL Servers based on their cached copies of the routing data. However, should a Primary fail, automated failover will not happen until the MySQL Fabric node is returned to service and so it's important to recover the process as quickly as possible.

In contrast, MySQL Cluster has the technology to allow multiple management nodes. You will have to look for multiple-instance usage of MySQL Fabric in future releases. That would have to be on Oracle's roadmap.

VERY WILD SUGGESTION

If you use VMWare or Amazon EC2, you need to make an OS instance with the following:

  • MySQL Fabric installed
  • MySQL Fabric configure to see all MySQL Servers

Then, clone the VM multiple times and spin up an instance on demand.

EPILOGUE

It is not best practice to attach directly to a Master in a MySQL Fabric farm. Why? Suppose MySQL Fabric comes back and elects another MySQL Server to be the new Master. If the Master you are attached to becomes a Slave, MySQL Fabric could possibly make the Master become read-only. This may also confuse MySQL Fabric because it may not be aware of your independent usage of the Master. In essence, you will rip the Fabric and have to sew it all back together then.

UPDATE 2016-03-18 18:20 EDT

I tried out MySQL Fabric for a week and quickly learned something. There are intermittent times when the Fabric node dies. You can simply kill the Fabric service and start it back again. I already mentioned in from the Documentation, but I actually saw it in practice. Fabric-aware connectors do indeed work but DO NOT LET THE FABRIC NODE STAY DEAD FOR LONG.

Improve this answer
1
  • 2
    When MySQL Fabric comes back, it should not change the current situation unless the master have failed (which the application will notice anyway). You should be safe writing to the master in the group while MySQL Fabric node is away. If there is any problems here, please report a bug at bugs.mysql.com
    – Mats Kindahl
    Commented Jun 28, 2014 at 8:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.