I deployed Redis replication cluster on 2 data centers (ping time between 2 DCs is around 1ms).

I used Master (Node1 on DC 1) - Slave (Node2 in DC2) replication model and Sentinel to control promoting or demoting. I faced some scenarios with issues of network and VMs (all Redis nodes using virtual machine)

  1. Node 1 (Master) has network interface issue, all nodes cannot reach Node1, Sentinel promote Node2 to Master. After 1-2 minutes, Node1 back normally, what happens in this case? (Node1 Master and Node 2 Master also)
  2. Node 1 (Master) down, Sentinel promote Node2 to Master. After 2 hours, Node1 back, when Node1 synchronizes data from Node2, suddenly Node2 down. What happens with Node1? (assume node2 cannot back totally)
  3. Node 1 (Master) and Node 2 (Standby) in turn down after 5 minutes. Ex: at 10:00 am, Node1 down, at 10:05 am Node2 down, node1 back. At 10:10am, Node1 down, node2 back and so on... What happens?

Case 1 and 2 are reality. Case 3 is my assumption when node1 and node2 unstable often. My solution for 3 cases is using script manual promote or demote and does not use Sentinel because I don't believe it can process complex cases.

Could you give me advice on my cases?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.