-1

There are many ways to force a secondary node to become a primary when majority nodes are down in mongo replica set. But we need to do this manually. For eg, forcing reconfig: cfg = rs.conf(); cfg.members = [cfg.members[1]]; rs.reconfig(cfg, {force : true}); How can we automate this task to run in the background which kind of checks for the heartbeat and automatically promotes secondary to primary.

  • First, why do you want to have a primary when majority of nodes are done? And what is the configuration change you are doing to make any remaining nodes become primary? What is your replica set configuration? This will help to understand your problem. – Mani Mar 9 at 5:52
  • 1
    You should design your replica set with appropriate fault tolerance to allow automatic failover. Forced reconfiguration is a last resort option which can lead to data loss if used incorrectly. Per the docs: "Use this procedure only to recover from catastrophic interruptions. Do not use force every time you reconfigure. Also, do not use the force option in any automatic scripts and do not use force when there is still a primary." – Stennie Mar 9 at 6:15
  • Yes, @Stennie I'm only wanting to force it during a catastrophic situation, but is there a way I can automate this using Python, Java where it pings the nodes at intervals and only if there is one node, then to force set it to primary? – sushruth r Mar 9 at 22:17
  • @Mani, I'm just trying to do this in case of catastrophic events. I have 3 replica nodes and one of them is primary when the primary goes down, the nodes call for an election and appoint another node as primary for writes. But when this newly set primary goes down, it cannot conduct an election since it needs minimum votes and therefore I'm force setting the only node as primary. I just want to automate the process. – sushruth r Mar 9 at 22:22
  • If a majority of your replica set members are unavailable I would strongly encourage having alerts to investigate rather than automating. The reconfig command is easy to run; automating it is dangerous and may lead to data loss or more significant recovery problems if action is triggered inappropriately. For example: in the event of a network partition you can force configure multiple primaries which cannot be automatically reconciled when the partition heals without rolling back data to a common point. Automation is appropriate for common scenarios; catastrophic ones demand human intervention – Stennie Mar 9 at 23:27
0

Please find below the official documentation from MongoDB.

Reconfigure by Forcing the Reconfiguration

This procedure lets you recover while a majority of replica set members are down or unreachable. You connect to any surviving member and use the force option to the rs.reconfig() method.

The force option forces a new configuration onto the member. Use this procedure only to recover from catastrophic interruptions. Do not use force every time you reconfigure. Also, do not use the force option in any automatic scripts and do not use force when there is still a primary.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.