A MongoDB replica set should have at least 3 members in order to help you achieve high availability.
The rule for MongoDB replica set failover (when a new member becomes a primary) is that the MAJORITY of the "surviving" nodes vote for it to become the new primary.
So let's think about this:
- If you have a replica set of one member, and that member fails, there is no other node to become primary.
- If you have a replica set with 2 members, if one of the members fail, there is only one "surviving" member. The total number of members in the replica is 2 and there is only 1 survivor. Majority means MORE than half, but 1 survivor is just half of the nodes. Since there is no majority, the only survivor node, will stay as a secondary and consequently be read-only until the other node recovers.
- If you have a replica set with 3 members, if one of the members fail, there are 2 surviving nodes. If the survivor nodes can "talk" via the network, the election algorithm will ensure that one of them becomes the "new" primary.
If for any reason you are not interested in investing in a 3rd server for your MongoDB replica set, you can consider using an arbiter. The arbiter can be a very small machine, and it requires minimum amounts of CPU and storage. But an arbiter votes in an election and it is counted as one of the nodes.
Look at this link in the MongoDB documentation: [https://docs.mongodb.org/v3.0/core/replication-introduction/#automatic-failover][1]
When a primary does not communicate with the other members of the set
for more than 10 seconds, an eligible secondary will hold an election
to elect itself the new primary. The first secondary to hold an
election and receive a majority of the members’ votes becomes
primary.
In a situation where you have 2 "regular" members of a MongoDB replica set, plus an arbiter, in case one of the members fails, the replica and the surviving node, will form a MAJORITY which is needed for the election of a new primary.
[https://docs.mongodb.org/v3.0/core/replica-set-arbiter/][1]
nms02
is listed with priority=2, which is why it becomes primary when it joins the replicaset. Also, do you have a reason to prefer one member over another as PRIMARY?