Is it possible to run an arbiter on a machine that also hosts a secondary member?

Question

I have two machines (it is a restriction) configured as a replica set with MongoDB 3.6.5.

Is it possible to run an arbiter on a machine that also hosts a secondary member?

Even if it is not advisable, but if it is possible, how can I install an arbiter on the secondary server for when the primary goes down and the secondary becomes the primary?

Once installed, how can I configure the replica set for when the original primary returns and make a copy of the secondary (do not use the replication)?

Answer 1

Is it possible to run an arbiter on system that also hosts a secondary member?

There is no technical restriction on running multiple processes on the same host, as long as they are listening to different port numbers. For example, you could start an arbiter listening to port 27018 (instead of the default 27017). However, you do need to consider the operational implications of colocating processes.

Even if it is not advisable, but if it is possible, how can I install an arbiter on the secondary server for when the primary go down the secondary become the primary?

This is the standard replica set behaviour. Assuming you have a preferred primary (which will have a higher priority value), your replica set configuration would look similar to:

{
   _id : "myreplset",
   members: [
      { _id: 0, host: "machine1.example.net:27017", priority: 2 },
      { _id: 1, host: "machine2.example.net:27017" },
      { _id: 2, host: "machine2.example.net:27018" }
   ]
}

With three voting members in your replica set, the strict majority required to elect (or maintain) a primary is two votes. If machine1 is unavailable, the members on machine2 can still elect a primary.

However, because machine2 has the majority of the voting members there are a few operational consequences:

• If machine2 is unavailable, machine1 cannot elect or sustain a primary without manual intervention to force reconfigure the replica set.
• In the event of a network split, machine2 will become primary.

If the arbiter (or another voting member) was on a third machine, both of these scenarios would be avoided.

You may want to consider colocating the arbiter on machine1 , so in the event of a network split the primary will remain on machine1 (reversing the operational scenarios noted above so that failover to machine2 would be a manual reconfiguration).

For a production deployment I would also recommend using a third secondary (instead of an arbiter). When your Primary-Secondary-Arbiter deployment is down a member (so effectively Primary-Arbiter) there is no longer any active replication and your deployment will not be able to acknowledge majority writes. It is definitely not ideal to have multiple data-bearing members competing for resources on the same host machine, but data redundancy and availability of majority writes is often a more important factor.

Once installed, how can I configure the replica set for when the original primary returns and make a copy of the secondary?

Assuming you have configured the member on machine1 with a higher priority as per the above example, the expected outcome is that when machine1 is available again it will automatically resume syncing from the current primary on machine2 (as a secondary). Once the member on machine1 catches up and is eligible to be primary, it will initiate an election based on a higher priority member being available (referred to as a priority takeover election).

Answer 2

I have two machines (it is a restriction) configured as a replica set with MongoDB 3.6.5. Is it possible to run an arbiter on a machine that also hosts a secondary member?

As I am not discarding anything of @Stennie but I would like to say that as per MongoDB documentation here The minimum architecture of a replica set has three members. A three member replica set can have either three members that hold data, or two members that hold data and an arbiter.

Even if it is not advisable, but if it is possible, how can I install an arbiter on the secondary server for when the primary goes down and the secondary becomes the primary? Once installed, how can I configure the replica set for when the original primary returns and make a copy of the secondary (do not use the replication)?

First things why you want to do that which is not a advisable, As per MongoDB blog documentation here Consider you have 2 MongoDB nodes. One act as primary and the other once act as secondary node. If any one of the node goes down then MongoDB cannot able to choose a primary node by itself because MongoDB needs majority of members (at least two active nodes) to choose a primary node in a multi node replica set. As a result of this all write operations are failed and a lock occurs. In this scenario if an arbiter exists then the arbiter will vote for the available node to become primary.

Necessity of Arbiter:

When a replica set is initiated (or) when a primary node goes down then MongoDB will perform an election process to determine a new primary node. All the secondary nodes and arbiter node present in a replica set can participate in the election. All write operations are done in the primary node. So when failover happens, the election mechanism will select a new primary node within seconds, and restore the write capability without any manual intervention. An odd number of members ensures that the replica set is always able to elect a primary. If you have an even number of nodes add an arbiter to get an odd number.

The following table will illustrate the majority of members/nodes required for a given replica set

Number of Members   Majority Required to Elect a New Primary    Fault Tolerance
3                                        2                        1
4                                        3                        1
5                                        3                        2
6                                        4                        2

For your further ref here, here and here