The main question is the following: What is the best way to restore the backup?
If you want to restore a
mongodump backup into a replica set, you need to
mongorestore into the current primary. Data will be replicated to other replica set members via the normal means. You can use
mongorestore --drop ... to drop each collection (if it already exists) before import, but in a full recovery scenario you would normally be rebuilding the replica set starting from an empty deployment. As at MongoDB 3.6,
mongorestore only inserts documents - it will not drop any existing databases or upsert/update/replace any existing documents.
Q1: I have to follow these steps?
The only applicable step in this list is "Restore backup to primary". With
mongodump you are exporting data and index definitions, and must
mongorestore into the primary to rebuild the data files. Your secondaries must be online while you are restoring and be able to keep up via replication. You can use the
--writeConcern majority option to wait for a writes to be acknowledged by a majority of the replica set and ensure your restore doesn't outpace replication.
Q2: Restore the backup on primary without stopping any of the servers?
This is the correct approach.
Q3. Make secondary as primary and restore the backup to secondary ( previous primary), and make it primary.
A replica set can only have one primary, so you wouldn't gain anything by swapping the roles of the current primary and secondary. The data still needs to be replicated.
Q4: Is there is any way to restore both the server? Please suggest the best approach to restore MongoDB.
You have to restore the
mongodump into the primary in order to build the data files. If you are looking for alternatives to replicating the data to each of your secondaries: you could
mongorestore into a single node replica set, stop the primary, and then copy the data files to seed new secondaries and avoid initial sync.
There are certainly much better ways to take a backup, especially if your objective is to have faster (and less impactful) backup & recovery times. Backing up a large deployment with
mongodump can have a significant impact on performance (particularly if your data set is much larger than available RAM) and has a longer time to restore because all of the data files and indexes need to be rebuilt.
For a general comparison of backup strategies see Backup and its Role in Disaster Recovery. The MMS product referenced in this comparison is now known as MongoDB Cloud Manager, but the general considerations and comparison points are still applicable.