I am looking for a way that I can backup my 12 MongoDB shards into a single server. Disk space is not an issue but getting the backups to be on a single node would be the best. I am not sure if there is a way to run more than one MongoDB server on a single machine or if there is a way to have all 12 shards merged into a single MongoDB instance. I am running on AWS and will have the primary data in US-East but want the backups in US-West. I dont mind if there is lag on the backup itself, but trying to avoid the app servers for waiting for the backup to be written.
- Anybody can ask a question
- Anybody can answer
- The best answers are voted up and rise to the top
You need to be a little more clear about your requirements:
You can approach this from many different ways, but I will cover the most common approaches:
Put secondaries in US-West
This approach means you have a full, complete copy of your databases on the west coast. You can make them hidden and priority 0 to make sure they don't serve slaveOK reads or become primary by accident. To make them primary (in a disaster recovery scenario) you will need to re-configure the set manually if they are hidden and priority 0.
This is quite an easy approach, though it could be expensive. You can run multiple mongod processes on a single host but they will be competing for resources and it will still have to be fast enough to keep up with the primaries on the east coast.
Use EBS snapshots
Assuming you are running with journaling enabled, you can just take an EBS snapshot of your data folder (dbpath) and you can back that up to S3 or use it to "seed" hosts on the west coast.
Generally I would recommend a combination of the first two in order to combine the best resilience options with the most flexibility.
Ideally you would have one or more replica set. Within the replica set I suggest you always have odd numbers for the number of members with one node that is hidden locate in us west. To do so into a three members replica set you can use something like this:
If you keep this sort of config to do backups on a mongo cluster in aws is as simple as have a list of hidden nodes in us-west region. Obtain their instance id, lock the database - call aws api to describe the ebs volumes you have attached to those instance id and ask to snapshot it, after all this is done unlock the database. Also you need to take care of the automatic balancer - first stop one of the config server (by stop i mean you put a lock - if you can afford to shut it down is even simpler - than just copy the files) - mongodump the config database and start the balancer.
As an operation sequence you should do the following