Take the 2-minute tour ×
Database Administrators Stack Exchange is a question and answer site for database professionals who wish to improve their database skills and learn from others in the community. It's 100% free, no registration required.

I am looking for a way that I can backup my 12 MongoDB shards into a single server. Disk space is not an issue but getting the backups to be on a single node would be the best. I am not sure if there is a way to run more than one MongoDB server on a single machine or if there is a way to have all 12 shards merged into a single MongoDB instance. I am running on AWS and will have the primary data in US-East but want the backups in US-West. I dont mind if there is lag on the backup itself, but trying to avoid the app servers for waiting for the backup to be written.

share|improve this question
add comment

2 Answers 2

You need to be a little more clear about your requirements:

  • Do you want to keep all 12 databases intact (12 instances versus all 12 in a single instance)?
  • Are you intending to serve any queries from the backup servers?
  • Do you want multiple snapshots?

You can approach this from many different ways, but I will cover the most common approaches:

Put secondaries in US-West

This approach means you have a full, complete copy of your databases on the west coast. You can make them hidden and priority 0 to make sure they don't serve slaveOK reads or become primary by accident. To make them primary (in a disaster recovery scenario) you will need to re-configure the set manually if they are hidden and priority 0.

This is quite an easy approach, though it could be expensive. You can run multiple mongod processes on a single host but they will be competing for resources and it will still have to be fast enough to keep up with the primaries on the east coast.

Use EBS snapshots

Assuming you are running with journaling enabled, you can just take an EBS snapshot of your data folder (dbpath) and you can back that up to S3 or use it to "seed" hosts on the west coast.

Mongodump

mongodump can be used to dump out all of your data from the Eastern hosts to the Western hosts. However, the output (BSON files) will need to be restored before it can be used.

Generally I would recommend a combination of the first two in order to combine the best resilience options with the most flexibility.

share|improve this answer
    
Keeping the 12 different databases intact or not is not the major issue just having the database backed up and not having to span a lot of nodes to do it. I may run querys from it but not production or something like that. –  WojonsTech Nov 1 '12 at 19:40
add comment

Ideally you would have one or more replica set. Within the replica set I suggest you always have odd numbers for the number of members with one node that is hidden locate in us west. To do so into a three members replica set you can use something like this:

{
"_id" : "rs0",
"version" : 1,
"members" : [
         {
                 "_id" : 0,
                 "host" : "mongodb0.example.net:27017"
         },
         {
                 "_id" : 1,
                 "host" : "mongodb1.example.net:27017"
         },
         {
                 "_id" : 2,
                 "host" : "mongodb2-us-west-region.example.net:27017",
                 "priority": 0,
                 "hidden": true
         }
        ]
 }

If you keep this sort of config to do backups on a mongo cluster in aws is as simple as have a list of hidden nodes in us-west region. Obtain their instance id, lock the database - call aws api to describe the ebs volumes you have attached to those instance id and ask to snapshot it, after all this is done unlock the database. Also you need to take care of the automatic balancer - first stop one of the config server (by stop i mean you put a lock - if you can afford to shut it down is even simpler - than just copy the files) - mongodump the config database and start the balancer.

As an operation sequence you should do the following

  • stop balancer
  • lock the db
  • snapshot the db
  • make a backup of the balancer (mondodump or copy the files)
  • unlock the db
  • start the balancer
share|improve this answer
    
I have seen this method i have been avoiding it because I dont want to have 12 instances more than i already have to keep backups want to be able to merge them into one node. –  WojonsTech Nov 1 '12 at 19:39
    
since you have to lock the db do be able to have data consistency you can do it on a regular secondary with the note that clients will be blocked for that time. –  silviud Nov 3 '12 at 1:25
    
I have two sets of secondarys already which are located in the same area zone of AWS East my boss wants a backup in US West, does not need to be something that people run on but if they have a full copy of all the data in one place. –  WojonsTech Nov 3 '12 at 6:22
    
so you need a node in us west - since you don't want to query against it you need to set it as hidden. –  silviud Nov 6 '12 at 18:33
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.