Mongodb incremental backups

Question

I was given the task to set up incremental backups for MongoDB replicaset, as start point, of course, I googled it and could not find anything on MongoDB docs, I did find however this question on Stack Overflow, which encouraged to develop my own solution as didn't find Tayra very active.

I read about oplog and realized it was very easy to develop something to replay the log, but it turns out that I didn't have to as mongorestore does that for me.

Now I have a working solution with bash scripts and it was quite easy, that's the reason I am asking here if there is any flaw in my logic, or maybe something that will bite me in the future.

Below how I implemented that:

Full backup procedure

1. lock writes on a secondary member db.fsyncLock()

2. Take snapshot

3. Record last position from oplog

   db.oplog.rs.find().sort({$natural:-1}).limit(1).next().ts
   

4. Unlock writes db.fsyncUnlock()

Incremental backup procedure

1. lock writes on a secondary member

2. Dump oplog from the recorded oplog position on full (or latest incremental ) backup:

   mongodump --host <secondary> -d local -c oplog.rs -o /mnt/mongo-test_backup/1 
       --query '{ "ts" : { $gt :  Timestamp(1437725201, 50) } }'
   

3. Record latest oplog position (same way as for full backups)

4. Unlock writes

Full backup restore procedure

1. stop all instances of mongod
2. copy snapshot to data dir of the box which will be the primary, but make sure to exclude all local* and mongod.lock this restore technique is called reconfigure by breaking mirror
3. Start primary
4. reconfigure replicaset
5. start secondaries without any data, let them perform the initial sync. Or copy the data from the new primary with fresh local database

Restore incremental backup

When we created incremental backup it stored it like this:

/mnt/mongo-test_backup/1/local/oplog.rs.bson
/mnt/mongo-test_backup/1/local/oplog.rs.metadata.json

We're instered on oplog.rs.bson but we will have to rename it, so here are the steps:

1. change directory to the backup: cd /mnt/mongo-test_backup/1/local

2. delete the json file rm *.json

3. rename the bson file mv oplog.rs.bson oplog.bson

4. restore it:

    mongorestore -h <primary> --port <port> --oplogReplay /mnt/mongo-test_backup/1/local
   

I have it all scripted, I may commit it on GitHub later.

The question is if there is any flaw in the logic? I am a bit suspicious as the procedure is quite straight forward and still I couldn't find it documented anywhere.

Answer 1

To answer your question. No! There is no fail on your logic and it should work without problems. However, if LVM snapshots can be used, it's better way to do backups.