4

I am not able to take an incremental oplog dump. I am not getting any error messages the connection happens fine and also there is a operation recorded(shows up in db.currentOp()), but still nothing happens, not sure why?

I have executed the same command on a smaller database where everything works fine no problems at all, but the same command on a larger database(talking about 10-11 billion records which is increasing day by day), does not work.

The command is mentioned below:

mongodump --host $MONGODB_HOST:$MONGODB_PORT --authenticationDatabase admin -u $MONGODB_USERNAME -p $MONGODB_PASSWORD -d local -c oplog.rs -o backup/oplogDump/$currentTime --query '{"ts":{$gt: Timestamp( 1452157469, 37)}}'

After executing this command, the entire secondory mongo machine gets stuck, i mean letrely i need to restart the machine to get mongod start running again.

Another change that i have done recently is, i have increased the nssize to 1 GB as per developers requirement, I am not sure since when this issue started or what is causing this issue, Any Help will be really appreiciated?

Improve this question

1 Answer 1

Reset to default
4

You are essentially attempting to dump the entire oplog and hence paging in all of it, not just the most recent data. That is going to cause a lot of IO, especially when your oplog is too large to fit into memory. On a smaller database it's not a problem because the oplog is relatively small.

The reason why it's dumping the full table is because I believe you are missing some zeroes at the end of your time:

> new Date(1452157469)
ISODate("1970-01-17T19:22:37.469Z")

I assume you wanted this:

> new Date(1452157469000)
ISODate("2016-01-07T09:04:29Z")

The speed this dump happens will depend on a number of factors, but mostly it will depend on how much data has been inserted into the oplog recently and how much of it is still in memory when you start the dump.

Improve this answer
3
  • Hi Adam C, thank you, for your reply, can you suggest any other way of taking a mongodump of the oplog, so that it only reads the data from the passed timestamp, to avoid to much IO.
    – Prashanth Kumar B
    Commented Jan 11, 2016 at 12:08
  • 1
    You are doing it correctly, you are just passing the incorrect timestamp - fix that and you will be pulling the information you actually want.
    – Adam C
    Commented Jan 11, 2016 at 12:24
  • Thanks Adam. It's interesting because I had the same issue and the date was taken by executing mongo --eval 'printjson(db.getSiblingDB("local").oplog.rs.find().sort({$natural:-1}).limit(1).next().ts)'. Mongo v3.0.15. So seems like a bug to me from Mongo!
    – TanisDLJ
    Commented Sep 8, 2017 at 8:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.