1

When I execute the below query in mongodb shell, it gives me the number of records updated after the mentioned timestamp.

db.getCollection('oplog.rs').find({"ts":{"$gt":Timestamp(1582823397,2)}})

I wish to execute a similar query with mongodump command. Below is how I'm trying to do it.

query.json
{ "ts" : { "$gt" : Timestamp(1582823397,2) } }

mongodump -h 127.0.0.1:27018 -d local -c oplog.rs --queryFile=query.json -o - > D:/MongoDB/Backup/1582823
397_2_oplog.bson

The above command doesn't work and throws the below mentioned error.

Failed: error parsing query as Extended JSON: invalid JSON input. Position: 19. Character: T

Which seems valid as the query.json doesn't follow the JSON rules.

If I change the query.json file as below.

query.json
{"ts":{"$gt":"Timestamp(1582823397,2)"}}

The query doesn't return any records.

All of the above is a part of a shell script that is supposed to give me incremental dumps.

Reference of the script is taken from here. Referred this question as well.

I'm open to any other suggestions of getting incremental backups of MongoDB.

Adding in response to Wernfried's comment.

{
    "ts" : Timestamp(1582823446, 1),
    "t" : NumberLong(4),
    "h" : NumberLong(0),
    "v" : 2,
    "op" : "i",
    "ns" : "uis.healthCheck",
    "ui" : UUID("60d2d4de-48a3-46ec-a948-6fcb676ba52f"),
    "wall" : ISODate("2020-02-27T17:10:46.010Z"),
    "o" : {
        "_id" : 2,
        "status" : "Success2"
    }
}
  • In order to create a oplog dump you may use the --oplog option. – Wernfried Domscheit Feb 28 at 12:00
  • 1
    MongoDB Extended JSON format is like this: {"$timestamp":{"t":1565545664,"i":1}} – Wernfried Domscheit Feb 28 at 12:06
  • @RahulParyani, What is the MongoDB version(x,y,z)? – Md Haidar Ali Khan Mar 2 at 4:20
  • @MdHaidarAliKhan MongoDB version is 4.2.2... Additionally, I have this setup on Windows Server 2012 – Rahul Paryani Mar 2 at 9:44
  • @WernfriedDomscheit I have added the way document is stored oplog.rs collection. I have no idea how to apply the suggestion given by you. Once again, I need a query that returns records greater than a specific timestamp from oplog.rs... Thanks – Rahul Paryani Mar 2 at 9:53
0

I was able to achieve what I wanted by the suggestions given in the comments by @WernfriedDomscheit and @prasad_

Now my mongodump command is as below

mongodump -h 127.0.0.1:27018 -d local -c oplog.rs --query "{\"ts\":{\"\$gt\":{\"\$timestamp\":{\"t\":${TIMESTAMP_LAST_OPLOG_ENTRY},\"i\":${INC_NUMBER_LAST_OPLOG_ENTRY}}}}}" -o - > ${OUTPUT_DIRECTORY}/${TIMESTAMP_LAST_OPLOG_ENTRY}_${INC_NUMBER_LAST_OPLOG_ENTRY}_oplog.bson 2>> "$LOG_FILE"

"TIMESTAMP_LAST_OPLOG_ENTRY" and "INC_NUMBER_LAST_OPLOG_ENTRY" are being fetched from the previous '.bson' file.

Thanks, Rahul Paryani

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.