9

For production backup, MongoDB recommends mongodump instead of mongoexport for accuracy of data. However, I would need to "scrub" data off a MongoDB database before backing it up. I am not aware of any server-side data scrubbing options other than mongoexport. Two questions:

  1. Does mongoexport access MongoDB cache in RAM? Would it alter the working set in RAM like mongodump does?
  2. mongodump command has this --query option. "Provides a JSON document as a query that optionally limits the documents included in the output of mongodump." Does it take a query to exclude certain fields in a document?
  • 1
    It sounds like you're looking for an export rather than a backup. If you only export a subset of fields, presumably you won't be able to restore that and have a fully workable database. There are a few other options to consider: encrypt any fields with sensitive data (via application code) so that you can do a normal mongodump or filesystem snapshot backup, or write a custom export if mongoexport doesn't do what you need. Any tools or programs that read data from mongod could alter your working set if your data to export is bigger than avail RAM. Filesystem snapshots are less disruptive – Stennie Jul 21 '15 at 13:07
  • Thanks for clarification. Encryption on fields is something I haven't thought of, good suggestion. – Howard Lee Jul 21 '15 at 18:07
6

As an answer to your first question, both tools (by default) will just walk the _id index to fetch the data and then write it out to disk. So, yes, both tools will similarly impact your working set which is why I would generally recommend running them against a secondary (preferably a hidden secondary if possible). I'll echo Stennie in the comments here and recommend other backup methods if you are dealing with large amounts of data.

For the second question, I assume you are looking for a mongodump equivalent of the --fields option from mongoexport to only dump out specific fields. The query option can be used to filter results, but it cannot be used with a projection (to select the fields returned) - this is a feature request that is being tracked in TOOLS-28 but is not yet scheduled. As Stennie also mentioned, the other option here is to write a custom exporter that fits your needs (and again, I would still recommend running it against a secondary to protect your working set).

6
  1. (sorry, dunno)

  2. --query will select or exclude entire documents, not fields.

It would be a good idea though :

--query '{datetime:{$gt:ISODate("2014-01-01T00:00:00.000Z")}},{_id:0,name:1,address:1,interests:1}'* )
  1. Mongodump uses bson file structure and preserves the data types. Mongoexport will lose data type of the values. Such as NumberLong("1431677405876") would be converted to just 1431677405876. That's why mongodump is advised if you need to import back to MongoDB.
  • Thanks for your clarification on --query, and the example on mongoexport loss data type. – Howard Lee Jul 21 '15 at 18:05
0

Mongoexport actually does preserve types but it makes the data inconsistent. If you're exporting a field with Ints and Longs, it will export it as Numbers and JSON objects respectively, which can make it harder down the road.

"Strict mode. Strict mode representations of BSON types conform to the JSON RFC. Any JSON parser can parse these strict mode representations as key/value pairs; however, only the MongoDB internal JSON parser recognizes the type information conveyed by the format."

https://docs.mongodb.com/manual/reference/mongodb-extended-json/

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.