I'm in charge of building a nodejs module that dumps varied amount of data from a mongodb into a file and then put it into S3 (sometimes a single document, sometimes thousands, like a whole collection).
The project where this module will be already uses "mongodb" driver for different use cases.
I was told that for larger dumps, using
mongoexport could be better in performance, than running the query within nodejs, but this solution would imply spawning a new process and depending on a different tool (mongoexport), it certainly adds complexity to the solution.
My concern isn't the performance in terms of speed, but in terms of overhead to the DB.
Is it true that mongoexport will dump a whole collection faster (and with less "effort") than simply querying and writing the result with a fs stream?
are there guidelines or best practices to do things like this?