It's definitely possible to
mongodump from a sharded cluster and
mongorestore to another deployment (standalone, replica set, or sharded cluster).
With this approach there are some general considerations to be aware of:
- You should always
mongodump data via
mongos for a sharded cluster.
mongodump needs to read all requested data through the memory of the
mongod process(es), so this can have a significant resource and performance impact for the deployment being backed up (particularly if the uncompressed data set is much larger than available RAM).
mongorestore will rebuild all indexes for collections being restored, which can have a significant performance impact on the target cluster. If you are using a version of MongoDB server older than 4.2, the default foreground index builds will block other reads & writes to the destination database. MongoDB 4.2 has less impactful index build process (see: Index Builds on Populated Collections).
mongodump does not take a point-in-time backup, so if the source cluster is being actively updated your backup may not be consistent.
- If your deployment is using MongoDB 4.2 or newer and distributed transactions,
mongorestore should not be used (the lack of point-in-time consistency is particularly problematic in this case).
Your original approach of writing a script to migrate the data would also be viable if you iterate rather than trying to store the whole result set in RAM. You could make this approach resumable by scanning a collection in a predictable order (such as sorting by
_id, assuming this is monotonically increasing if new documents continue to be inserted).
Unless you have very custom requirements for data export or throttling, I would be inclined to use
mongorestore with appropriate options for compression and concurrency (for example parallel collections to dump and number of insertion workers per collection).