MongoDB / WiredTiger: reduce storage size after deleting properties from documents

Question

I use MongoDB 3.4 with WiredTiger storage engine on a replica set of 4 nodes.

I shrinked a lot a documents by removing properties that takes most of the space, but the documentation says that this operation doesn't reduce the storage size (only the data size), and I can confirm it's true.

So I tried using the compact command: some space was freed but the storage size is still way bigger than data size. Is it because it only moves documents but does not reduce already allocated space per documents?

Do I need to delete and recreate all documents to really reduce the storage size?

Answer 1

Just to clarify, please be careful about using repairDatabase on a replica set node. repairDatabase is meant to be used to salvage readable data i.e. after a disk corruption, so it can remove unreadable data and let MongoDB start in the face of disk corruption.

If this node has an undetected disk corruption and you run repairDatabase on it, this could lead into that particular node having a different data content vs. the other node as a result of repairDatabase. Since MongoDB assumes all nodes in a replica set contains identical data, this could lead to crashes and hard to diagnose problems. Due to its nature, this issue could stay dormant for a long time, and suddenly manifest itself with a vengeance, seemingly without any apparent reason.

WiredTiger will eventually reuse the empty spaces with new data, and the periodic checkpointing that WiredTiger does could potentially release space to the OS without any intervention on your part.

If you really need to give space back to the OS, then an initial sync is the safest choice if you have a replica set. On a standalone, dump/restore will achieve the same result. Otherwise, compact is the safer choice vs. repairDatabase. Please backup your data before doing any of these, since in my opinion this would qualify as a major maintenance.

Answer 2

If you inspect the database using db.stats(), you will find dataSize and storageSize. storageSize may be bigger after you delete documents from the database. The use of db.repairDatabase() or compact command may not reduce storageSize. In that case, the more reliable way to reclaim disk space is to create a dump archive with mongodump, drop the database, and then restore it with mongorestore.

mongodump --gzip --archive=dump.gzip
mongo
> db.dropDatabase()
mongorestore --gzip --archive=dump.gzip

This solution will require downtime based on the size of the database.

FYI: MongoDB does not release disk space after you delete a document, instead, it will reuse that space for future documents, hence the storageSize being bigger than dataSize.

Answer 3

If you use replica set with more than 2 nodes, you can start initial sync on one secondary at time (and finally step down your primary to secondary for initial sync). By deleting all data files from nodes dbPath.

OR, if you have "enough" free disk space, you can use repaireDatabase command. Command will rebuild all data files again.

Of course, the third option is export (dump) DB with all it's collections to disk, drop the database and then import everything back.

The fourth option would be (requires extra space) copy database to the new name (temporary name), drop the old database, make again copy back to the original name and then drop that temporary database. This because there is no "renameDatabase" command in MongoDB.

Answer 4

I succeed by deleting and recreating documents, and then running the compact command on my collection.