1

I'm backing up a MongoDB server using (for better or worse) mongodump. After running db.stats() I get a storageSize of 8.9GB and a dataSize of 44.5GB. This makes sense to me as I expect a lot of the data (at least 1/2) to be highly compressible.

My question is this, why would my backup's be 45GB in size, if the storage size is only 9GB - I've also checked my actual disk usage (the disk is used ONLY for mongo and has slightly over 9GB of disk usage.

I'm running mongodump with the following options:

/usr/bin/mongodump -h $HOST -d $DBNAME --username=$DBUSER --password=$DBPASS --archive="$OUTPUT"

I'm aware of the gzip option, but if the data is stored compressed, surely it should be backed up compressed too - and gzip should simply be a second level of compression?

  • what is the MongoDB version in your environment? – Md Haidar Ali Khan Mar 16 '18 at 15:43
2

Actually, if you don't use --gzip parameter, data is at bson format what is not compressed and you can look that data with bsondump program. What mongodump do, is read data from database and that result is of course not compressed.

Here is different between those two from my system

-rw-r--r-- 1 root root 61M Mar 16 06:48 data.bson -rw-r--r-- 1 root root 5.3M Mar 16 06:48 data.bson.gz

Over 10 fold compression ration.

1

As per MongoDB BOL documented here. After an unclean shutdown of a mongod using the Wired Tiger storage engine, count and size statistics reported by db.stats may be inaccurate.

The amount of drift depends on the number of insert, update, or delete operations performed between the last checkpoint and the unclean shutdown. Checkpoints usually occur every 60 seconds. However, mongod instances running with non-default --syncdelay settings may have more or less frequent checkpoints.

Run validate on each collection on the mongod to to restore the correct statistics after an unclean shutdown.

For example the following example converts the returned values to kilobytes:

db.stats(1024)

The method db.stats(scale) returns statistics that reflect the use state of a single database. Where parameter scale number is Optional. The scale at which to deliver results. Unless specified, this command returns all data in bytes.

For example here I have checked for test database.

> db.stats(1024)
{
        "db" : "test",
        "collections" : 10,
        "views" : 0,
        "objects" : 46,
        "avgObjSize" : 165.89130434
        "dataSize" : 7.4521484375,
        "storageSize" : 180,
        "numExtents" : 0,
        "indexes" : 11,
        "indexSize" : 180,
        "fsUsedSize" : 236695904,
        "fsTotalSize" : 296127484,
        "ok" : 1}

and also db.stats() without the scale parameter

> db.stats()
{
        "db" : "test",
        "collections" : 10,
        "views" : 0,
        "objects" : 46,
        "avgObjSize" : 165.8913043478261,
        "dataSize" : 7631,
        "storageSize" : 184320,
        "numExtents" : 0,
        "indexes" : 11,
        "indexSize" : 184320,
        "fsUsedSize" : 242376503296,
        "fsTotalSize" : 303234543616,
        "ok" : 1
}

here you can see the some subtle difference size of both command.

Note: The scale factor rounds values to whole numbers.

The db.stats() method is a wrapper around the dbStats database command.

The command takes the following syntax:

db.runCommand({ dbStats: 1, scale: 1 })

where scale is optional and defaults to 1.

dbStats.dataSize

The total size of the uncompressed data held in this database. The dataSize decreases when you remove documents.

For databases using the MMAPv1 storage engine, dataSize includes preallocated space and the padding factor. The dataSize does not decrease when documents shrink.

For databases using the WiredTiger storage engine, dataSize may be larger than storageSize if compression is enabled. The dataSize decreases when documents shrink.

dbStats.storageSize

The total amount of space allocated to collections in this database for document storage. The storageSize does not decrease as you remove or shrink documents. This value may be smaller than dataSize for databases using the WiredTiger storage engine with compression enabled.

For your further ref here, here and here

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.