Take the 2-minute tour ×
Database Administrators Stack Exchange is a question and answer site for database professionals who wish to improve their database skills and learn from others in the community. It's 100% free, no registration required.

I recently compacted my collection using the command:

 db.<collectionName>.runCommand( "compact" )

And now my collection size seems to be bigger than size on disk!

SECONDARY> db.<collectionName>.stats()
{
"ns" : "<databaseName>.<collectionName>",
"count" : 2937359,
"size" : 5681676492,                   # 5.6 GB
"avgObjSize" : 1934.2805874256433,
"storageSize" : 4292853728,            # 4.2 GB
"numExtents" : 2,
"nindexes" : 2,
"lastExtentSize" : 2146426864,
"paddingFactor" : 1.669999999836597,
"flags" : 1,
"totalIndexSize" : 220735648,
"indexSizes" : {
    "_id_" : 162326304,
    "e_1_" : 58409344
},
"ok" : 1

}

I don't understand how this is possible. Aren't all mongodb collections backed-by-disk at all times?

Can anyone explain these results?

share|improve this question
    
I've seen stats like that before, but don't have an explanation. Try running a validate? –  Wes Freeman Apr 28 '12 at 2:15
add comment

1 Answer

up vote 3 down vote accepted

storageSize is the sum of all extents for that data, excluding indexes.

So that collection takes up 2 extents, they are ~2GB each, hence ~4GB. size includes indexes and I believe a couple of other things which inflate the number. Neither really represents the proper on-disk size. For disk size, db.stats() has a filesize field which is closer to what you want I think you're looking for.

The manual is somewhat better at outlining what the various fields mean, see here for collections:

http://docs.mongodb.org/manual/reference/collection-statistics/

And here for database stats:

http://docs.mongodb.org/manual/reference/database-statistics/


Some other potentially relevant information:

The compact command does not shrink any datafiles; it only defragments deleted space so that larger objects might reuse it. The compact command will never delete or shrink database files, and in general requires extra space to do its work, usually a minimum of one extra extent.

If you repair the database it will essentially rewrite the data files from scratch, which will remove padding and store them on disk as efficiently as you are going to get. However you will need to have ~2x the size on disk to do so (actually less, but it's a decent guide).

One other thing to bear in mind here - repair and compact remove padding. The padding factor varies between 1 (no moves of documents caused by documents growing), to 2 (lots of moves caused by documents growing). Your padding factor of ~1.67 would indicate you are growing (and hence causing moves) quite a bit.

When you compact or repair a database you remove that padding - subsequent document growth is therefore going to trigger even more moves than before. Because moves are relatiely expensive operations, this can have a serious impact on your performance. More info here:

http://www.mongodb.org/display/DOCS/Padding+Factor

share|improve this answer
    
Thanks for your response @Adam, I am somewhat familiar with padding factors and compacting, what confuses me in this instance is that, no matter how effective compaction is we should never be able to store more data in the database than we're storing on harddisk! i.e., how do you fit 5.6GB of mongo data in 4.2GB of disk? –  Chris W. May 22 '12 at 1:31
    
4.2GB of disk is just the data, 5.6GB is the data plus indexes, and then for actual disk size you are probably going to have to look at the database level statistics instead –  Adam C Jun 7 '12 at 16:46
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.