MongoDB Disk Usage with Capped Collections

Question

I am using MongoDB using C#.

I am currently using only a single server.

Since I have to create Table events for MongoDB collections I have created a single replica set(primary). And using oplog.rs table.

I have created only one Database : MyDatabase I also have a capped collection in this Database, and some 5 or 6 tables with very less data.

The Issue here MongoDB size become so huge. There is not much data present , the data wont be more than 500MB but my MongoDb took 20GB data of my Harddisk.

When I checked inside data\db folder

there is 6 copies of my database MyDatabase has been created automatically. Each of which consuming 2GB

MyDatabase.0
MyDatabase.1
MyDatabase.2
MyDatabase.3
MyDatabase.4
MyDatabase.5

and

local.0
local.1
local.2
local.3
local.4
local.5

In the Overview it shows understorage the oplog allocated 8.5GB and the capped collection 4.7 GB

The confusion here is No data more than 500MB is present in the Database but still mongoDB took 20GB.

Am I doing something wrong here with the settings. Please help me in this

Thanks

Answer 1

You have basically allocated space by defining two capped collections, the oplog and the second capped collection in MyDatabase.

Unless you specify the oplogSize, the oplog will be allocated at 5% of free space on the volume containing the MongoDB data (so I am guessing you had ~170GB free).

The second capped collection size would have been defined by you, and then the _id index would be created and grown in addition to the initial allocation as you added data. The files also contain the data from the other collections, as you mentioned.

The key thing to remember here, is that capped collections pre-allocate the entire amount specified in advance and never grow or shrink. Besides the indexes that will be added on user defined capped collections (the oplog has no indexes), the capped collection will remain the same size regardless of how much data you put into it, whether that is 500MB or 500TB - think of it as a fixed-size circular buffer.

If you want to see this in action, try creating another capped collection of, say, 500MB in a different database called foo, you will end up with foo.0 through foo.4 (possibly foo.5 depending on the version) and have 960MB (or 1984MB, again depending on version) of files on disk. That represents the minimum needed to contain a capped collection of that size.

Answer 2

Important thing to consider before a shrink: MongoDB grows it's data files by doubling so the dfs may be 64MB,128MB,256 [...] up to 2GB. (then it spawns multiple files)

You can use a server side Javascript to do shrinking and run that JS via Mongo's shell from time to time via a job.

The function that does the shrink is db.repairDatabase(). Use the db. .storageSize() and db. .totalSize() functions to get size of actual and allocated.