2

I was trying to create two compound indexes in background in mongodb. As per mongo doc and other sources, background index creation should not affect read/write.
But severe performance issue was observed. Page fault jumped to 180/s from 4/s, queries that takes 300ms to complete, started taking more than 2 seconds.
Db size : 40GB
Documents : 40 million
Previous index size : 24GB
Compound index1 : {field1 : 1, field2 : -1}
Compound index2 : {field3 : 1, field4 : -1}

DB stats : { "db" : "response", "collections" : 4, "objects" : 42253780, "avgObjSize" : 1008.5294166817738, "dataSize" : 42614180096, "storageSize" : 43648786160, "numExtents" : 44, "indexes" : 14, "indexSize" : 22113921536, "fileSize" : 79352037376, "nsSizeMB" : 16, "extentFreeList" : { "num" : 53, "totalSize" : 5861306368 }, "dataFileVersion" : { "major" : 4, "minor" : 22 }, "ok" : 1 }

Free -m :
total used free shared buffers cached Mem: 64314 63944 370 1 867 56077 -/+ buffers/cache: 7000 57314 Swap: 3999 434 3565

  • We have faced this problem in production mongo replicate set of 3 servers. Anyone can give a quick idea behind this behaviour of background indexes? – sohel Sep 29 '16 at 5:12
3

Index builds can impact the performance of your database. While background index builds will not block read or writes, it is not correct to say that "index creation should not affect [the performance of] read/write".

Building an index will require paging the entire collection through memory; this is cause of the page faults being seen. You will likely want to only build a single index at a time to reduce the impact of this process.

  • +1. This is a correct answer. Index creation is literally making indexes on a disk. It will be hardened to disk so it can't just be stored in memory. This will take up IOPS to do so. Hardening anything to disk will always do this. – Ali Razeghi Sep 30 '16 at 23:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.