I have a MongoDB database with ~100GB data size. I ran a test with 300 threads and queries are all reads, no writes (except writes to db profiler I guess). I enabled database profiler to keep track of slow queries. I noticed that queries with high 'numYields' result in high 'millis'; likewise, queries with low numYields responded very fast in low millis.
90% of the queries ran very fast, in 1 ~ 2 ms, however, around 2% of queries end up in 60,000 ms or higher.
According to MongoDB doc:
numYields is a counter that reports the number of times the operation has yielded to allow other operations to complete.
Typically, operations yield when they need access to data that MongoDB has not yet fully read into memory. This allows other operations that have data in memory to complete quickly while MongoDB reads in data for the yielding operation.
I understand a slow query was trying to read data from disk, while yielding to other queries that already have data in memory. However, if this result in 60,000ms for that particular query, it becomes unreasonable.
Perhaps there's a way to limit numYields? Or perhaps try to fit everything to memory? Any suggestions?