First query is really slow with WiredTiger, not with mmapv1
As per MongoDB blog documentation How to Tell if Your MongoDB Server Is Correctly Sized For Your Working Set For MongoDB users, knowledge of working sets is critical. Understanding the interactions between your working set and physical memory can make a major difference in how your system performs.
In order to optimize MongoDB performance you should consider and know the following:
How working sets fit into the scheme of MongoDB in the first place;
Why it's important to fit your working set within your available RAM;
Why you should avoid pulling from disk; and
When and how it's possible to tell that there's a mismatch between your working set and your RAM.
What is a working set?
MongoDB's documentation defines a working set as follows:
A working set represents the total body of data that the application
uses in the course of normal operations… For best performance, the
majority of your active set should fit in RAM.
Why's this important? Fitting your working set to RAM is vital for performance: Going to disk is costly. As this breakdown from a UC Berkeley study on latency numbers shows, reading from memory is significantly faster than reading from disk. As this visualization of 2017 speeds shows, even if you're using SSD, accessing from memory is an order of magnitude more efficient than accessing from disk.
There are two primary storage engines available for MongoDB:
MMAP
- As Mongo's documentation
explains, "MMAPv1 is MongoDB’s original storage engine based on
memory mapped files. It excels at workloads with high volume inserts,
reads, and in-place updates." As of
MongoDB version 3.2, however,
MMAP is no longer MongoDB's default storage engine—WiredTiger now
holds that spot.
- With MMAP, MongoDB will use all available memory, as necessary. There
are no configuration options to limit the data kept in-memory.
WiredTiger
- WiredTiger has an internal data cache, but is configured to also
leave memory for the Operating System’s file cache. It's generally
recommended to leave your cache size at its default setting.
- These defaults
settings will use the larger of either of the following: 50% of
RAM minus 1 GB.
- 256 MB.
- Outside WiredTiger's cache, the Operating System will cache
frequently accessed files for the database in its own file cache.
WiredTiger compresses data files by default, so leveraging the OS
cache actually lets more data stay in the server's physical memory.
How can I tell if my working set fits my RAM?
For MMAP, a high number of page faults is a leading indicator that instead of finding data in RAM, it's going to disk to fetch it. That, of course, means the working set isn't fitting within your available RAM. Inversely, you can look directly at your RAM usage: As explained above, MMAP uses all available memory automatically. Therefore, if you're not using all of the RAM, then you can safely conclude that the working set fits into RAM, even with room to spare.
The total number of page faults. The extra_info.page_faults counter may increase dramatically during moments of poor performance and may correlate with limited memory environments and larger data sets. Limited and sporadic page faults do not necessarily indicate an issue.
Windows draws a distinction between hard page faults involving disk I/O, and soft page faults that only require moving pages in memory. MongoDB counts both hard and soft page faults in this statistic.
"freeMonitoring" : {
"state" : <string>,
"retryIntervalSecs" : <NumberLong>,
"lastRunTime" : <string>,
"registerErrors" : <NumberLong>,
"metricsErrors" : <NumberLong>
},
For WiredTiger, you can also begin by looking directly at memory usage. If your server has free RAM, that inherently means the working set fits into your memory, as WiredTiger automatically uses its set cache plus your operating system's cache. If your server looks to be using most of its memory, take a look at the pages-read-into-cache and unmodified pages evicted metrics reported for WiredTiger by MongoDB’s serverStatus command. High activity for those two metrics indicates that the server is cycling data through the cache, which means your working set does not fit in memory.
The last metric to consider, of course, is volume of disk reads — if you see a lot of read activity on your drive, it's generally a good sign that your working set is overflowing your memory. Because WiredTiger interacts with your OS cache, this may be a more reliable metric to evaluate working set ratio — e.g. you may be able to see high cache evictions but low IO because of how your cache behaves.
Even though as documented here WiredTiger allows for tuning some of the storage engine parameters. The 3 most important to consider are the cache size, checkpoint interval and logging.
for further your ref here , here and here
mongodprocess on an empty data directory. It gets stuck on the very first query (db.somecollection.findOne). Running on a MacBook Pro, 16GB RAM, about 8GB free.