I have been advised on stackoverflow to ask this question on dba.stackexchange instead -
We are using MongoDB version 3.0 with WiredTiger storage.
As newbies to MongoDB, we may have designed our schema naively based on limited knowledge from various books and articles and want to improve the design for better performance.
One or 2 collections have an average object size of 52.3 KB and these collections may likely have millions of records eventually which we may shard. What I want to know is - what would be the impact on RAM if we query on the collection. Please note that the sizes of the documents will not grow much in time.
Eg - 1 document (with avg object size of 52KB) has 91 fields/attributes which include arrays and sub-documents. Say I am interested in around 5 fields in a particular query and I specify these fields in the projection argument- I have verified that appropriate indexes are being used on my query. Will mongoDB load only those 5-6 fields into RAM - the ones I am interested in - or the entire document with 91 fields and size 52KB. My question is for both -
- Normal queries
- Aggregation based queries
This will help in estimating my working set size. And another thing, there are lots of other kinds of queries requiring different set of attributes on the same collection and documents - so covered query indexes may not be feasible for all of them.
Should I explore the possibility of splitting to different collections depending on usage patterns even though they may all be really 1:1 relationships? On the flip-side this will not guarantee atomic writes if lots of attributes are getting updated together.
The reason I ask is that I recently observed that geoNear aggregation queries are definitely faster if I trim down the collection to only a few essential attributes. I have a hunch that probably MongoDB may bring entire documents into RAM since it memory maps data files.