0

Lets say for example i need to have 5 collections, each collection is about 10GB. What is the difference in performance, with emphasis on memory usage, between assigning each said collection to a database, versus having all of these collections in the same database?

Also, in this scenario, whats the difference between MMAPv1 storage engine and WiredTiger?

EDIT: Spoken to a MongoDB team and they assured me there should be no difference between having multiple collections in a single database versus one collection for each multiple databases.

  • MongoDB version 2.4 – Tomer Something Jan 22 at 9:54
  • 1
    Are you definitely using MongoDB 2.4? If you are concerned about performance I would recommend upgrading to a modern version of MongoDB. MongoDB 2.4 was first released in March, 2013 and reached end of life in March, 2016. This predates the WiredTiger storage engine and MMAP would be your only storage option until you upgrade to at least MongoDB 3.0. With WiredTiger each collection and index is stored in a separate file, so there would be no appreciable difference in memory usage or performance for the same number of collections in one database or multiple. – Stennie Jan 24 at 4:51
  • @Stennie,The OP already said that he is using the MongoDB version(2.4)? Even my answer is based on new version of MongoDB. – Md Haidar Ali Khan Jan 26 at 15:26
  • @MdHaidarAliKhan I was questioning 2.4 since it is a very old version of MongoDB. MongoDB 3.4 would be more plausible, particularly given mention of WiredTiger. – Stennie Jan 27 at 4:47
  • Both comments were helpful. I am aware that 2.4 is pretty old and upgrading to a more recent version did came up to mind, and is also probably what were going to do. All your comments and answers were helpful. thank you. – Tomer Something Jan 28 at 7:52
1

MongoDB - Database vs Collection memory-wise

As you said you have MongoDB version 2.4, the MongoDB documentation here and here specifications

WiredTiger Storage Engine

  1. Starting in MongoDB 3.2, the WiredTiger storage engine is the default storage engine.
  2. Minimum log record size for WiredTiger is 128 bytes. If a log record is 128 bytes or smaller, WiredTiger does not compress that record.

Memory Use

With WiredTiger, MongoDB utilizes both the WiredTiger internal cache and the filesystem cache.

Starting in 3.4, the WiredTiger internal cache, by default, will use the larger of either:

50% of (RAM - 1 GB), or
256 MB.

By default, WiredTiger uses Snappy block compression for all collections and prefix compression for all indexes. Compression defaults are configurable at a global level and can also be set on a per-collection and per-index basis during collection and index creation.

Note : For existing deployments, if you do not specify the --storageEngine or the storage.engine setting, the version 3.2+ mongod instance can automatically determine the storage engine used to create the data files in the --dbpath or storage.dbPath.

MMAPv1 Storage Engine

  1. MMAPv1 is the default storage engine for MongoDB versions 3.0 and earlier.
  2. MongoDB 4.0 deprecates the MMAPv1 Storage Engine and will remove MMAPv1 in a future release. To change your MMAPv1 storage engine deployment to WiredTiger Storage Engine, see:

    a.) Change Standalone to WiredTiger

    b.) Change Replica Set to WiredTiger

    c.) Change Sharded Cluster to WiredTiger

Memory Use

With MMAPv1, MongoDB automatically uses all free memory on the machine as its cache. System resource monitors show that MongoDB uses a lot of memory, but its usage is dynamic. If another process suddenly needs half the server’s RAM, MongoDB will yield cached memory to the other process.

Technically, the operating system’s virtual memory subsystem manages MongoDB’s memory. This means that MongoDB will use as much free memory as it can, swapping to disk as needed. Deployments with enough memory to fit the application’s working data set in RAM will achieve the best performance.

For example i need to have 5 collections, each collection is about 10GB. What is the difference in performance, with emphasis on memory usage, between assigning each said collection to a database, versus having all of these collections in the same database?

As per google group forum by Mr.Kevin Adistambha defined here it is not advisable to push any technology to such extreme numbers without careful thought due to security, availability, and performance issue.

MongoDB is perfectly capable of doing 5 collections , as each collection with 10 GB in a single database.

The page MongoDB Limits and Thresholds describes the limits of MongoDB capabilities in detail.

For further your ref here and here

  • 1
    The current answer doesn't address the original question of the memory or performance implications of storing collections in a single database versus having the same number of collections in multiple databases. – Stennie Jan 25 at 4:48
  • @Stennie, I have been given the MMAPv1 & WiredTiger differences according to their MongoDB version pros & Cons. Which is also the OP has asked in his question. Even somewhere I have try to include the memory performance issue with respect to "Storage Engine". There is no any snippet code specific memory performance issues. That's why I have not included in my answer. – Md Haidar Ali Khan Jan 26 at 15:23
  • @Stennie, Thanks for notify to me with respect to the "memory performance". – Md Haidar Ali Khan Jan 26 at 15:30

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.