2

Is MongoDB feasibly suited to handle 90,000,000 documents per day? Version 3.2.

I did some rough estimates on my data size. There will be about 90,000,000 recorded events per day, each a maximum size of 16KB.

Despite the size of this data which our RDBMS is currently handling but not very well, can Mongo feasibly handle that and more if this grows?

Improve this question
4
  • Are you saying the total daily write will be 16KB, or there will be 90M new records per day, each of 16KB i.e. 1.4TB of new data per day? Most likely both Mongo and RDBMS would handle it given sufficiently good storage. Violin reckon they can do 1M IOPS at 1GB+ per sec. That should see the load go through in a couple of minutes. Of course, if you insist on running it on an old laptop you found in the cupboard neither will suffice. – Michael Green Mar 30 '16 at 2:00
  • I understand the hardware issue here, I ask about Mongo because to the need for read and write performance. The collection rate is 10 times per second. And potentially 100 times per second. – htm11h Mar 30 '16 at 11:52
  • Sorry for the confusion 90M, each less than 16KB – htm11h Mar 30 '16 at 14:07
  • 1
    Since you are using 3.2, and presumably wiredTiger as your storage engine (which does not have padding), the output of db. yourcoll.stats() of the collection holding your existing (sample) data would be really helpful. There is quite a difference between 90M * 1kB or 90M * 16kB, let alone the indices and whatnot. In general, yes, if done properly, MongoDB should be able to handle this. I am more concerned of financing: with the max growth of 1.4TB, you'd actually need to add a shard (at least one server) every few days (this applies to all DBMS). – Markus W Mahlberg Mar 30 '16 at 18:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.