Scaling out mongodb on the same storage

Question

We are planning to have some tests to predict how mongo db operates and functions with 2 terabyte of inforamtion on a single collection.

So the plan is to have 15-20 sharded machines (on the same environment / cabinet).
All are considering to have shared EMC storage

I wanted to ask before we start the test :

IS there any different between:
* Having physical machines writing to independent disks (on the machine).
* Having physical machines writing to the same storage.
* Having virtual machines writing to same storage.
About memory consumption, What are the hardware considerations about querying this amount of data? If I understand correctly, the advantage when you have massive writes is to do sharding with small-level servers, but is it going to change when you need to have some reads?

Thanks

Can you clarify your point 2? The thing is, in a sharded environment, you don't have a SINGLE sever doing the query and readings. The processing is distributed across all the shards. Each shard processes the query according to the data it has and at the end there is a kind of merge (might not be linear and sometimes is a quite complex process). — joao, Commented Feb 20, 2014 at 9:48
@joao Yes, you are correct, I am mixing with replica sets. Thanks — SexyMF, Commented Feb 20, 2014 at 11:24
About your 1st point I believe the best option is "Having physical machines writing to independent disks" because you will take the best advantage of sharding parallel processing (and disk I/O here might be a huge factor to consider). (Btw, I'm not submitting an answer because I don't have mongoDB experience in a sharded env with 2 teras of data... but If you run some tests and reach any conclusion I would like to hear about them and your results) — joao, Commented Feb 23, 2014 at 9:57

Ted Elliott · Accepted Answer · 2014-02-25 05:22:05Z

If you have multiple databases, writing to independent disks (with the directory per db) option will be your best bet for performance. Even if you just have one big huge sharded database, you can still get some performance boost with multiple disks by putting your journal and local database (database where replication info goes) on separate disks, but your main database data files would all have to be on a single disk.
Not sure what you're after here. The recommendation is that your working set fit in memory for best performance and that still applies for sharding, its just split across multiple servers and thus your memory requirements are also split up across those servers. That will of course also depend on how well you pick your sharding key, hopefully the load is distributed evenly across all the nodes. If you plan on returning lots of data from your queries then you might need extra memory on your mongo-s servers as they have to do some sorting and aggregation of what you're bringing back.

1 Answer 1