63

I am just confuse about the Sharding and Replication that how they works..According to Definition

Replication: A replica set in MongoDB is a group of mongod processes that maintain the same data set.

Sharding: Sharding is a method for storing data across multiple machines.

As per my understanding if there is data of 75 GB then by replication (3 servers), it will store 75GB data on each servers means 75GB on Server-1, 75GB on server-2 and 75GB on server-3..(correct me if i am wrong)..and by sharding it will be stored as 25GB data on server-1, 25Gb data on server-2 and 25GB data on server-3.(Right?)...but then i encountered this line in the tutorial

Shards store the data. To provide high availability and data consistency, in a production sharded cluster, each shard is a replica set

As replica set is of 75GB but shard is of 25GB then how they can be equivalent...this makes me confuse a lot...I think i am missing something great in this. Please help me in this.

93

A Replica-Set means that you have multiple instances of MongoDB which each mirror all the data of each other. A replica-set consists of one Master (also called "Primary") and one or more Slaves (aka Secondary). Read-operations can be served by any slave, so you can increase read-performance by adding more slaves to the replica-set (provided that your client application is capable to actually use different set-members). But write-operations always take place on the master of the replica-set and are then propagated to the slaves, so writes won't get faster when you add more slaves.

Replica-sets also offer fault-tolerance. When one of the members of the replica-set goes down, the others take over. When the master goes down, the slaves will elect a new master. For that reason it is suggested for productive deployment to always use MongoDB as a replica-set of at least three servers, two of them holding data (the third one is a data-less "arbiter" which is required for determining a new master when one of the slaves goes down).

A Sharded Cluster means that each shard of the cluster (which can also be a replica-set) takes care of a part of the data. Each request, both reads and writes, is served by the cluster where the data resides. This means that both read- and write performance can be increased by adding more shards to a cluster. Which document resides on which shard is determined by the shard key of each collection. It should be chosen in a way that the data can be evenly distributed on all clusters and so that it is clear for the most common queries where the shard-key resides (example: when you frequently query by user_name, your shard-key should include the field user_name so each query can be delegated to only the one shard which has that document).

The drawback is that the fault-tolerance suffers. When one shard of the cluster goes down, any data on it is inaccessible. For that reason each member of the cluster should also be a replica-set. This is not required. When you don't care about high-availability, a shard can also be a single mongod instance without replication. But for production-use you should always use replication.

So what does that mean for your example?

                            Sharded Cluster             
             /                    |                    \
      Shard A                  Shard B                  Shard C
        / \                      / \                      / \
+-------+ +---------+    +-------+ +---------+    +-------+ +---------+
|Primary| |Secondary|    |Primary| |Secondary|    |Primary| |Secondary|
|  25GB |=| 25GB    |    | 25 GB |=| 25 GB   |    | 25GB  |=| 25GB    |   
+-------+ +---------+    +-------+ +---------+    +-------+ +---------+

When you want to split your data of 75GB into 3 shards of 25GB each, you need at least 6 database servers organized in three replica-sets. Each replica-set consists of two servers who have the same 25GB of data.

You also need servers for the arbiters of the three replica-sets as well as the mongos router and the config server for the cluster. The arbiters are very lightweight and are only needed when a replica-set member goes down, so they can usually share the same hardware with something else. But Mongos router and config-server should be redundant and on their own servers.

  • 1
    Thanks a lot for the detail answer...one more question...if the primary is down while a write or read operation is being carried out then..1) what is the delay in selecting the primary from the secondaries and 2) during that delay where will be the data be stored temporarily? – Saad Saadi Nov 22 '13 at 6:58
  • 4
    @SaadSaadi The primary election process is described in the documentation. It takes between 10 and 12 seconds for the secondaries to notice that the primary is down. The primary election itself will usually only take milliseconds. The replica-set is read-only while there is no primary. Any attempts from applications to write data during this time will fail. – Philipp Nov 22 '13 at 8:23
  • 1
    @Philipp: Just two comments: (1) the shard key cannot be modified (i.e. you cannot shard using a different key) and (2) you can read from the secondary nodes of the replica set but consistency depends from the write concern (in order to be consistent the w option should be equal to the replica set sth which is not viable since each shard may have different replica set sizes deliberately or due to node failures). – Mike Argyriou Sep 17 '15 at 9:30
  • @Philipp can you please answer further follow up questions on dba.stackexchange.com/questions/208482/… ? – user3198603 Jun 1 '18 at 13:37
14
  • Sharding partitions the data-set into discrete parts.
  • Replication duplicates the data-set.

These two things can stack since they're different. Using both means you will shard your data-set across multiple groups of replicas. Put another way, you Replicate shards; a data-set with no shards is a single 'shard'.

A Mongo cluster with three shards and 3 replicas would have 9 nodes.

  • 3 sets of 3-node replicas.
  • Each replica-set holds a single shard.
  • For one large file, is it stored into one shard or multiple shard (thus across the nodes)? – Tony Oct 26 '16 at 15:42
  • Note that in MongoDB 3.4 or higher, you'll also need mongoDB servers for configuration, and an additional server to act as the mongos router. This brings the total of the 3x3 cluster in your example to 13 servers total. – dthrasher Nov 13 '17 at 21:29
5

By sharding, you split your collection into several parts.
Replicating your database means you make mirrors of your data-set.

3

In terms of functionality delivered. Sharding provides scalability and parallelism. Replication provides availability

  • nope, replication only also provides scalability&parallelism given that reads are much more frequent than writes – Kristóf Szalay Mar 21 '17 at 0:30

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.