1

I have created 2 shard servers named rs1 and rs2. Now in the database collection there are multiple records those should split on these 2 servers in chunks. The sharding key is TxnMonth. It is not unique.

Now I have given the chunk size as 1 MB. When I inserted records corresponding to "TxnMonth" : 1, "TxnMonth" : 2 and "TxnMonth" : 3, the data was splitted onto 2 shards where on rs1 there were 2 chunks and on rs2 there was 1 chunk.

On checking, data for TxnMonth : 1 and 2 was on rs1 and data for TxnMonth : 3 was on rs2.

On running, sh.status(), it gave this output-

{  "_id" : "shardPOC",  "primary" : "rs1",  "partitioned" : true }
                shardPOC.machines
                        shard key: { "TxnMonth" : 1 }
                        unique: false
                        balancing: true
                        chunks:
                                rs1     2
                                rs2     1
                        { "TxnMonth" : { "$minKey" : 1 } } -->> { "TxnMonth" : 1 } on : rs1 Timestamp(2, 1)
                        { "TxnMonth" : 1 } -->> { "TxnMonth" : 3 } on : rs1 Timestamp(1, 2)
                        { "TxnMonth" : 3 } -->> { "TxnMonth" : { "$maxKey" : 1 } } on : rs2 Timestamp(2, 0)
        {  "_id" : "NewTestDB",  "primary" : "rs1",  "partitioned" : false }

After this, I started inserting records for TxnMonth : 4, 5, 6, 7 and 8, to check when the chunk on rs2 will split but even if the chunk size exceeded 1 MB size, it didn't split. Below is the output of db.collection.getShardDistribution() -

Shard rs1 at rs1/JCB-DB1:27012
   data : 57KiB docs : 80 chunks : 2
   estimated data per chunk : 28KiB
   estimated docs per chunk : 40
Shard rs2 at rs2/JCB-DB1:27013
   data : 1.1MiB docs : 1560 chunks : 1
   estimated data per chunk : 1.1MiB
   estimated docs per chunk : 1560
Totals
   data : 1.15MiB docs : 1640 chunks : 3
   Shard rs1 contains 4.87% data, 4.87% docs in cluster, avg obj size on shard : 740B
   Shard rs2 contains 95.12% data, 95.12% docs in cluster, avg obj size on shard : 740B

What I expected was that the chunk will split after it exceeds 1 MB but it did not. Can someone explain the reason? Thanks in advance.

  • I would try to go beyond 20%~ of your chunk size, meaning 1.2MB and see if that triggers a split. – SqlWorldWide Jul 5 '17 at 11:36
  • I tried adding more data to 1.31 MB and it's still the same Shard rs1 at rs1/JCB-DB1:27012 data : 57KiB docs : 80 chunks : 2 estimated data per chunk : 28KiB estimated docs per chunk : 40 Shard rs2 at rs2/JCB-DB1:27013 data : 1.31MiB docs : 1870 chunks : 1 estimated data per chunk : 1.31MiB estimated docs per chunk : 1870 Totals data : 1.37MiB docs : 1950 chunks : 3 Shard rs1 contains 4.1% data, 4.1% docs in cluster, avg obj size on shard : 740B Shard rs2 contains 95.89% data, 95.89% docs in cluster, avg obj size on shard : 740B – Venus Garg Jul 5 '17 at 11:47
  • I would double check the ranges, max chunk size, any jumbo chunk (which I doubt in your case). If nothing open a support case (if you have a support contract). – SqlWorldWide Jul 5 '17 at 11:54
  • db.getSiblingDB("config").actionlog.find().pretty() and check that there is no errors. – JJussi Jul 5 '17 at 12:35
  • Can you post an sh.status() showing the outcome after you inserted the additional data? Currently the sh.status() only shows the before scenario. Also, what specific version of MongoDB are you using? Lastly, how many distinct values of TxnMonth do you have? – Stennie Jul 6 '17 at 5:04
0

Your chosen shard key for a collection defines the granularity of possible chunk splits. Chunks represent a range of values for the shard key. With a single field in your shard key (TxnMonth), that means the smallest possible range is a single value such as TxnMonth: 3.

The chunk size setting determines when MongoDB will try to split chunk ranges in order to help distribute data in a sharded cluster, but does not put an upper bound on the size of the data in a chunk range.

When the maximum chunk size is reached for a given value, that chunk will be marked as an indivisible jumbo chunk and the balancer will no longer attempt to split or migrate that chunk. New data will continue to be added to that chunk leading to an imbalance in data distribution as you observed in your testing.

there are 8 distinct values of TxnMonth from 1 to 8

This limits your sharded collection to a maximum of 8 chunks (or the number of unique TxnMonth values in this collection).

For effective sharding you need to choose a more appropriate shard key with higher cardinality.

|improve this answer|||||
  • there is no jumbo chunk. There were 8 TxnMonth values so I expected it to split into more chunks (max being 8). The number of records corresponding to all 8 TxnMonth values are: 30, 50, 70, 90, 50, 80, 80 and 1500 respectively. There are much records in TxnMonth : 8 so as to increase the chunk size and cross 1 MB so it can trigger the split. Didn't happen though. With such lesser records, I doubt there is any Jumbo Chunk. Records of TxnMonth 1 and 2 are on rs1 (2 chunks) and rest all are on rs2 (all in 1 chunk). Kindly help! – Venus Garg Jul 7 '17 at 7:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.