0

I created a sharded mongodb cluster with 2 shards. Then I created a sharded collection by range and inserted 10000 documents with random documents with random values for the sharded key using Java. For some reason right after the insertion, the chunks are evenly distributed among the shards, however the number of documents are not.

This is what I see right after the insertion of the 10000 documents (in addition to existing 2 documents)


Shard shard0000 at 127.0.0.1:27017
 data : 2.65MiB docs : 4402 chunks : 7
 estimated data per chunk : 387KiB
 estimated docs per chunk : 628

Shard shard0001 at 127.0.0.1:27018
 data : 6.02MiB docs : 10002 chunks : 8
 estimated data per chunk : 770KiB
 estimated docs per chunk : 1250

Totals
 data : 8.67MiB docs : 14404 chunks : 15
 Shard shard0000 contains 30.57% data, 30.56% docs in cluster, avg obj size on shard : 631B
 Shard shard0001 contains 69.42% data, 69.43% docs in cluster, avg obj size on shard : 631B

First I don't understand why I see 14,404 documents instead of 10,000 that I inserted.

To my surprise, the next day, I come and run ```db.myCollection.getShardDistribution()``, and suddenly I see a completely different picture. The data suddenly became well distributed.

So I started experimenting and found out that if I run the following commands:

use config 
db.settings.update(
   { _id: "balancer" },
   { $set: { activeWindow : { start : "11:06", stop : "11:07" } } },
   { upsert: true }
)

And wait for that time window to pass the data is suddenly making sense:

mongos> db.myCollection.getShardDistribution()

Shard shard0000 at 127.0.0.1:27017
 data : 2.65MiB docs : 4402 chunks : 7
 estimated data per chunk : 387KiB
 estimated docs per chunk : 628

Shard shard0001 at 127.0.0.1:27018
 data : 3.36MiB docs : 5600 chunks : 8
 estimated data per chunk : 431KiB
 estimated docs per chunk : 700

Totals
 data : 6.02MiB docs : 10002 chunks : 15
 Shard shard0000 contains 44.03% data, 44.01% docs in cluster, avg obj size on shard : 631B
 Shard shard0001 contains 55.96% data, 55.98% docs in cluster, avg obj size on shard : 630B

In my Java code that inserts 10000 documents, all I do is generate 10,000 Document objects and run an insertMany(documents) in the end.

  List<Document> documents = new ArrayList<>();

  for (int i = 0; i < 10000; i++) {
     Document document = new Document();
     document.append("name", generateName())
                    ...
     documents.add(document);
  }

  collection.insertMany(documents);

So what is going on here?

Note: I set the max chunk size to 1MB just to experiment with sharding Thanks

Update:

A the key on which I'm sharding is a random string generated like this:

 private static String generateName() {
        StringBuilder name = new StringBuilder();

        name.append(RandomStringUtils.randomAlphabetic(1).toUpperCase());
        name.append(RandomStringUtils.randomAlphabetic(5, 10).toLowerCase());

        return name.toString();
    }

So for the large number of documents, the distribution should be fairly uniform.

  • The MongoDB balancer distributes based on chunks, not documents. If you have variable sized documents or delete large ranges of data (creating empty chunks), it is possible that your shards will be balanced by number of chunks but have very different distribution of documents. In your example it sounds like you may have chosen a shard key which does not uniformly distribute data, so immediately after insertion you may be seeing the results of migrations in progress which complete by the time you check the following day. Can you edit your question to include a description of your shard key? – Stennie Sep 30 at 1:52
  • Thanks for your response. I just added the information about the key, The numbers just don't add up, it looks like initially shard0001 contains all the inserted data for some reason, and shard0000 contains a copy of half of the chunks or something like that. – Michael P Sep 30 at 3:25
  • @Stennie. I don't have variable sized documents, they are all more or less the same size, I also did not delete any ranges, I just did one single bulk insertion. The numbers just don't add up, it looks like shard0000 has about 50% of the data, and shard0001 has 100% of the data though shard0000 has 50% of the chunks, and shard0001 has 50% of the chunks – Michael P Sep 30 at 15:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.