Query sharded collection returns all the results from the primary shard AND from the other shards

Question

I tried to set up a shard to test this functionality and maybe migrate some of our data into it.

I followed the documentation guide https://docs.mongodb.com/v3.4/tutorial/convert-replica-set-to-replicated-shard-cluster/

So I have a first replica rs0 (containing 2 servers), a replica rs1 (containing 2 servers) and a config replica (containing 2 servers for my tests, I understand I should use at least 3 in production)

Now, when I'm connecting on mongos instance, I see that rs0 is my primary shard. And the collection I sharded contains 1,000,000 documents. rs1 contains 1 chunk which contains approximately 500,000 documents. If I execute db.partitioned_collection.count() in mongos it returns 1,500,000. Whereas I would expect it to return 1,000,000.

I get that the primary shard contains all the unsharded documents, but mongos should only query data inside rs0 chunks no ?

What am I doing wrong ?

My sh.status() result is :

--- Sharding Status ---
sharding version: {
    "_id" : 1,
    "minCompatibleVersion" : 5,
    "currentVersion" : 6,
    "clusterId" : ObjectId("5b992a952e6c702f9dd5b675")
}
shards:
    {  "_id" : "rs0",  "host" : "rs0/mongo-test:27017,mongo-test:27018",  "state" : 1 }
    {  "_id" : "rs1",  "host" : "rs1/mongo-test:27019,mongo-test:27020",  "state" : 1 }
active mongoses:
    "3.6.7" : 1
autosplit:
    Currently enabled: yes
balancer:
    Currently enabled:  yes
    Currently running:  no
    Failed balancer rounds in last 5 attempts:  0
    Migration Results for the last 24 hours:
            2 : Success
databases:
    {  "_id" : "config",  "primary" : "config",  "partitioned" : true }
            config.system.sessions
                    shard key: { "_id" : 1 }
                    unique: false
                    balancing: true
                    chunks:
                            rs0     1
                    { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : rs0 Timestamp(1, 0)
    {  "_id" : "test",  "primary" : "rs0",  "partitioned" : true }
            test.partitioned_collection
                    shard key: { "number" : 1 }
                    unique: false
                    balancing: true
                    chunks:
                            rs0     2
                            rs1     1
                    { "number" : { "$minKey" : 1 } } -->> { "number" : 4794 } on : rs1 Timestamp(2, 0)
                    { "number" : 4794 } -->> { "number" : 9586 } on : rs0 Timestamp(2, 1)
                    { "number" : 9586 } -->> { "number" : { "$maxKey" : 1 } } on : rs0 Timestamp(1, 2)

UPDATE #1 :

It seems I completely misread the following documentation https://docs.mongodb.com/v3.6/core/sharded-cluster-shards/#primary-shard . Indeed the primary shard does not hold the unsharded documents but the unsharded collections ...

So the question now is why my primary shard (rs0) which should only contains 2 chunks, is still containing the whole collection data ?

Answer 1

I finally tested again to follow exactly the example at https://docs.mongodb.com/v3.6/tutorial/convert-replica-set-to-replicated-shard-cluster/ . And this time it did work... It took some time (don't know how much exactly) for rs0 to delete the moved documents, but after a few minutes (or tens of minutes) they were not there anymore.

I have tried to remember what I did differently on the first time. I'm wondering if I did insert the test data after rs0 became a part of the shard directly on the rs0 replica set. And from what I understand we must not insert/update data on shard replica sets directly.