0

I need to change shard index from range to hashed in a sharded collection. So I need to mongodump and restore the collection. I have 3 shards with 3 replica sets.MongoDB version is: 3.2.11 I started mongodump from one of mongoses.

mongodump --collection Collname --db dbname

I have "count" : 12925651, "size" : 21233976913 in main collection.

But mongodump is done at %75 with

[##################......] dbname.Collname 9700124/12925651 (75.0%) done dumping dbname.Collname (9700124 documents)

I restore the dump to another db and collection size is smaller than the main collection.

It seems that mongodump doesn't take data from one of the shards. Shard distribution is :

Totals data : 19.77GiB docs : 12925651 chunks : 475 Shard shard01 contains 24.98% data, 24.98% docs in cluster, avg obj size on shard : 1KiB Shard shard02 contains 50.03% data, 50.03% docs in cluster, avg obj size on shard : 1KiB Shard shard03 contains 24.98% data, 24.98% docs in cluster, avg obj size on shard : 1KiB

What is the problem? And what can i do for replacing the shard index?

Improve this question
6
  • How about if you dump that collection directly from every replica set individually. Do get "out" all documents? After creating your sharded collection with new sharding key, you can import those three dumps one by one.
    – JJussi
    Commented Mar 21, 2018 at 6:17
  • When I dump individually and then restore all individual data to one shard, there are duplicate key errors and size decreased. does collection count increase when you shard the collection? I understand that collection parts reside in different shards. So same key_id can be found in individual shards. But can this situation cause increase in collection count?
    – Selcen Şahin
    Commented Mar 21, 2018 at 7:16
  • when i individually dump from every replica set, get out all documents. It is ok. But if i try to restore it to one shard after duplicate key errors document count decreased again.
    – Selcen Şahin
    Commented Mar 21, 2018 at 7:34
  • If the cluster is healthy, there are no documents with same sharding key at different shards. If there is "duplicates", those are "orphans" what should be cleaned from cluster. So, when you import data back to the cluster and you get multiple duplicates, hopefully that first document was that "right" document (newest of those many copies). So, it looks like that your first export (with mongos) was right, because that export exported ONLY one copy of document, that "actual" document. Make search (google) mongodb orphan document to get more information of orphans and how to remove those.
    – JJussi
    Commented Mar 21, 2018 at 14:59
  • Thank you, it solved the issue. My problem was orphaned documents.
    – Selcen Şahin
    Commented Mar 23, 2018 at 6:49

1 Answer 1

Reset to default
0

If the cluster is healthy, there are no documents with same sharding key at different shards. If there is "duplicates", those are "orphans" what should be cleaned from cluster.

Here more information about orphans and how to remove those.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.