0

In my sharded cluster of version 4.2, one collections is sharded on _id where _id is just a default ObjectId.

I'm seeing multiple documents with same _id on different shards' primaries.

When I specify shard key within the query filter I get the correct documents, but without it, mongos queries to all shards, and as a result, it returns an orphaned document.

I'm wondering if this is a common problem, or is there something wrong with using _id as a shard key?

0

Here is some information related to your questions.

In my sharded cluster of version 4.2, one collections is sharded on _id where _id is just a default ObjectId. I'm seeing multiple documents with same _id on different shards' primaries.

It is possible, but not common. See the following from this post: Is it possible to have duplicate _id in a shard zone?

If your _id values are using default ObjectIDs, the chance of collision should be extremely low. The _id field uniquely identifies a document within a given collection on a shard. If documents have the same _id values on different shards, attempted migration of those documents to the same shard will result in a duplicate key exception. Non-unique _id values are also likely to lead to logic errors for developers or tools assuming _id is a unique identifier.



When I specify shard key within the query filter I get the correct documents, but without it, mongos queries to all shards, and as a result, it returns an orphaned document.

Yes, without the shard key as part of the query filter, the query will result in a Broadcast (or Scatter-Gather) Operation. And, this kind of query is very inefficient. The shard key plays an important role in the sharding / distributing the data among the shards. The selected shard key should address the important queries of your application. And, there can be queries where the shard key cannot be used.

See Targeted Operations vs. Broadcast Operations.



I'm wondering if this is a common problem, or is there something wrong with using _id as a shard key?

In general, the _id field can be used efficiently as a shard key. By default, the _id is the MongoDB generated ObjectId value. Note the _id can be application generated too; and can be any type of field, except an array type.

Using the default ObjectId as the _id, is not considered efficient. The main considerations for selecting a shard key are the High Cardinality, Frequency and the Type of Change. The Type of Change refers to the change in value monotonically; and this is not recommended.

A shard key on a value that increases or decreases monotonically is more likely to distribute inserts to a single shard within the cluster.

Examples of monotonically increasing value types are the date, timestamp and the ObjectId; the ObjectId has the timestamp embedded in its value (See ObjectId Description).


About Choosing a Shard Key:

A detailed explanation about the three factors to consider in selecting a shard key can be read at: Choosing a Shard Key.

This note in the documentation is useful if you want to use the default ObjectId as the shard key:

If your data model requires sharding on a key that changes monotonically, consider using Hashed Sharding.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.