I've successfully created and deployed a self-managed MongoDB cluster that consists of 3 shards,1 router(mongos), and 1 config-server replica set, using this tutorial.
I want to create a shareded collection, with the next requirements:
- documents should distribute evenly across all shards(in respect to total size, not docs count).
- documents in this collection can contain any field, and the only field that is guaranteed to exists is the
_id
field. - the collection should be a time-series collection.
- MongoDB version 5.0.9
What I tried :
- It is ok to use the
_id
field exists in any doc in mongodb as your shard key for a sharded collection (not necessary time-series one)?
I've tested it(because could not find answers online), configuring the the sharded collection with an hashed index created from the_id
field:
sh.shardCollection("shardedDb.shardedCollection", { _id : "hashed" }
)
and streamed a lot of dumy data.
for 4.5TB and 460 Millions records, the distribution was even and
each of the 3 shard contained 1.5TB. Worked like charm. yet I need
to ask, is it ok? why I don't see this solution proposed as quick
solution anywhere else?
- When dealing with time-series collections in MongoDB, you can only create indexes from the
timeField
and themetaField
(this is true, for Mongo V5, in V6 there is no limitation, but I can't upgrade my cluster version). in other words, you cannot create index from_id
. so I tried using mytimeField
as the key shard but the data was not distributed evenly across the shards.
so what can I do? I can I reliably choose a shard key on time series predection collection on MongoDB V5 that would distribute the docs across all shards in a sized-even way? I can not guarantee a specific field and I can not create index from _id
field on Time-series collection...