0

I have a DB which holds very simple documents called "Item":

{_id:<ObjectId>, playerId:int, isRead:bool, content:string}

containing about 80M documents. Each Item belongs to a certain player, defined by a monotonically incrementing playerId. Each player has up to 1,000 documents in the collection.

I want to shard the collection based on hashed playerId, since the read queries are based on playerId. This way, I believe I will be able to get to the right node in the cluster efficiently, then get the relevant documents (worst case, 1000) and retrieve them.

I was advised not to use hashed playerId because it's not unique, and might retrieve a large amount of documents per query, since isRead is not included in the index. Queries that retrieve by playerID and isRead might impose an efficiency issue.

Is this indeed a problem? Are there any other considerations to think of?

Improve this question
5
  • Why not shard on a custom _id value? You can still put an index on playerId which should allow for good performance. Really, you should do some testing to see what works best as it may depend largely on your number of shards and overall collection size.
    – LowlyDBA - John M
    Commented Feb 3, 2015 at 19:37
  • What you mean retrieve a large amount of documents? you said that in the worst case are 1000 per playerId so your worst case is 1000 docs if you looking for a particular playerId. How much the isRead reduce the number of documents?
    – Antonios
    Commented Feb 3, 2015 at 23:14
  • @JohnM - I want to shard on the field that is used in most of the queries. Does using hashed _id + an index on playerId have the same performance as only hashed playerId?
    – Nir
    Commented Feb 5, 2015 at 12:20
  • @Antonis - isRead reduces 80%-90% of the documents if it's true.
    – Nir
    Commented Feb 5, 2015 at 12:20
  • @Nir An index on playerID should be more than sufficient for this, but again you really should do some testing to make sure. It depends on your architecture which we don't know yet.
    – LowlyDBA - John M
    Commented Feb 5, 2015 at 13:47

1 Answer 1

Reset to default
0

Why not shard on a custom _id value since it's guaranteed to be unique? With the size around 1,000 an index on playerId should allow for good performance as well.

Still do some testing of course because it can still vary based on number of shards and overall size of the collection.

Found a video I was looking for at the time from the Mongo University M102 course: Cardinality & Monotonic Shard Keys, which provides some more insight.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.