Recently MongoDB added hash index to its features but in the documentation doesn't explain much about hash index use case and its privileges to normal MongoDB index. So I'll be grateful if you guide me to know more about it.
1 Answer
Hashed indexes were added in MongoDB 2.4 (March 2013) specifically to support hashed shard keys. I'm not aware of any use case outside of sharding.
When choosing a shard key you can generally get the best outcome (appropriate balance of read/write locality) by defining your own compound index. However, an effective shard key requires an understanding of the attributes of your chosen field(s) (eg. cardinality, divisibility, randomness) as well as your application use case (common update/read queries). The field(s) included in your shard key index also need to be present in every document and cannot be changed after insertion.
Where there is no natural choice of shard key based on your data, a hashed shard key can be used to achieve a more uniform distribution of values which will help distribute writes across multiple shards. The field being hashed still needs to have good cardinality (i.e. large number of different values), so ObjectId values or timestamps work well. The downside of a hashed shard key is that it supports equality queries, but cannot be used for range queries since the values in the index are effectively distributed.
For example, the default _id (ObjectId) includes a leading timestamp component. While the _id field is unique, immutable, and present in every document .. an ObjectId is not a suitable choice for a shard key because the values are monotonically increasing. If you shard on an ever-increasing value like an ObjectId, all of the new inserts will target a single "hot" shard that currently has the highest shard key value. Documents will then have to be re-balanced to your other shards, which renders your sharding ineffective for scaling -- a poorly sharded collection will be limited by the write throughput of a single shard plus the overhead of frequent re-balancing of newly inserted data between shards.
For more information, see: Shard a Collection Using a Hashed Shard Key.
-
2Hm, since the hashes are signed (big?) integers, It may well be that sorting on a hashed index is more efficient. A use case for that would be if you wanted to traverse all documents in some order and wanted to prevent natural ordering, which may return a single document more than one time in large result sets. Oct 27, 2015 at 15:43
-
@MarkusWMahlberg Index type, natural ordering, and query isolation are distinct concepts. A hashed index will not affect natural ordering on disk, prevent a long running query from potentially returning the same document, or isolate cursors from concurrent inserts or deletes (query isolation depends on the storage engine implementation). You can iterate a unique index in sorted order to mitigate discovering the same document twice; MongoDB's hashed indexes are not unique so won't help with this use case.– StennieOct 27, 2015 at 19:38
-
What I was talking about that if a query is done without any sorting in place, the documents are returned in natural order, without any sort applied. For long running queries, (returning 100M docs and processing them), a document may be returned twice if no sort is applied (for example because it was balanced). See the explanation of the snapshot query modifier. This is mainly because with natural order, mongod reads record for record and returns it without dupe checks, which is different with a sort applied Oct 27, 2015 at 21:08
-
@MarkusWMahlberg Appreciate what you are referring to, but I still don't see the relation to a hash index use case. A hash index behaves the same as any other non-unique indexed field -- it does not provide any extra guarantee that a document cannot be discovered twice in a long running query. The
snapshotquery modifier avoids returning the same document twice by traversing the_idindex (which also restricts an_idfrom being updated): docs.mongodb.org/manual/faq/developers/….– StennieOct 27, 2015 at 21:18 -
The thing is that integers of a hashed index are easy to compare. Assume the default query
{}. A sort prevents documents from being returned twice, since first the index is traversed and then the documents are accessed via their mapped position in their data files. With natural order, only the datafile is traversed, from document to document. So with a hashed index, the index might be traversed faster than with ObjectId, providing the desired "once only" guarantee, but a bit faster. Just a theory. What do you think? Oct 27, 2015 at 23:27