3

There is a statement in the MongoDb documentation:

You cannot use Hashed Sharding when sharding the chunks collection.

It does not say that, but I guess it means that I can only use ranged sharding. But it seems VERY inefficient. In fact, they suggest that the shard key should be the files_id or the files_id + the chunk number. They both increase monolitically. But the documentation also says, that you should not use ranged sharding when the keys increase monolitically. So there is a contradiction here.

Why on earth they do not allow me to use hashed sharding on the chunks collection? At least it should be explained in the documentation.

1

GridFS is for storing large binary data chunked into smaller documents (by default, 255KB each). The access pattern for GridFS is different from a sharded collection where random document distribution might be more desirable (for example, with a hashed shard key).

With GridFS the documents relating to a single file are normally read sequentially: identified by unique files_id and ordered by chunk number n. The supported shard keys for GridFS enable range queries based on the order that drivers will reconstruct a GridFS byte stream. A hashed shard key does not support range queries so would be extremely unhelpful for read performance (n chunk lookups would be required and data would be randomly distributed).

In fact, they suggest that the shard key should be the files_id or the files_id + the chunk number. They both increase monolitically.

The default values for these two fields indeed increase monotonically, which will lead to a hot shard for writes if you shard an fs.chunks collection using the default ObjectID value for files_id.

The chunk number is expected to be a sequence, but you can (and should) provide your own custom IDs when uploading files if you want to improve write distribution for GridFS in a sharded deployment. Official MongoDB drivers should provide an API for setting the _id when creating a new GridFS file.

At least it should be explained in the documentation.

Definitely! I've raised DOCS-10993 to improve this information in the MongoDB manual. I suspect there are also improvements to be made in the driver documentation so will review those as part of documenting the overall recommendation.

  • Very helpful answer. So the chunks collection use range sharding on monolitically increasing values. Because of this feature, a single gridfs file will be most likely stored on a single shard. So would it be benefical to generate my own (hashed) _id values. If I don't do this, then will mongodb somehow make sure that they do distribute evenly? As I see now, the ideal shard key would include a hashed file _id value plus a ranged chunk number value. Am I right? – nagylzs Nov 8 '17 at 8:14
  • 1
    In other words: I see the benefits of using ranged sharding on the chunk number, but I also see the drawbacks of using a monolitically increasing object id on the file itself. The former helps sequential access to the file contents, but the later makes an uneven distribution of files in the cluster. If we could make the file_id non-monolitic, that would help a lot. But I'm a beginner to mongodb, so please correct me if I'm wrong. – nagylzs Nov 8 '17 at 8:19
  • 2
    The default GridFS API is going to generate ObjectIDs which won't be ideal for sharding. The recommended approach (as you have surmised) would be to generate your own (hashed) _id values. You could do so with a simple md5 of an ObjectID, but any uniform hash algorithm should be fine. Irrespective of replacing default ObjectIDs data will eventually be balanced in a sharded collection, but monotonically increasing IDs would result in a "hot shard" problem where the newest chunks would always be inserted into the shard currently having the highest ID for the collection. – Stennie Nov 8 '17 at 8:21
  • 1
    For sharding GridFS you typically would not shard fs.files since that is just a small amount of metadata per file. You would shard fs.chunks on { files_id: 1, n:1} where files_id is a randomly distributed value (so new writes are distributed to different shards) and the n value support chunk ordering (and range queries) as well as some granularity for potentially splitting large files across multiple shards. – Stennie Nov 8 '17 at 8:27
  • Okay so the overall consequence is: 1. generate my own uniformly distributed file id when I create a file 2. do not shard fs.files 3. shard fs.chunks on {files_id:1, n:1} – nagylzs Nov 9 '17 at 10:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.