0

Looking at the MongoDB documentation I see that users are discouraged from using MapReduce in favour of the Aggregation pipeline.

For example, on this page in the documentation.

But the same notice is ambiguous in terms of whether the Aggregation Framework completely replaces MapReduce or not:

Various map-reduce operations can be rewritten using aggregation pipeline operators, such as $group, $merge, etc. For map-reduce operations that require custom functionality, MongoDB provides the $accumulator and $function aggregation operators starting in version 4.4.

Does this mean that "every conceivable" MapReduce configuration can be replicated via the Aggregation Framework or not?

I understand that the MapReduce execution context might be slower than the Aggregation pipeline since the former requires a JavaScript engine which is/was/still is? slower than the Aggregation pipeline implementation.

Unlike other features, MapReduce has NOT been labeled as deprecated so far as I know. So, my question is:

Is MapReduce completely replaced (for every single possible use-case) by the Aggregation Framework? Or are there use-cases where MapReduce still provides for greater flexibility?

2 Answers 2

2

Starting in MongoDB 5.0, Map-Reduce has finally been deprecated. JavaScript function evaluation has more overhead than using inbuilt aggregation operators, and in general is strongly discouraged unless there is no alternative. In addition to needing a JavaScript engine to evaluate custom JS functions, there is overhead translating between MongoDB's native BSON document format and JavaScript objects. Aggregation operators work directly with BSON documents and the pipeline approach enables Aggregation Pipeline Optimization such as automatically reordering or coalescing stages where possible.

Does this mean that "every conceivable" MapReduce configuration can be replicated via the Aggregation Framework

As far as I'm aware, yes. Two of the most significant features required to unblock deprecation were support for custom JavaScript accumulators and functions, which were added to the aggregation framework in MongoDB 4.4. Aggregation features introduced in earlier releases included merging results into a specifed collection and regular expression pattern matching. Aggregation now has features beyond Map-Reduce including Window Operators, document sampling, and Set Expression Operators.

Borrowing the deprecation callout from the MongoDB 5.0 Map-Reduce documentation:

  • Instead of map-reduce, you should use an aggregation pipeline. Aggregation pipelines provide better performance and usability than map-reduce.
  • You can rewrite map-reduce operations using aggregation pipeline stages, such as $group, $merge, and others.
  • For map-reduce operations that require custom functionality, you can use the $accumulator and $function aggregation operators, available starting in version 4.4. You can use those operators to define custom aggregation expressions in JavaScript.

For examples of aggregation pipeline alternatives to map-reduce, see:

3

The answer is no, the aggregation framework does not replace "every conceivable" MapReduce configuration. Without becoming a general purpose MapReduce framework with fully customizable functions, it can't. That's what you get with the existing MapReduce functionality - any functions you want (as long as they can be written in Javascripts of course).

I haven't used it in a long time (years), but it is not just because of the use of the javascript engine that the MapReduce funcionality is slower, it still does things like taking a global write lock (docs) when writing output which will block all other operations on a mongod - ouch. Even several version prior to this, when the Aggregation Framework was nowhere near as featureful as it is now the advice was: if you can use the aggregation framework, do so, only use MapReduce if you have no other choice.

1
  • 1
    Thanks. I had come across that in the docs, but it never really hit home.
    – Zach Smith
    Aug 25, 2020 at 6:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.