25

In mongoDB3 appeared a new storage engine: WiredTiger. Yet, MMAPv1 is still the default choice in Mongo.

One might not be better than the other, it's often a matter of use case and choosing the right tool for the job. But which engine is right for what job?

In fact, while MMAPv1 is the default engine, WiredTiger seems better in almost every field. It has the same features as MMAPv1 plus:

  • better write performance,
  • document level concurrency,
  • compression,
  • snapshots and checkpoints system.

I found a comparing table on MongoDB's blog:

WiredTiger and MMAPv1 comparison

So except if you are on Solaris, is there a reason not to choose WiredTiger?


EDIT

Here are two videos that explain in details the internals of WiredTiger and MMAPv1.

26

Personally, I prefer the mmapv1 storage engine as of now for three reasons.

Reason 1: Maturity

It isn't that WiredTiger is immature. But mmapv1 is well understood and battle tested all way up and down, back and forth and above and beyond. WiredTiger has had some serious issues (see http://jira.mongodb.com for details) fairly recently, and I am not willing to have my customers find the next one the hard way.

Reason 2: Features

Given, WT has some f... undamentally awesome features. The thing is: I have not seen anyone benefitting from them. Compression? Either way, you sacrifice rather hard to achieve performance for rather cheap disk space. Lack of the document migration problem for expanding documents? Well, we still have the 16MB size limit and added complexity for embedded documents, especially when embedding is overdone.

There are other features, but in general: I do not see to much benefit from them as of now.

Reason 3: Total cost of ownership

For new projects, WT might be fine, especially since 3.2 , since the following does not apply.

Doing data migrations is expensive. It needs to be planned, the plan needs to be agreed upon by all stakeholders, emergency contingency plans have to be created and agreed upon, the migration needs to be prepared, executed and reviewed. Now multiply the time needed with the stakeholders who are part of this process, and the costs for the data migration skyrocket. The return on investment on the other hand seems rather small. You can scale quite a bit instead of doing a migration if you take those factors into account. To give you an impression: I'd estimate roughly one "man-week" per stakeholder if a migration is planned, executed and reviewed properly. With costs of $100 per hour per person, and only three people involved (manager, DBA and developer), that amounts to $12.000. Note that this is a conservative estimate.

Conclusion

All those factors above have brought me to the conclusion not to use WT whatsoever. At the moment.


Update

This post is some months old now, so it deserves an update

On maturity

My original comments on maturity are sort of obsolete. WiredTiger has not had any major issues for a while now and has become the default storage engine as of MongoDB 3.2

On Features

My original comments still hold some validity, imho.

Compression

However, when being tight on budget or, more generally speaking, performance is not the primary concern, the performance tradeoff is rather small, and you basically trade slight performance impacts (when compared to uncompressed WT) for disk space, utilizing what otherwise would idle around: the CPU.

Encryption

MongoDB 3.2 Enterprise introduced the ability to have WiredTiger storages encrypted. For data with enhanced security needs, this is a killer feature and makes WT the only storage engine of choice, both technically (MMAPv1 does not support encryption) and conceptually. Setting aside the possibility of encrypted disk partitions, of course, although you might not have that option in some environments.

Document level locking

I have to admit that I basically omitted that feature of WT in my above analysis, mainly because it did not apply for me or my customers when I wrote the original answer.

Depending on your setup, mainly when you have many concurrent writing clients, this feature may provide a great performance boost.

On Total cost of ownership

Doing migrations is still expensive. However, taking the changes in maturity and the changed view on the features into account, a migration might be worth the investment if:

  • You need encryption (Enterprise Edition only!)
  • Performance is not your absolute primary concern and you can save money in the long run (calculate conservatively) using compression
  • You have a lot of processes writing concurrently, since the increase in performance might save you vertical or horizontal scaling.

Updated Conclusion

For new projects, I use WiredTiger now. Since a migration from a compressed to an uncompressed WiredTiger storage is rather easy, I tend to start with compression to enhance the CPU utilization ("get more bang for the buck"). Should the compression have a noticeable impact on performance or UX, I migrate to uncompressed WiredTiger.

For projects with a lot of concurrent writers, the answer to wether to migrate or not is almost always "Yes", too – unless the project's budget prohibits the investment. In the long run, the performance increase should pay for itself, if the deployment was otherwise reasonably planned. However, you need add some development time to the calculation, since in some cases the driver needs to be updated, and there might be problems which need to be dealt with.

For projects which are tight on budget and can not afford more disk space for the moment, migrating to a compressed WiredTiger can be an option, but the compression puts a bit load on the CPU, something unheard of with MMAPv1. Furthermore, the migration costs might be prohibitively expensive for such a project.

  • Thank you Markus for your anwser. I understand your arguments. Would you even advise to defaults back to MMAPv1 for new projects? I mean, if performance is a concern, compression can also be totaly disabled. Disk space is cheap, but compression might help working set fit in RAM… and thus gain in perf. Or am I wrong? – Buzut Nov 16 '15 at 10:20
  • 1
    As far as I know, compression is only applied to the data files. As for new projects, let me put it that way: I'd call for a management decision, displaying pros and cons. I personally use WT in one of my projects and did not run into problems yet, but there might be other factors such as SLAs (which I can calculate pretty good based on experience with mmapv1), tight budgets (which would call for compressed WT to save disk space) and lots of other factors. Weighing risks and opportunities is not a DBAs decision. A DBA has to provide the information needed to make a call. – Markus W Mahlberg Nov 16 '15 at 10:44
  • 1
    This article seems to indicate that the way indexes are stored is a pretty huge benefit because it can reduce the space your indexes take up by 10 times: ilearnasigoalong.blogspot.com/2015/03/… . Disk space is cheap, but ram isn't. – B T Dec 11 '15 at 2:07
  • I'm a bit confused about your point on Features, are you saying there are features MMAPv1 has that WiredTiger doesn't? It would be nice if that section was a little clearer. – B T Dec 11 '15 at 2:11
  • @BT Not at all. What I was trying to say is that WT has some pretty cool features which for some use cases may be helpful, but are not in general. I prefer a battle-tested storage engine over the cutting edge when it comes to data storage. As per the article: Yes, it is possible with the prefix compression to save RAM, and this may well be a good idea if there weren't other concern. Imagine you could be held reliable for data loss or performance issues. – Markus W Mahlberg Dec 11 '15 at 9:21
5

My two cents:

Journaling in WiredTiger can lose updates in hard shutdowns because it uses in-memory buffering for storing the journal records.

In between write operations, while the journal records remain in the WiredTiger buffers, updates can be lost following a hard shutdown of mongod.

Journaling in MMAPv1 writes changes in on-disk journal files.

If the mongod instance were to crash without having applied the writes to the data files, the journal could replay the writes to the shared view for eventual write to the data files.

4

We shifted to WiredTiger from MMAPv1 on the lure of 7x to 10x performance gain. We had to revert back to MMAPv1 as MongoDB would lock up when the WiredTiger cache would hit 100%. We've documented our experience here - https://blog.clevertap.com/sleepless-nights-with-mongodb-wiredtiger-and-our-return-to-mmapv1/

  • 2
    Nice writeup. Kinda reminds me of Uber going from MySQL to PostgreSQL in 2013 and then going back to MySQL in June 2016. – RolandoMySQLDBA Sep 28 '16 at 15:13
  • well explained we also have same experience with wired tiger so revreted it to MMAPV1 – viren Dec 13 '16 at 10:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.