1

This is what MongoDB says about journaling:

To provide durability in the event of a failure, MongoDB uses write ahead logging to on-disk journal files.

But with WiredTiger:

Important

In between write operations, while the journal records remain in the WiredTiger buffers, updates can be lost following a hard shutdown of mongod.

If journaling in WiredTiger can't fully guarantee that there will not be lost updates we have to write our application to be aware that some documents may just "disappear", so why enable journaling at all, considering that it will slow down writes?

It seems that if you can't lose updates* you have to choose MMAPv1 with journaling and if you can you may prefer to choose WiredTiger without journaling.

(*) You still may lose updates in a MongoDB cluster due to rollbacks altought you can avoid rollbacks with w: majority write concern or do something with the files in the rollback/ folder.

Improve this question
4
  • 1
    You got that wrong. If you use MMAPv1 and your update is not synced to the journal files yet (every 100ms), you can loose data in case the plug is pulled or sth. The same applies for WT. Or for any DBMS which does not immediately write everything to disk (reducing performance a great deal). To prevent loss of data, increase the writeConcern to an appropriate level.
    – Markus W Mahlberg
    Mar 25, 2016 at 16:18
  • But the write isn't acknowledged until that update is synced to the journal file. docs.mongodb.org/manual/reference/write-concern/#wc-j
    – gabrielgiussi
    Mar 25, 2016 at 21:30
  • I think you've missed some important aspects of the journal approach: journal data is written in a fast append-only format which provides durability in the event of an unexpected shutdown; MMAP, WiredTiger, and other database storage engines have a similar approach in terms of committing to a journal or writeahead log before syncing updates to data files; disabling journaling generally increases your exposure to data loss unless you reduce your performance by syncing changes to the data files more frequently. There are storage tradeoffs between durability, performance, and write amplification.
    – Stennie
    Jun 22, 2016 at 6:48
  • For more context see Write-ahead logging and Journaling file system.
    – Stennie
    Jun 22, 2016 at 6:48

1 Answer 1

Reset to default
3

It doesn't matter if it's a wiredTiger or MMAPv1, they are just storage engines. Whether you lose updates or not depends on write concerns.

Your app communicates with MongoDB server that writes mostly to memory that has a cache of pages which is periodically written and read from the persistent disk. There is also a journal that logs everything.

When your write to the database it writes both to pages and journal simultaneously. Pages will write to the disk depending on memory pressure. By default you don't wait for acknowledgement from the journal because the journal may not write to the disk for a while, which is represented by j=false. And by default w=1 which means it only acknowledges write to the cache pages.

Default: w = 1, j = false - this is fast, but if something happens before the journal has a chance to write to the disk then the data is lost. What's worse is that it will show as if the data has been written/persisted if you rely on w = 1.

w = 1, j = true - in this case you wait for the acknowledgement from the journal that it has written to the disk. This is a lot slower, but ensures the data is persisted.

enter image description here

Improve this answer
2
  • Thanks for answering. You're right, I've missed this line of the journaling process in WiredTiger: If the write operation includes a write concern of j: true, WiredTiger forces a sync of the WiredTiger journal files.
    – gabrielgiussi
    Mar 31, 2016 at 13:18
  • Two notes here: journals are synced every 100ms, so data loss should be at max the inserts of the last 100ms + seek latency, with a median of 50ms + seek latency. Furthermore, wt does merge the private view into the shared view, and does not load applied inserts from disk, which this diagram could suggest. +1 anyway
    – Markus W Mahlberg
    Apr 4, 2016 at 10:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.