MongoDB disk operations with journaling disabled

Question

Assuming MongoDB 3.6, usage of WiredTiger, with journaling disabled:

How does mongod write data to hard drive? It performs write operation per every document? Per every chunk? And similarly: how about reads? Also: are those reads and writes random or sequential?

Going further: are writes made during writing journal file similar to those when creating snapshot?

Further explanation of question: I'm trying to limit usage of hard drive by Docker containers running MongoDB, so I started by benchmarking my disk following this guide: binarylane.com - How to benchmark disk I/O . It states that

Again, databases and many other programs will read very small chunks of data - 4 kilobytes is a good working estimate.

I'm wondering if this assumption is true for MongoDB.

Answer 1

There are two primary methods for WiredTiger to ensure the durability of your data: journaling and checkpointing.

Checkpoints store the valid and consistent state of the whole database. Should the server crash for any reason, it can restart from the last checkpoint in a consistent state. WiredTiger by default checkpoints the database every 60 seconds.

Journals store the writes that happened in-between checkpoints. If the database crashed between checkpoints, the journal data will be replayed on top of the last known good checkpoint. The journal is persisted to disk every 50 ms. The journaling process is optimized to be a lot more lightweight compared to checkpoints (and it also does a lot less work), so it's typically much faster than a checkpoint.

If journaling is disabled, the behaviour is slightly different between a mongod running in standalone mode or in replica set mode:

• MongoDB 3.6.4 standalone with journaling disabled: only the complete state of the whole database is written to disk on every checkpoint. That is, if your database crashes between checkpoints, you can potentially lose the last 60 seconds of your writes (if using default checkpoint timing values).

• MongoDB 3.6.4 replica set with journaling disabled: by design, a replica set requires more durability since it must record every operation in the oplog and ensure that the oplog is safely persisted. Thus, it will perform a checkpoint on every write instead. This will slow down the replica set tremendously.

The tradeoff you make by disabling journaling:

• In a standalone deployment, since the main goal of a database is to persist your data, limiting disk usage (space or IOPS) will result in more data being lost between allowed writes. However, it's not recommended to run a standalone MongoDB for production purposes.

• In a replica set, the write process is considerably slowed down due to the need to persist the oplog. This will also make the deployment use more disk IOPS. In the future versions of MongoDB, running a replica set with no journal will not be allowed (see SERVER-30347).

For more information regarding journaling, please see the Journaling Process page.

Answer 2

How does mongod write data to hard drive? It performs write operation per every document? Per every chunk? And similarly: how about reads? Also: are those reads and writes random or sequential?

As MongoDB BOL Here Journaling Process Changed in version 3.2.

With journaling, WiredTiger creates one journal record for each client initiated write operation. The journal record includes any internal write operations caused by the initial write. For example, an update to a document in a collection may result in modifications to the indexes; WiredTiger creates a single journal record that includes both the update operation and its associated index modifications.

MongoDB configures WiredTiger to use in-memory buffering for storing the journal records. Threads coordinate to allocate and copy into their portion of the buffer. All journal records up to 128 kB are buffered.

For the journal files, MongoDB creates a subdirectory named journal under the dbPath directory. WiredTiger journal files have names with the following format WiredTigerLog.<sequence> where <sequence> is a zero-padded number starting from 0000000001.

Journal files contain a record per each write operation. Each record has a unique identifier. MongoDB configures WiredTiger to use snappy compression for the journaling data.

Note : To provide durability in the event of a failure, MongoDB uses write ahead logging to on-disk journal files.

As MongoDB documented Here A sequential, binary transaction log used to bring the database into a valid state in the event of a hard shutdown. Journaling writes data first to the journal and then to the core data files. MongoDB enables journaling by default for 64-bit builds of MongoDB version 2.0 and newer. Journal files are pre-allocated and exist as files in the data directory.

Going further: are writes made during writing journal file similar to those when creating snapshot?

If you place the journal on a different filesystem from your data files, you cannot use a filesystem snapshot alone to capture valid backups of a dbPath directory. In this case, use fsyncLock() to ensure that database files are consistent before the snapshot and fsyncUnlock() once the snapshot is complete.

Note : db.fsyncUnlock() is an administrative operation.

For Further your ref Here Here

Answer 3

While not a full answer to your question:

The WiredTiger storage engine does not perform a disk operation for each write, instead it flushes data to disk at intervals of 60 seconds. More detailed information can be found here.

Read operations will first look for data in the WiredTiger cache and then pull data into cache as needed.

Journal behavior is different than that of snapshots. Journal write behavior is described here.