I have several virtual machines on Azure running a mongoDB replicaSet with the WiredTiger storage engine. I'm not sure about the proper host caching option to set on the disks that contain the MongoDB data.
Azure docs regarding disk caching state the following:
By default, the OS disks have ReadWrite caching enabled. We have recently added support for ReadWrite caching on data disks as well. If you are using ReadWrite caching, you must have a proper way to write the data from cache to persistent disks. For example, SQL Server handles writing cached data to the persistent storage disks on its own. Using ReadWrite cache with an application that does not handle persisting the required data can lead to data loss, if the VM crashes.
I use disk snapshots to backup the replicaSet. I am unsure as to how mongo manages write caches and prefer not to take the risk of getting inconsistent data in my snapshot or data loss when a vm crashes, so I disabled write caching on the disks containing the Mongo data.
Today I ran into the following note in the Mongo documentaion, under Production notes referencing Azure:
For all MMAPv1 MongoDB deployments using Azure, you must mount the volume that hosts the mongod instance’s dbPath with the Host Cache Preference READ/WRITE. This applies to all Azure deployments running MMAPv1, using any guest operating system.
I assume that if MMAPv1 knows how to handle host write-caching and flushing, WiredTiger can do the same without risking inconsistency. The production notes don't mention WiredTiger though.
Are disk snapshots of MongoDB data consistent when using WiredTiger storage engine and Azure host write-caching to the data disk is enabled?