Can I disable journalling to improve performance

Question

We have a 16 shard MongoDB 2.4 installation running in AWS that is eating money.

• The data is volatile, it only gets used within 15 minutes of creation
• If the cluster goes down we wipe it out and restart ~ 30 seconds downtime

I note that the majority of disk stress appears to be journalling

Is there any reason I cannot disable journalling?

Answer 1

It should be noted that on any MongoDB system with significant write volume, most of the continuous/ongoing disk usage will come from the journal - after all it syncs to disk every 100ms - so this observation is not unusual.

If you truly can wipe out 16 shards and restore in 30 seconds (which is impressive), then you can eliminate the journal with minimal risk, yes, but you could also be less drastic. But, before you do, consider that getting rid of the journal means that a non-clean restart of a mongod leaves your data in an unknown state (it might be OK, it might not). You will want to be sure you have ways to detect any discrepancies (that might not present as an outage) if you run in this mode.

It's also worth noting that this would essentially be an unsupported mode of operation for production, so getting any help with a problem will likely mean having to turn journaling back on.

Finally, before you turn off the journal, consider these somewhat less drastic steps first:

1. Move the journal to its own disk/volume - it's a standard write ahead journal meaning sequential writes - very different to the usual data writes, and so having it on its own disk/volume can be significantly more efficient, especially if your other IO is more randomly distributed (common with MongoDB). Finally, as of writing this, you should never have the journal on an NFS mount - it does not play well (may not apply here, but good advice in general).
2. Change the commit interval of the journal - the default is 100ms, but you can increase to 300ms (see the docs here) - this is a minor tweak but may help
3. Although it is not directly related to journaling, the other competing writes will come from the fsync process for data (every 60 seconds by default). If you alter this you may improve that performance also and remove stress from the disk also. Again, see the docs on syncdelay for details.
4. Take a look at how you are writing data - sometimes large array updates, large or un-needed indexes etc. can translate into more writes than they should.