Take the 2-minute tour ×
Database Administrators Stack Exchange is a question and answer site for database professionals who wish to improve their database skills and learn from others in the community. It's 100% free, no registration required.

We have a 16 shard MongoDB 2.4 installation running in AWS that is eating money.

  • The data is volatile, it only gets used within 15 minutes of creation
  • If the cluster goes down we wipe it out and restart ~ 30 seconds downtime

I note that the majority of disk stress appears to be journalling

Is there any reason I cannot disable journalling?

share|improve this question

1 Answer 1

up vote 3 down vote accepted

It should be noted that on any MongoDB system with significant write volume, most of the continuous/ongoing disk usage will come from the journal - after all it syncs to disk every 100ms - so this observation is not unusual.

If you truly can wipe out 16 shards and restore in 30 seconds (which is impressive), then you can eliminate the journal with minimal risk, yes, but you could also be less drastic. But, before you do, consider that getting rid of the journal means that a non-clean restart of a mongod leaves your data in an unknown state (it might be OK, it might not). You will want to be sure you have ways to detect any discrepancies (that might not present as an outage) if you run in this mode.

It's also worth noting that this would essentially be an unsupported mode of operation for production, so getting any help with a problem will likely mean having to turn journaling back on.

Finally, before you turn off the journal, consider these somewhat less drastic steps first:

  1. Move the journal to its own disk/volume - it's a standard write ahead journal meaning sequential writes - very different to the usual data writes, and so having it on its own disk/volume can be significantly more efficient, especially if your other IO is more randomly distributed (common with MongoDB). Finally, as of writing this, you should never have the journal on an NFS mount - it does not play well (may not apply here, but good advice in general).
  2. Change the commit interval of the journal - the default is 100ms, but you can increase to 300ms (see the docs here) - this is a minor tweak but may help
  3. Although it is not directly related to journaling, the other competing writes will come from the fsync process for data (every 60 seconds by default). If you alter this you may improve that performance also and remove stress from the disk also. Again, see the docs on syncdelay for details.
  4. Take a look at how you are writing data - sometimes large array updates, large or un-needed indexes etc. can translate into more writes than they should.
share|improve this answer
    
I guess my major concern is whether some mechanisms internal to the server rely on journalling for consistency and how that might prejudice my strategy –  Paddy Carroll Sep 5 '13 at 18:25
    
The journal is how mongod guarantees consistency, particularly in the case of a crash - that is its main purpose, hence the "unknown state" comment above. In terms of functionality, the only thing I am aware of that you will be unable to use is the j:true write concerns which returns when something is committed in the journal –  Adam C Sep 6 '13 at 8:14

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.