1

As the title said, when is the time that Couchbase write documents to disk? every 60 seconds? every 1 seconds? every 100 ms? or everytime there are read command to that document?

  • Just to be clear, are you talking about the write from the managed cache, through the disk write queue to the persistence engine or much lower level of when things are flushed to disk from buffers? – Kirk Dec 16 '14 at 20:42
  • I think both would be nice.. – Kokizzu Dec 17 '14 at 3:28
1

I will answer your question as I lay out the higher level flow objects take through Couchbase on their way to disk.

  1. Object is written to the managed cache by the application
  2. Simultaneously a pointer to that object is placed into a queue (Disk Write Queue) for the object to be persisted to the DB storage engine, currently called Couchstore. A pointer is also put into a replication queue to replicate that object to a replica, but that is another discussion.
  3. Depending on server resources, namely disk write speed, the objects will then be moved to Couchstore. The disk write queue can grow quite large if the amount of incoming data is large and the disk are slow. This is an area that needs to be looked at as part of sizing a cluster. It is not just all about raw read/write speeds, but also how the queue is filling and draining. You want the disk fill and drain rates to be as close as possible and the disk write queue to be as close to 0 as you can get it. This is especially critical if you are going to be using PERSIST_TO style writes.
  4. Once it gets to Couchstore, it is persisted to disk. There is some batching in there, 1000 per vBucket if I recall correctly. This is to improve performance. Also, while there is an OS buffer (I was incorrect in using that term before), Couchbase does not depend on it, but there is an fsync after each batch.

One thing to remember, pretty much all of this can be monitored using the Admin Rest API or looked at in the WebUI. These are all great candidates for being monitored on an ongoing basis if you care about performance.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.