0

Assume there is a 7 member replica set, where each node is a voting member and no arbiter node is present. For a write w1, Using j=true and w:majority would ensure that a write is written to the journal and let's assume to the disk also after the default time interval.

If 3 nodes fail (all having the write w1 mentioned above) at the same time including the primary, only one of the remaining 4 nodes has the write w1. If this node does not get enough votes (say it's overloaded and will not be suitable for role of a primary), and some other node gets all enough votes to become primary, we have basically lost the write w1.

I know this is a rare case, but it still can happen. So isn't it true that using j= true and w:majority also will not assure that the writes are durable?

Improve this question

1 Answer 1

Reset to default
1

This is not how it works. With j:true and w: majority, w1 would only be acknowledged when the majority of nodes actually had reported to the primary that they have put the operation to the journal. Since the primary went down, this acknowledgement would never happen. So in this edge case, you would only have to do what you'd need to do what you always need to do if a failover happens before a write operation is persisted: redo it.

Let us expand the example a bit: let us assume that the write was successful, and then the nodes which already received the writes minus one go down. Now, an election takes place. The new primary is elected basically by each remaining node announcing the latest timestamp they have in the oplog. So, the node which received w1 would win (since the oplog is processed consecutively for each entry) and the remaining nodes would start to process the oplog from their latest position to the new primaries latest position - w1 will be replicated and hence it will be eventually consistent on all nodes.

Improve this answer
6
  • Thanks Markus. 1) You mentioned that "So, the node which received w1 would win (since the oplog is processed consecutively for each entry)" - This would mean that irrespective of the health of the node which has latest modifications to oplog, it will be elected as primary. 2) Contrary to my question, what if the system crashes (say power goes down) after the client is notified of SUCCESS by the primary and before the oplog is actually written to the disk files. In this case, we would still be OK, since during reconciling, the oplog will be replayed and eventually be written to the disk?
    – Aseem Goyal
    Commented Jan 3, 2018 at 9:23
  • @AseemGoyal 1) A node which is down isn’t eligible to be elected. Other than that, there id not really such a notion as health. 2) If all nodes go down simultaneously because of a power outage, I‘d first kick the systems architect for not providing a IPS. Then, there is a slight chance that on all the machines which acknowledged the write, the journal has not made it to the disk yet. On average, there is a 100ms/number of running data bearing nodes timeframe during which a write could be lost. The precondition for that is that all nodes which acknowledged the write fail within =<100ms.
    – Markus W Mahlberg
    Commented Jan 6, 2018 at 16:44
  • 1
    The 100ms mentioned is the default value for journal commit interval (storage.journal.commitIntervalMs). However, using a write concern withj:true forces a faster sync: immediate for WiredTiger and a third of the commit interval for MMAPv1 (as mentioned in docs for commitIntervalMs). MongoDB 3.6 also includes a Retryable Writes feature that simplifies handling transient network errors or replica set elections.
    – Stennie
    Commented Jan 9, 2018 at 3:23
  • @stennie Absolutely correct. But even without those admittedly important details, the numbers I provided alone clearly show that it is rather unlikely that one would loose data, even with a sloppy setup.
    – Markus W Mahlberg
    Commented Jan 10, 2018 at 19:58
  • @Stennie j:true implies that updates will be written to the journal and NOT on disk data files as per docs.mongodb.com/manual/reference/write-concern/#writeconcern.j. Therefore, the sync is not immediate.
    – Aseem Goyal
    Commented Jan 22, 2018 at 12:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.