I am trying to devise a scheme with minimal repair frequency to not overload the system, and have a problem to understand how can I ensure that the data is not getting resurrected.
I have a cassandra scheme with
gc_grace_seconds specification of 48 hrs (172800 seconds)
I know it is recommended that the repair job
[must be run before gc_grace expires to ensure deleted data is not resurrected]
I run the nodetool repair with the
–pr key, so that it take less time to complete (it takes about 1.5 hours per node with the –pr option and around 6-7 hours per node without the
Now, I don't see a scenario where I can be sure the data is not becoming visible again after the tombstone is invalidated.
In my understanding, if we create a tombstone at a date
1/1 00:00, it will be gone on
If the repair job is scheduled to start every 2 days at the same times, the repair will start at
3/1 00:00 and until it will take care of the disappeared tombstone, we have a period that the data is resurrected.
If the repair job is scheduled to run, say, every day at the same times, then still we have the same problem - there will be a delay between the tombstone invalidation and nodetool repair action which will start at
00:00 every day.
If we make
gc_grace_period more then 48 hours, say, 60 hours, then in my understanding the "resurrection period" will get even longer - the tombstone will get expired at
3/1 12:00 and "wait" till
4/1 00:00, even if we run the repair every day.
So how will the nodetool take care of the tombstones so that there is no period that the data is going back to life?