3

According to the documentation, restart_lsn is

The address (LSN) of oldest WAL which still might be required by the consumer of this slot and thus won't be automatically removed during checkpoints...

This is not very clear. The oldest LSN required by the consumer should be confirm_flush, because that is the point upto which the client has seen the data in a durable way. So my questions are

  • How is restart_lsn initialized?
  • When is it updated? and with what value?
  • How is it used?
0

2 Answers 2

1

Perhaps I'll illustrate it like this:

ereport(LOG,
        (errmsg("starting logical decoding for slot \"%s\"",
                NameStr(slot->data.name)),
         errdetail("Streaming transactions committing after %X/%X, reading WAL from %X/%X.",
                   LSN_FORMAT_ARGS(slot->data.confirmed_flush),
                   LSN_FORMAT_ARGS(slot->data.restart_lsn))));

restart_lsn - this is the WAL position where we should start WAL decoding.

confirmed_flush - this is the WAL position up to which the recipient acknowledged receipt of the decoded transactions.

These are different things, because the WAL stream is monotonous, but when which transaction ends is unpredictable. For example:

  • at LSN X we started a transaction and update a row in the table. We cannot send this modified row to the subscriber, this transaction is not yet committed. It is not known whether it will be committed at all.
  • at LSN Y another transaction was started and changed another row
  • at LSN Z the second transaction was committed - we send the changes made in this transaction
  • there was a power failure, we managed to commit transaction 1 but did not manage to send it to the subscriber
  • we need to somehow continue logical replication

Here we have:

  • confirmed_flush = LSN Z - we managed to confirm receipt of this transaction and do not want to receive it again
  • restart_lsn = LSN X - logical decoding didn't change the replication slot position because there was a transaction in progress starting on this LSN

If we continue WAL decoding from position LXN Y or LSN Z - then we will miss the table changes made between LSN X and LSN Y.

Therefore, we should start decoding changes from restart_lsn, but only starting from the position confirmed_flush we start sending changes to the subscriber.

0

This is what I understand from the source code. Although the server can discard records older than flush_lsn, in practice, pgoutput needs record older than flush_lsn to perform the logical decoding, i.e., to translate physical changes to logical changes. These older records are related to schema changes. Thus, usually restart_lsn < flush_lsn and restart_lsn is the minimum LSN required to perform logical decoding.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.