1

pg_receivewal creates a .partial file for incomplete WAL segment. My question is how to deal with this file during recovery. If I just leave it as it is, last transactions don't recover. If I rename the file removing .partial post-fix, the recovery procedure crashes with the error:

FATAL:  archive file "000000010000000C00000080" has wrong size: 152 instead of 16777216

The only way I found is to remove this .partial segment after the previous recovery crash and start the procedure again. This way db seems fully recovered.

What the proper way to deal with these .partial segments?

My db version is 11.5.

Update:

Following Laurenz advice I ended up with running next command right before starting the recovery.

  • it testes if there's no another version of the partial segment
  • decompresses it (as I use pg_receivewal with --compress flag)
  • augments it with nulls
  • and copies the fixed segment into pg_wal folder
find /backup/wal -name "*.partial" -exec sh -c ' \
    NAME=`basename "$0" .gz.partial`; \
    DEST=$PGDATA/pg_wal/$NAME; \
    test ! -f $DEST && \
    zcat $0 > $NAME && \
    truncate -s16M $NAME && \
    mv $NAME $DEST \
' '{}' \;

Notice it's relying on $PGDATA environment variable which usually isn't set.

2
  • Why is the file short? What pg_receivewal usually does is create a zero-filled full-length file, then overwrite it incrementally with real data. Why is that not working for you?
    – jjanes
    Aug 22 '19 at 15:20
  • @jjanes hmm... it's probably because of compression
    – imbolc
    Aug 22 '19 at 15:40
2

Hmm, good question.

I suggest that you rename it and fill it with zeros at the end, for example using

dd if=/dev/zero of=000000010000000C00000080 oflag=append conv=notrunc bs=1024 count=16416

This could be part of your restore_command.

2
  • so there's no an official way to do it? I'm really surprised as anyone testing their recovery procedure should encounter this problem. If I'd like to keep the backup intact, would be there any difference if I copy this last partial segment into $PGDATA/pg_wal and fill it with zeros there?
    – imbolc
    Aug 22 '19 at 11:14
  • 1
    I haven't found anything better. You can also copy it to pg_wal, but I think restore_command with a shell script is the natural way. Aug 22 '19 at 11:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.