I have a patroni managed Postgres cluster. I've set up wal-g for creating backups in Google Cloud Storage. Everything works fine and I'm able to restore the cluster or bootstrap a new one using GCS bucket directly.
I need to ship these backups to tape and then prove that I can restore the backups from there. The problem is the volume of data. My base backups are being generated nightly and WAL archives are constantly filling up. The whole data has grown to TBs. Shipping this data to tape and restoring is posing challenges now.
I can copy a base backup from tape into cloud and restore but then the recovery keeps looking for WAL files. I'm unable to find a reasonable way of identifying the WAL files associated with a base backup so I can restore the database to a consistent state. And copying TBs of WAL files is certainly not feasible and ideal.
Can anyone suggest what is the general practice for restoring postgres from base backups and WAL files? My WAL directory is growing beyond bounds and is also a concern.