I am running Postgres and WAL-E in a Docker container built from a custom image based on postgres:9.4. WAL-E works great and I have restored from backups a number of times. However, the backups take a while to run and they consume a TON of disk space. The space seems to be consumed by segments and log files, and they all get cleared out and the space is restored as soon as the restore completes. But is there any way to do the backup in batches, so that my disk doesn't fill up during restore?
It was operator error. My process to run daily base backups was not being run, so I had weeks of WAL segments, and that's what was causing it to fill up the disk and take forever. Once I got them all imported, I ran a base backup and then did the import again, and it was done in less than a minute. Do frequent base backups!