I have an application which persists itself to a PostgreSQL 11 server. It does this by truncating the tables (removing the old state) and then uploading its current state (many statements) as part of one transaction - So, if something goes wrong the old state will be preserved. It's difficult to calculate a delta, which seemed like a sensible compromise

I've recently noticed occasional SIGKILLs wiping out the DB after my application runs for a long time (the information to save grows), with the Failed process running: being this persistence cycle. I'm wondering if this is due to the single mega transaction being a large blob in memory which eventually OOMs, and if so would breaking it up into subtransactions/savepoints address the problem? Or is there a better way to do this?

  • 1
    Large transactions aren't usually a problem, unless you have deferred constraints or are using the new statement level triggers and their "virtual" tables – a_horse_with_no_name Jun 5 '19 at 1:05
  • You should disable memory overcommit. – Laurenz Albe Jun 5 '19 at 10:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.