1

I'm currently deploying an application to a Kubernetes cluster that has x86 and ARM machines. My applications are usually packaged to support both CPU architectures.

However, this application depends on Postgres. Without special configuration, Kubernetes could schedule it to run on either the x86 machines or the ARM machines, and the data directory would be directly moved between them.

Right now I've set it up to only run it on one kind of machine, to prevent any issues with binary incompatibility of the data files. I know that there are such incompatibilities between major versions of Postgres.

However, I'm curious: if I'm using the same version of Postgres on all the machines, then is it possible to use the same on-disk data files from different architectures -- for example, inserting data on an x86 machine, then stopping the server process, and moving the data directory to an ARM machine and reading the data there?

Improve this question
5
  • ARM and x86 / AMD64 share "little-endianness", so at least numbers should basically agree. But I am not sure whether all file formats agree as well. And there are more possible complications. Postgres can rely on OS libraries for encoding and locale information, with possible ramifications on physical storage and index sort order. And I doubt that OS versions for ARM and AMD64 will always be in perfect sync. I am not sure if it's possible, but I am very skeptical.
    – Erwin Brandstetter
    Commented Oct 1, 2023 at 19:24
  • @ErwinBrandstetter It turns out that my container actually did run on the wrong architecture for a bit, and a couple rows that were there survived this. But I would like to have some more confirmation before I put production data through that. What I'm interested in is, are there any particular portability guarantees made with regards to this format, like there are with SQLite.
    – Danya02
    Commented Oct 1, 2023 at 20:16
  • I know the data directory is not portable between Windows and Linux. See: dba.stackexchange.com/a/98245/3684 But I don't know if it could be portable between two instances with the same OS (they can never be exactly the same), but different CPU architectures. I certainly would not try with different OS versions, or different Postgres versions (like you found yourself already).
    – Erwin Brandstetter
    Commented Oct 1, 2023 at 20:35
  • I second Erwin's doubt, and I would certainly not risk it with data that are important to me. In addition, as mentioned, the version of glibc (or libicu, whatever is in use) must be identical on both systems.
    – Laurenz Albe
    Commented Oct 2, 2023 at 2:32
  • Hi, and welcome to dba.se! SQLite specifically guarantees that it will work across platforms. PostgreSQL does not make any such guarantee! Failing that, it would be a brave man or woman who would risk their data on such a wish and a prayer! You could try asking this on the pg-hackers mailing list, but I'd bet my paycheques from now till Christmas that the answer will be no! Best of luck...
    – Vérace
    Commented Oct 2, 2023 at 3:05

1 Answer 1

Reset to default
-1

The answer to your question - today - seems to be yes, but ...

Do not assume that it is.
Tick; you've asked the question.

Do not assume that it will remain so forever.
"Forever" is a painfully long time in Computing.

Stop thinking about your database as a bunch of files.
SqlLite maybe, but anything that uses a server process that you "connect" to? Let its files well alone.
I do not pretend to understand your setup, but moving a Postgres Data Directory around based on some "scheduling" criteria strikes me a really Bad Idea.
If (and when) you have to move a Database, use Database Tools to do so and issues like this simply disappear. You'll also avoid a bunch of other problems as well.

Improve this answer
3
  • That's not a very conclusive answer.
    – Laurenz Albe
    Commented Oct 2, 2023 at 8:04
  • The way my setup works is: persistent data is stored on a block device file (provided by Longhorn and replicated across multiple machines). A random-ish machine is selected to run the Postgres program, and at that point it downloads that block device and mounts it. Changes to the block device are propagated quickly, and if the machine currently running Postgres goes down, then another machine is selected to run Postgres on the same data volume. This setup means that the one machine running the Postgres program can be replaced quickly if it crashes with no data loss.
    – Danya02
    Commented Oct 4, 2023 at 18:56
  • DATABASE != FILE. Just because an update has been committed in Postgres DOES NOT mean that that change has been written to the data files on your block device. Postgres does a LOT of stuff in memory and only writes to disk when it has the time, or a specific need, to do so. This is highly risky and I fully expect you WILL lose data. Switch out the application machine as much as you like but NOT your database machine - or use an inferior, file-based "database", like SqlLite.
    – Phill W.
    Commented Oct 5, 2023 at 6:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.