1

I want to run ~10k PostgreSQL clusters.

Each cluster will be quite small with moderate traffic.

I have will have roughly 100 virtual machines. Each VM should run roughly 100 clusters. Each cluster should run in one container.

100 VMs. Each running 100 containers. One cluster per container --> 10k clusters

Features which are not available from PG out-of-the-box:

  • If I create a new cluster I need to check: Which VM has room for a new cluster?
  • I want to be able to move a cluster to a different VM

Before I re-invent the wheel: Is there already a PostgreSQL-way to handle such a scenario?

Does it make sense to use a Kubernetes Postgres-Operator or Patroni?

Background: It will be some kind of open source learning/teaching environment. Each user should get full database superuser permissions. That's why it is not enough to give each user a database. Each user needs a cluster. At least that's my current impression. Please leave a comment if you have an idea for a different architecture.

Improve this question
6
  • 1
    Hmmm... I know very little about this, but would you not be better with 100 Containers? Less overhead? Do your users require persistent storage? There's the pg_quota tool here which might be of use - should really be in the main code IMHO - it's obviously very important for enterprises... although maybe not - businesses would probably only have one primary server - dbas can keep an eye on disk space - we used to to this on Oracle! Anyway, best of luck with your project!
    – Vérace
    Commented Jan 6, 2022 at 9:26
  • thank you for your comment. I updated the question: Each cluster should run in one container. Is it more clear now? If not, please leave a comment.
    – guettli
    Commented Jan 6, 2022 at 11:06
  • You're still talking about VMs - for me, a VM is a full OS whereas a container is very different - but the distinction is becoming less hard... However, I think that you just say containers if you're not planning on running OS's on top of other OS's!
    – Vérace
    Commented Jan 6, 2022 at 11:33
  • Goal: 10k clusters. 100 VMs. Each running 100 containers --> 10k
    – guettli
    Commented Jan 6, 2022 at 16:01
  • How many real machines? What's their spec? I hope you're not planning to run all of this on a Raspberry Pi box? :-)
    – Vérace
    Commented Jan 6, 2022 at 16:24

1 Answer 1

Reset to default
2

Features which are not available from PG out-of-the-box:

  • If I create a new cluster I need to check: Which VM has room for a new cluster?
  • I want to be able to move a cluster to a different VM

Neither of the above is, could be, or should be the Postgres' responsibility. Yours is not an HA cluster, but simply a constellation of independent Postgres instances, so Patroni won't help you either.

Both features you want is exactly why container orchestration software, such as Kubernetes, exist. Whether you want to use an operator to manage Postgres instance deployments or maintain them yourself is for you to decide.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.