7

I've been fortunate enough to have the use of a 20GB Linode instance running Ubuntu 64 bit. I want to try to optimize PostGres for this service, but I don't know what I should prioritize changing.

I have several datasets of 20,000 or so rows and the calculations that are being performed are memory intensive queries (spatial analyses) with a small number of rows being written after each request. The total number of users is very small (10 - 50).

I've read through this article on the Postgresql site but I don't know enough about how this works, to know what I should prioritize. I've also looked at advice on what to change for geo type work here.

For example, I tried changing the shared_buffers to 200MB (which is much less than 75% of 20GB). This resulted in the following error message:

  • Restarting PostgreSQL 9.1 database server
    • The PostgreSQL server failed to start. Please check the log output: 2013-03-10 12:21:58 EDT FATAL: could not create shared memory segment: Invalid argument 2013-03-10 12:21:58 EDT DETAIL: Failed system call was shmget(key=5432001, size=47742976, 03600). 2013-03-10 12:21:58 EDT HINT: This error usually means that PostgreSQL's request for a shared memory segment exceeded your kernel's SHMMAX parameter. You can either reduce the request size or reconfigure the kernel with larger SHMMAX. To reduce the request size (currently 47742976 bytes), reduce PostgreSQL's shared memory usage, perhaps by reducing shared_buffers or max_connections. If the request size is already small, it's possible that it is less than your kernel's SHMMIN parameter, in which case raising the request size or reconfiguring SHMMIN is called for. The PostgreSQL documentation contains more information about shared memory configuration.

I returned this to it's original value and tried changing:

work_mem = 50MB
maintenance_work_mem = 256MB 

My problem is that I don't know which values I should try changing, or how I should prioritize which values are key to experiment with and test. What should I specifically do to optimize this database?

4
  • In most cases, setting effective_cache_size to 20GB, and shared_buffers to xx MB will suffice.
    – wildplasser
    Mar 10, 2013 at 17:09
  • The error message you quoted in full (thankyou) refers to the operating system settings you must change in order to increase shared_buffers. See postgresql.org/docs/current/static/runtime-config-resource.html, postgresql.org/docs/current/static/… Mar 10, 2013 at 23:21
  • As for figuring out what you need to do, that depends on all sorts of factors. Your workload - read or write heavy? Lots of small queries or fewer big queries? Client count? Using connection pooling? Many short transactions or fewer longer ones? How big is the database on disk and how much of it is "hot" ie actively and regularly queried? What's the performance of the associated storage system like? Mar 10, 2013 at 23:23
  • @CraigRinger I do mention several of these measures in the question - I'm not very familiar with how to measure or categorize client pooling. What do you mean by associated storage system?
    – celenius
    Mar 11, 2013 at 2:40

2 Answers 2

16

I would recommend using pgtune written by Greg Smith.

Simply run it on your server as follows:

pgtune -i postgresql.conf -o postgresql-tuned.conf

It has few more options, but just doing that and using generated postgresql.conf will do wonders for your server performance.

I think PostgreSQL should include this in standard install, and even run it by default - such that people cannot complain about bad PostgreSQL performance.

12
  • 3
    Fantastic! This is exactly what I need. I'd stumbled towards some of the settings but was not familiar with all the things that I should change.
    – celenius
    Mar 10, 2013 at 20:35
  • 2
    +1 Useful tool. Didn't get updated much since Postgres 8.4. But mostly still works for the latest version 9.2. Mar 10, 2013 at 21:40
  • Once I fixed the SHMMAX problem (solution here: askubuntu.com/questions/44373/… ) pgtune worked like a charm.
    – celenius
    Mar 11, 2013 at 2:42
  • 1
    @mvp, here is a tuner I wrote.
    – Asclepius
    Feb 10, 2015 at 3:28
  • 1
    As mentioned, the CLI tool hasn't been updated for a while, but there's an online version that is up-to-date, even for PG 13: pgtune.leopard.in.ua
    – Jacob
    Apr 29, 2021 at 9:51
5

You will need to increase the shmmax, shmall, msgmax and msgmnb settings on your server. PGTUNE, as mentioned in the other answer, should tell you what to set them to.

You can do that like this:

sysctl -w kernel.shmmax={{value}}
sysctl -w kernel.shmall={{value}}
sysctl -w kernel.msgmax={{value}}
sysctl -w kernel.msgmnb={{value}}

You can check what the values already are like this:

cat /proc/sys/kernel/shmmax
cat /proc/sys/kernel/shmmall
cat /proc/sys/kernel/msgmax
cat /proc/sys/kernel/msgmnb
1
  • 1
    As a word of caution, users must remember to always first check the existing value before updating it. If the existing value is already higher, and it may very well be, it then doesn't need to be updated.
    – Asclepius
    Jan 22, 2015 at 23:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.