1

I enabled hugepages on a new Postgres 13 DB, running on EL8 Linux (huge_pages=try), and noticed that Shmem is still growing and no hugepages are being used yet (but I see that the hugepages are set). THP are disabled.

I tried doing SELECT * on a large table, and shmem grew a couple GB but still zero hugepages are being used.

The last time I set hugepages on a Postgres DB, I think the same thing happened, but eventually it started using the hugepages and the shmem usage dropped down to near zero (presumably only locks and whatnot are being used in shmem).

What is the point that hugepages start getting used instead of shmem? What happens internally to drive the behavior/decision?

Presumably I need an op that uses anon pages within Postgres, perhaps SELECT * is all file pages? I would think not though. Is there a better (harmless, unobtrusive, other than RAM intensive) query to force Postgres to use a bunch of anon pages (hugepages)?

1 Answer 1

1

Currently (pg16 and yearlier) postgresql uses huge pages only for allocating a shared memory segment and, accordingly, only when starting the DBMS. This does not change in the future operation of the database.

When using huge_pages=try the DBMS at startup will first try to request a memory segment from the operating system using huge pages; if this fails, it will request a regular memory segment. Since your database was unable to allocate huge pages at startup, it means there were not enough free huge pages.

I would like to draw your attention to the fact that here we are talking about the entire shared memory segment. It is not equal to the size of shared_buffers, but 2-3% larger since shared memory segment includes many other data structures that must also be shared between database processes.

3
  • Thanks, I had a problem where Postgres 13 wasn't using the 100,000 hugepages I configured based on the 193G shmem value prior to setting it, and it turns out it needed 150,000 huge pages, otherwise it ignored the hugepages on startup (when huge_pages=try). I guess try is an all or nothing option! Commented Mar 25 at 18:37
  • Also, what options are needed to ensure that these things (where things LIKE '%the use or failure to use hugepages, etc%') are logged? Commented Mar 25 at 18:39
  • Use huge pages or refuse to start if allocation fails - this is huge_pages=on, as documented.
    – Melkij
    Commented Mar 26 at 7:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.