0

I am running SQL Server 2019 CU15 on Linux in kubernetes and trying to investigate why SQL Server pod gets killed by OOMKiller. I see some inconsistency in reporting of memory consumption. SQL Server POD has a limit of 16 Gigs of RAM (both as a limit and as a startup argument) now and only 14000 as Max Server Memory. Currently Target Server Memory is reported as 11397920 Kb, which is aligned with what I get from memory clerks by summing up. So inside of SQL Server it all looks consistent. If I query sys.dm_os_process_memory then virtual_address_space_committed_kb equals to 11410248 which is almost the same number. But I can't get that order of number from OS level by any means. I am using image from Microsoft, so I cannot add anything for extended diag. RSS value in ps shows 15 gigs. same is for top. I have a gut feeling that this difference is exactly what crosses the limit and causes OOMKiller to kill the pod. Knee-jerk would be to add this gap to any SQL Server pod's limit while keeping max server memory the same. But I still want to have an understanding of what is that extra mem used for and how to track it internally?

    $ ps aux
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
1000630+       7  0.0  0.0  61636 23524 ?        Sl   Aug03   0:00 /opt/mssql/bin/sqlservr
1000630+       9 26.9 11.4 278592548 15122076 ?  Sl   Aug03 12325:54 /opt/mssql/bin/sqlservr
1000630+  400010  0.0  0.0   2628   544 pts/0    Ss   11:15   0:00 sh -i -c TERM=xterm sh
1000630+  400016  0.0  0.0   2628   548 pts/0    S    11:15   0:00 sh
1000630+  400019  0.0  0.0   6192  3388 pts/0    S+   11:15   0:00 top
1000630+  400254  0.0  0.0   2628   612 pts/1    Ss   11:42   0:00 sh -i -c TERM=xterm sh
1000630+  400260  0.0  0.0   2628   612 pts/1    S    11:42   0:00 sh
1000630+  400262  0.0  0.0   6192  3296 pts/1    S+   11:42   0:00 top
1000630+  400880  0.0  0.0   2628   544 pts/2    Ss   12:51   0:00 sh -i -c TERM=xterm sh
1000630+  400886  0.0  0.0   2628   544 pts/2    S    12:51   0:00 sh
1000630+  400954  0.0  0.0   5912  2904 pts/2    R+   12:59   0:00 ps aux
Improve this question

1 Answer 1

Reset to default
2

But I still want to have an understanding of what is that extra mem used for and how to track it internally?

You really don't/can't, easily. You can absolutely use profiling tools as normal but it's not going to be that helpful. SQL Server on Linux is essentially SQL Server on Windows, wrapped in a emulation layer so that it can be used on Linux. That will need its own memory for regular usage and startup, tracking all of the internal Windows or external Linux items to make translations, etc. so it's not going to use the same amount of memory as it would on Windows.

Additionally, the clerks don't hold everything about the memory usage. There are items that are not contained in there, such as threads, emulation and tracking (above), external allocators, heap usage, etc.

Personally, I'm not a fan of SQL Server on Linux, and honestly if you're going to use SQL Server on Linux then you'll want to disable oomkiller. What's the point of killing a process you know is designed to use as much memory as possible?

Improve this answer
1
  • Thanks for your answer Sean. I have a similar impression overall. The problem is that this is Openshift (Kubernetes) cluster and resources are shared. We cannot disable OOMKiller completely. Also this looks impractical to add 8 extra gigs to 16 Gigs of mem, used by SQL Server now. But it seems that this is really the only option available now.
    – Oleg Lavrov
    Sep 5 at 7:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.