1

This might not be the correct board for this, but since I'm a DBA and I'll ask here.

I have several RHEL 7.4 servers, running a mix of MariaDB (10.1, 10.2) and MongoDB (3.4). The problem was happening with RHEL 7.3 as well.

All of these servers have 256Gb of memory with local SSD array storage, and even given heavy workloads the highest active memory-in-use footprint is less than 100Gb at any one time. I've been profiling this for quite some time:

On each of these servers, even though there is always plenty of free memory available, the systems are incrementally going into swap. I've tried setting the vm.swappiness value to 1, but incremental jumps of swap are still happening.

Is this happening to anyone else? Does anyone know, with a large amount of memory available, if setting swappiness to 0 has ill effects?

Thanks

5
  • 1
    What do you mean by "going into swap"? Do you actually see paging happening, or are you simply talking about the swap space utilization?
    – mustaccio
    Jan 12, 2018 at 21:17
  • If you have heavy MySQL workload, do you have the slowlog turned on?
    – Rick James
    Jan 12, 2018 at 21:28
  • I have a datadog agent that tracks swap space available. Our swap files are standardized at 8Gb. Each night, with my heaviest workload - MariaDB 10.2, with a mix of InnoDB and MyISAM tables, with plenty of memory free, the swap usage increases a little. Last night, for example, memory used never crested 72Gb but swap usage increased almost 100Mb. This will happen incrementally until all swap is used and never gets released/cleared. This behavior also exists with my MongoDB instances.
    – zstokes
    Jan 12, 2018 at 21:49
  • Further, I know that I have some slow queries, but that's expected: this process trolls all of our NFS filesystems, down a link tree to each file and feeds a drillable, graphical web chart. It does this each night and overwrites the database each night, about 1Tb in size.
    – zstokes
    Jan 12, 2018 at 21:51
  • 1
    You didn't answer if you observed paging. Non-zero swap utilization by itself is not a problem
    – mustaccio
    Jan 13, 2018 at 13:47

2 Answers 2

0

If you have 8GB swap, 256 GB physical memory and swappiness=1, with memory usage nearing 72Gb, it's acceptable to swap operations almost 700MB (72GB * 1%), then the 100MB swap usage that you had seen is pretty normal.

Be sure that your 1TB daily data load doesn't exhaust your physical memory and you can change swappiness to 0 without side effects.

But do you have a quite low swap rate in your setup. Do you really need to avoid swap?

3
  • It's not that I'd mind occasional swap usage if that space got freed up, but it just incrementally grows and grows until it consumes the swap file. So if I grew the size of it, it would take longer but the problem would persist. The system has 256Gb of memory, well over 150Gb is always free. Swap shouldn't be used at all.
    – zstokes
    Jan 16, 2018 at 20:34
  • 1
    Did you tried to change swapiness to 0?
    – Rogerlr
    May 1, 2018 at 19:03
  • I have done that, which alleviated the swap usage for this particular workload, which I know will never reach physical memory and OOM the process after some careful study.
    – zstokes
    Jul 12, 2018 at 20:04
0

Don't worry. Unless memory gets swapped back in from swap, there is no problem. You can reduce vm.swappiness even more, down to 0, but if the system still swaps without memory pressure, it is just a Linux oddity. Again, don't worry unless you see high disk activity that indicates that memory is swapped in.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.