1

I've been using MySQL 5.7.20 on Ubuntu 16.04.2 for a while and I've noticed that it currently uses far too much memory than what I would expect. I have around 700 tables which are all MyISAM.

I've a 128GB RAM server (which uses the memory to do other stuff) and the MySQL memory usage is around 51GB. Here is what ps aux | grep mysql gives me

mysql 7306 105 41.2 63640784 54426684 ? Ssl Dec06 5272:34 /usr/sbin/mysqld

I would expect to be more around 40GB. When I start MySQLTuner, I get the following

[--] Physical Memory : 125.8G [--] Max MySQL memory : 42.4G [--] Other process memory: 43.8G [--] Total buffers: 40.2G global + 8.8M per thread (256 max threads) [--] P_S Max memory usage: 72B [--] Galera GCache Max memory usage: 0B [OK] Maximum reached memory usage: 42.4G (33.68% of installed RAM) [OK] Maximum possible memory usage: 42.4G (33.67% of installed RAM)

This confuses me since it says that the max mysql memory is 42.4GB, whereas when I do my ps aux I have almost 10GB more!

Finally here is my mysqld.conf file

#
# The MySQL database server configuration file.
#
# You can copy this to one of:
# - "/etc/mysql/my.cnf" to set global options,
# - "~/.my.cnf" to set user-specific options.
#
# One can use all long options that the program supports.
# Run program with --help to get a list of available options and with
# --print-defaults to see which it would actually understand and use.
#
# For explanations see
# http://dev.mysql.com/doc/mysql/en/server-system-variables.html

# This will be passed to all mysql clients
# It has been reported that passwords should be enclosed with ticks/quotes
# escpecially if they contain "#" chars...
# Remember to edit /etc/mysql/debian.cnf when changing the socket location.

# Here is entries for some specific programs
# The following values assume you have at least 32M ram

[mysqld_safe]
socket      = /var/run/mysqld/mysqld.sock
nice        = 0

[mysqld]
#
# * Basic Settings
#
user        = mysql
pid-file    = /var/run/mysqld/mysqld.pid
socket      = /var/run/mysqld/mysqld.sock
port        = 3306
basedir     = /usr
datadir     = /var/lib/mysql
tmpdir      = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking

#
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address        = 127.0.0.1
#
# * Fine Tuning
#
key_buffer_size = 36G
max_allowed_packet = 64M
tmp_table_size = 4G
max_heap_table_size = 4G
max_connections = 256
table_open_cache = 16384
bulk_insert_buffer_size = 512M
thread_stack        = 192K
thread_cache_size       = 8
sort_buffer_size = 8M
myisam_sort_buffer_size = 8M

# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
myisam-recover-options  = BACKUP
#max_connections        = 100
#table_cache            = 64
#thread_concurrency     = 10
#
# * Query Cache Configuration
#
query_cache_limit   = 1M
query_cache_size        = 16M
#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
# As of 5.1 you can enable the log at runtime!
#general_log_file        = /var/log/mysql/mysql.log
#general_log             = 1
#
# Error log - should be very few entries.
#
log_error = /var/log/mysql/error.log
#
# Here you can see queries with especially long duration
#slow_query_log=1
#slow_query_log_file=/var/log/mysql/slow-query.log
#long_query_time=1
#log-queries-not-using-indexes
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
#       other settings you may need to change.
#server-id      = 1
#log_bin            = /var/log/mysql/mysql-bin.log
expire_logs_days    = 10
max_binlog_size   = 100M
#binlog_do_db       = include_database_name
#binlog_ignore_db   = include_database_name
#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
#
# * Security Features
#
# Read the manual, too, if you want chroot!
# chroot = /var/lib/mysql/
#
# For generating SSL certificates I recommend the OpenSSL GUI "tinyca".
#
# ssl-ca=/etc/mysql/cacert.pem
# ssl-cert=/etc/mysql/server-cert.pem
# ssl-key=/etc/mysql/server-key.pem
[isamchk]
key_buffer_size = 36G
sort_buffer_size = 1G
read_buffer = 4M
write_buffer = 4M

Could you help me understanding where those missing 10GB are coming from, and how can I reclaim them ?

Thanks for your help !

1 Answer 1

1

(This does not directly address the question, but points out other memory issues.)

key_buffer_size = 36G

That should be more like 20% of available memory. The key_buffer is used for indexes only, the OS is used for caching data blocks. That is, a significant chunk of RAM should not be allocated, so that MySQL has some data caching.

tmp_table_size      = 4G
max_heap_table_size = 4G

Both of those control the use of temp tables inside complex queries. 4G is dangerously high since each connection could allocate one, or even more than one, chunk of RAM of that size for a SELECT. A safer value is 1% of available RAM (for both of them).

The actual memory usage for MySQL is elusive because some things (such as that 4G) can come and go. This may partially address the missing 10GB of the original question.

For many reasons, oft discussed in this forum, you should move from MyISAM to InnoDB.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.