5

I did my research on google read some blogs but I found nothing that really works in my case. After my, mysql crashed several times testing configurations, I have no option than ask for some help here.

Can somebody give me some hints about what values are recommended in my.cnf for a heavy duty database server (12 tables, more than 50G of data) with the following specifications: `

root@blah:/etc/mysql# lscpu 
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                16
On-line CPU(s) list:   0-15
Thread(s) per core:    16
Core(s) per socket:    1
Socket(s):             1
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 62
Stepping:              4
CPU MHz:               2600.092
BogoMIPS:              5200.18
Hypervisor vendor:     Xen
Virtualization type:   full
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              20480K
NUMA node0 CPU(s):     0-15
root@blah:/etc/mysql# free -m
             total       used       free     shared    buffers     cached
Mem:         64237       2747      61490         19        189       1811
-/+ buffers/cache:        746      63491
Swap: 

    2047          0       2047

This server is used only as database so I don't have to worry about anything else. Just to make sure that MySql is getting all the juice and use all resources in the best possible way.

my.cnf file I have is looking like this:

root@blah:/etc/mysql# cat my.cnf
#
# The MySQL database server configuration file.
#
# You can copy this to one of:
# - "/etc/mysql/my.cnf" to set global options,
# - "~/.my.cnf" to set user-specific options.
# 
# One can use all long options that the program supports.
# Run program with --help to get a list of available options and with
# --print-defaults to see which it would actually understand and use.
#
# For explanations see
# http://dev.mysql.com/doc/mysql/en/server-system-variables.html

# This will be passed to all mysql clients
# It has been reported that passwords should be enclosed with ticks/quotes
# escpecially if they contain "#" chars...
# Remember to edit /etc/mysql/debian.cnf when changing the socket location.
[client]
port        = 3306
socket      = /var/run/mysqld/mysqld.sock

# Here is entries for some specific programs
# The following values assume you have at least 32M ram

# This was formally known as [safe_mysqld]. Both versions are currently parsed.
[mysqld_safe]
socket      = /var/run/mysqld/mysqld.sock
nice        = 0

[mysqld]
#
# * Basic Settings
#
user        = mysql
pid-file    = /var/run/mysqld/mysqld.pid
socket      = /var/run/mysqld/mysqld.sock
port        = 3306
basedir     = /usr
datadir     = /var/lib/mysql
tmpdir      = /tmp
lc-messages-dir = /usr/share/mysql
#skip-external-locking
#
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
#bind-address       = 127.0.0.1
#
# * Fine Tuning
#
key_buffer      = 16M
max_allowed_packet  = 16M
thread_stack        = 192K
thread_cache_size       = 8
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
myisam-recover         = BACKUP
#max_connections        = 100
#table_cache            = 64
#thread_concurrency     = 10
#
# * Query Cache Configuration
#
query_cache_limit   = 1M
query_cache_size        = 16M
#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
# As of 5.1 you can enable the log at runtime!
#general_log_file        = /var/log/mysql/mysql.log
#general_log             = 1
#
# Error log - should be very few entries.
#
log_error = /var/log/mysql/error.log
#
# Here you can see queries with especially long duration
#log_slow_queries   = /var/log/mysql/mysql-slow.log
#long_query_time = 2
#log-queries-not-using-indexes
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
#       other settings you may need to change.
#server-id      = 1
#log_bin            = /var/log/mysql/mysql-bin.log
expire_logs_days    = 10
max_binlog_size         = 100M
#binlog_do_db       = include_database_name
#binlog_ignore_db   = include_database_name
#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
#
# * Security Features
#
# Read the manual, too, if you want chroot!
# chroot = /var/lib/mysql/
#
# For generating SSL certificates I recommend the OpenSSL GUI "tinyca".
#
# ssl-ca=/etc/mysql/cacert.pem
# ssl-cert=/etc/mysql/server-cert.pem
# ssl-key=/etc/mysql/server-key.pem



[mysqldump]
quick
quote-names
max_allowed_packet  = 16M

[mysql]
#no-auto-rehash # faster start of mysql but no tab completition

[isamchk]
key_buffer      = 16M

#
# * IMPORTANT: Additional settings that can override those from this file!
#   The files must end with '.cnf', otherwise they'll be ignored.
#
!includedir /etc/mysql/conf.d/

migrated from stackoverflow.com Feb 29 '16 at 16:03

This question came from our site for professional and enthusiast programmers.

2

I see you have 64GB RAM, and its all for MySQL server, which is nice. First I would suggest increasing your cache sizes, tmp_tables, etc, you can search those on Google. If you are using InnoDB tables, I suggest you tweak those too. Below I will give some suggestions, but ultimately its up to you what to do, and the best you can do is read up and test out different configs.

To check config params, use this query, modify like part or omit it at all, this way you know for sure if your settings are there:

show variables like '%log%'; 

Skip name resolve, this will make mysql faster as it doesn't need to resolve DNS, be sure that you use IP addresses when connecting to you server afterwords:

[mysqld]
skip-name-resolve

Recommended giving 64M for both values for every 1 GB of RAM on the server. This boosts performance, change these values to your needs

tmp_table_size= 2000M
max_heap_table_size= 2000M
max_tmp_tables=300

Log slow queries, long_query_time is number of seconds for query to be long, adjust accordingly

[mysqld]
slow-query-log = 1
slow-query-log-file = /var/log/mysql-slow.log
long_query_time = 1

Some settings that you need to tweak I give below.

sort_buffer_size=10M 
read_buffer_size=10M 
table_open_cache=8000
query_cache_limit=50M
join_buffer=10M

These are generally the main settings you need to tweak, numbers here are for your reference only, you should change/tweak them accordingly. Read Mysql docs for meaning of the each of those.

IF you are running phpMyAdmin, there will be Database Status, go there and it will show you red flags. Other than that you can manually check up on some parameters like this: SHOW GLOBAL STATUS LIKE 'Opened_tables';

Once in a while check if you have fragmented tables or if they need repair. This command solves it:

mysqlcheck -u root --auto-repair --optimize --all-databases

Another userful tool is mysqltuner, install and run it too.

Hope this helps.

  • Thank you. mysqltuner is all green now. I will keep the server in observation but until now is looking good. – FranMercaes Feb 29 '16 at 16:14
  • Thats good, rarely see all green )) – Muhammed M. Feb 29 '16 at 16:24
1

I disagree with these:

tmp_table_size= 2000M
max_heap_table_size= 2000M

If you write some complex queries that need multiple tmp tables, you could quickly run out of RAM. I suggest no more than 1% of RAM.

table_open_cache should be several hundred. If you set it to 8000, you are implying that you might have that many tables, which would imply mis-design of the schema.

I see nothing for InnoDB?? Surely all of your tables will use the InnoDB Engine. The main settings for it are

innodb_buffer_pool_size = 45G  -- about 70% of RAM
innodb_buffer_pool_size_instances = 16
innodb_file_per_table = ON
innodb_log_buffer_size = 64M
innodb_log_file_size = 2G

If it is a production system with lots of writes, turn off the Query cache:

query_cache_type = 0
query_cache_size = 0

Further fine tuning can be done after you have been running for some time. Please provide SHOW VARIABLES and SHOW GLOBAL STATUS.

Do not use OPTIMIZE; it is time consuming, and usually provides no benefit, especially not for InnoDB. mysqltuner has some flaws; it complains about too many tables being 'fragmented'; this is a bogus warning.

0

Somebody can't suggest a finest configuration file like this. There is many things to be known before you are starting the configuration, for example, storage engine, indexes, used queries, and even what kind of apps are being used, is it multithreaded, how much connections are used, what version of MySQL server you are using, etc. Despite of that somebody does this... Yes he is having some Great experience, something that I can't able to do.

Few things in my mind: You have 64 GB ram thats very good for the server. Your default storage engine would be innoDB here, try to add buffer size for this, this can be about 60-65% of your RAM. if you are using only MyISAM then add myisam-recover FORCE,BACKUP options also (follow dev.mysql.com for your version) You can give max_heap_table_size and tmp_table_size to 1GB Generally, increase all the buffers.

  • Thank you for your answer. In the future I will try to provide some more information. My bad. – FranMercaes Feb 29 '16 at 16:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.