I have an active server running around 80-100 active websites. I would like to optimize the server and possibly reduce the amount of RAM and optimize my MySQL settings. I have 30 GB of RAM assigned and 12 Cores, CentOS7. I ran MySQLTuner and got this recommendations:
MySQL Tuner Recommendations:
MySQLTuner 1.7.19 - Major Hayden <[email protected]>
[--] Skipped version check for MySQLTuner script
[!!] Your MySQL version 5.5.66-MariaDB is EOL software! Upgrade soon!
[OK] Operating on 64-bit architecture
-------- Performance Metrics -----
[--] Up for: 31d 2h 31m 13s (309M q [115.168 qps], 1M conn, TX: 2455G, RX: 159G
[--] Reads / Writes: 95% / 5%
[--] Binary logging is disabled
[--] Physical Memory : 29.3G
[--] Max MySQL memory : 24.6G
[--] Other process memory: 0B
[--] Total buffers: 13.7G global + 22.4M per thread (500 max threads)
[--] P_S Max memory usage: 0B
[--] Galera GCache Max memory usage: 0B
[OK] Maximum reached memory usage: 14.5G (49.49% of installed RAM)
[OK] Maximum possible memory usage: 24.6G (84.00% of installed RAM)
[OK] Overall possible memory usage with other process is compatible with memory available
[OK] Slow queries: 0% (96/309M)
[OK] Highest usage of available connections: 7% (38/500)
[OK] Aborted connections: 0.59% (7936/1345660)
[!!] name resolution is active : a reverse name resolution is made for each new connection and can reduce performance
[!!] Query cache may be disabled by default due to mutex contention.
[OK] Query cache efficiency: 47.6% (263M cached / 553M selects)
[!!] Query cache prunes per day: 208303
[OK] Sorts requiring temporary tables: 0% (431 temp sorts / 2M sorts)
[!!] Joins performed without indexes: 37167
[OK] Temporary tables created on disk: 6% (31K on disk / 457K total)
[OK] Thread cache hit rate: 99% (38 created / 1M connections)
[!!] Table cache hit rate: 0% (400 open / 1M opened)
[!!] table_definition_cache(400) is lower than number of tables(34357)
[OK] Open file limit used: 0% (4/2K)
[OK] Table locks acquired immediately: 99% (45M immediate / 45M locks)
-------- Performance schema ---------
[--] Performance schema is disabled.
[--] Memory used by P_S: 0B
[--] Sys schema isn't installed.
-------- ThreadPool Metrics --------
[--] ThreadPool stat is enabled.
[--] Thread Pool Size: 12 thread(s).
[--] Using default value is good enough for your version (5.5.66-MariaDB)
-------- MyISAM Metrics -----------
[!!] Key buffer used: 18.7% (588M used / 3B cache)
[OK] Key buffer size / total MyISAM indexes: 2.9G/750.1M
[!!] Read Key buffer hit rate: 7.2% (78K cached / 72K reads)
[!!] Write Key buffer hit rate: 2.1% (2K cached / 60 writes)
-------- InnoDB Metrics ----------
[--] InnoDB is enabled.
[--] InnoDB Thread Concurrency: 0
[OK] InnoDB File per table is activated
[OK] InnoDB buffer pool / data size: 10.0G/6.1G
[OK] Ratio InnoDB log file size / InnoDB Buffer pool size: 1.0G * 2/10.0G should be equal to 25%
[!!] InnoDB buffer pool instances: 1
[--] InnoDB Buffer Pool Chunk Size not used or defined in your version
[OK] InnoDB Read buffer efficiency: 99.99% (2352937489 hits/ 2353198127 total
[OK] InnoDB Write log efficiency: 96.40% (264231435 hits/ 274111596 total)
[OK] InnoDB log waits: 0.00% (0 waits / 9880161 writes)
-------- AriaDB Metrics ----------
[--] AriaDB is enabled.
[OK] Aria pagecache size / total Aria indexes: 128.0M/1B
[OK] Aria pagecache hit rate: 100.0% (5M cached / 2K reads)
-------- TokuDB Metrics ----------
[--] TokuDB is disabled.
-------- XtraDB Metrics ---------
[--] XtraDB is disabled.
-------- Galera Metrics --------
[--] Galera is disabled.
-------- Replication Metrics -----
[--] Galera Synchronous replication: NO
[--] No replication slave(s) for this server.
[--] Binlog format: STATEMENT
[--] XA support enabled: ON
[--] Semi synchronous replication Master: Not Activated
[--] Semi synchronous replication Slave: Not Activated
[--] This is a standalone server
-------- Recommendations ----------
General recommendations:
Control warning line(s) into /var/lib/mysql/localhost.localdomain.err file
Control error line(s) into /var/lib/mysql/localhost.localdomain.err file
Increasing the query_cache size over 128M may reduce performance
We will suggest raising the 'join_buffer_size' until JOINs not using indexes are found.
See https://dev.mysql.com/doc/internals/en/join-buffer-size.html
(specially the conclusions at the bottom of the page).
Increase table_open_cache gradually to avoid file descriptor limits
Read this before increasing table_open_cache over 64: https://bit.ly/1mi7c4C
Read this before increasing for MariaDB
https://mariadb.com/kb/en/library/optimizing-table_open_cache/
This is MyISAM only table_cache scalability problem, InnoDB not affected.
See more details here: https://bugs.mysql.com/bug.php?id=49177
This bug already fixed in MySQL 5.7.9 and newer MySQL versions.
Beware that open_files_limit (2505) variable
should be greater than table_open_cache (400)
Variables to adjust:
SET innodb_stats_on_metadata = OFF
query_cache_size (=0)
query_cache_type (=0)
query_cache_size (> 500M) [see warning above]
join_buffer_size (> 128.0K, or always use indexes with JOINs)
table_open_cache (> 400)
table_definition_cache(400) > 34357 or -1 (autosizing if supported)
innodb_buffer_pool_instances(=10)
my.cnf and other cnfs
[mysqld]
innodb_file_per_table
slow_query_log
long_query_time=5
slow_query_log_file=/opt/db-backups1/mysqllog/mysql_slow.log
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
#######################################################################
#innodb_force_recovery=4
#######################################################################
max_connections = 500
thread_cache_size=2000
max_allowed_packet = 1024M
key_buffer_size = 3000M
read_buffer_size = 16M
read_rnd_buffer_size = 4M
bulk_insert_buffer_size = 256M
myisam_sort_buffer_size = 16M
myisam_max_sort_file_size = 256M
myisam_repair_threads = 1
innodb_buffer_pool_size = 10G
innodb_log_file_size = 1G
innodb_log_buffer_size = 100M
query_cache_limit = 60M
query_cache_size = 500M
query_cache_type = 1
Thanks for any recommendations, they are greatly appreciated. The error log is showing because I migrated some MyISAM databases physically from an old server and some tables that were empty did not have any corresponding table files and the system thinks that it's missing. I think the solution for that will be to export and import the MyISAM databases with the proper tool instead of physically copying it. It is important to mention there are dates and months where the access rate rises exponentially and the server should be ready for this heavy load.
EDIT 1: ADDING top and ulimit RESULT
[root@www ~]# cat topres.txt
top - 16:55:44 up 34 days, 3:59, 1 user, load average: 0,02, 0,08, 0,12
Tasks: 204 total, 3 running, 201 sleeping, 0 stopped, 0 zombie
%Cpu(s): 4,6 us, 1,5 sy, 0,0 ni, 93,9 id, 0,0 wa, 0,0 hi, 0,0 si, 0,0 st
KiB Mem : 30714928 total, 3197812 free, 13266944 used, 14250172 buff/cache
KiB Swap: 8257532 total, 7476220 free, 781312 used. 16947012 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
14463 apache 20 0 674920 42676 19516 R 43,8 0,1 0:00.34 httpd
2965 mysql 20 0 16,3g 12,0g 7744 S 18,8 41,0 1117:52 mysqld
14496 root 20 0 162048 2296 1540 R 6,2 0,0 0:00.01 top
1 root 20 0 190980 3228 2032 S 0,0 0,0 4:07.76 systemd
2 root 20 0 0 0 0 S 0,0 0,0 0:00.91 kthreadd
ulimit -a:
[root@www ~]# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 119888
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 119888
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
EDIT 2: Adding mysql tuner results after high traffic on our server. Any recommendations are greatly appreciated. We have changed the procesor amount to 16 virtual cpus.
>> MySQLTuner 1.7.19 - Major Hayden <[email protected]>
>> Bug reports, feature requests, and downloads at http://mysqltuner.com/
>> Run with '--help' for additional options and output filtering
[--] Skipped version check for MySQLTuner script
[!!] Your MySQL version 5.5.66-MariaDB is EOL software! Upgrade soon!
[OK] Operating on 64-bit architecture
-------- Log file Recommendations ---------------------------------------------- --------------------
[OK] Log file /var/lib/mysql/localhost.localdomain.err exists
[--] Log file: /var/lib/mysql/localhost.localdomain.err(29M)
[OK] Log file /var/lib/mysql/localhost.localdomain.err is readable.
[OK] Log file /var/lib/mysql/localhost.localdomain.err is not empty
[OK] Log file /var/lib/mysql/localhost.localdomain.err is smaller than 32 Mb
[!!] /var/lib/mysql/localhost.localdomain.err contains 8 warning(s).
[!!] /var/lib/mysql/localhost.localdomain.err contains 5699 error(s).
[--] 8 start(s) detected in /var/lib/mysql/localhost.localdomain.err
[--] 1) 200220 19:00:40 [Note] /usr/sbin/mysqld: ready for connections.
[--] 2) 200220 18:42:35 [Note] /usr/sbin/mysqld: ready for connections.
[--] 3) 200114 12:55:56 [Note] /usr/sbin/mysqld: ready for connections.
[--] 4) 200114 12:54:30 [Note] /usr/sbin/mysqld: ready for connections.
[--] 5) 200114 9:43:11 [Note] /usr/sbin/mysqld: ready for connections.
[--] 6) 200114 9:39:02 [Note] /usr/sbin/mysqld: ready for connections.
[--] 7) 200110 15:38:03 [Note] /usr/sbin/mysqld: ready for connections.
[--] 8) 200110 15:28:21 [Note] /usr/sbin/mysqld: ready for connections.
[--] 7 shutdown(s) detected in /var/lib/mysql/localhost.localdomain.err
[--] 1) 200220 19:00:35 [Note] /usr/sbin/mysqld: Shutdown complete
[--] 2) 200114 12:55:30 [Note] /usr/sbin/mysqld: Shutdown complete
[--] 3) 200114 12:52:41 [Note] /usr/sbin/mysqld: Shutdown complete
[--] 4) 200114 9:42:12 [Note] /usr/sbin/mysqld: Shutdown complete
[--] 5) 200114 9:38:55 [Note] /usr/sbin/mysqld: Shutdown complete
[--] 6) 200110 15:37:55 [Note] /usr/sbin/mysqld: Shutdown complete
[--] 7) 200110 15:22:13 [Note] /usr/sbin/mysqld: Shutdown complete
-------- Storage Engine Statistics -----------------------------------------------------------------
[--] Status: +ARCHIVE +Aria +BLACKHOLE +CSV +FEDERATED +InnoDB +MEMORY +MRG_MYISAM +MyISAM +PERFORMANCE_SCHEMA
[--] Data in MyISAM tables: 290.0M (Tables: 3)
[--] Data in InnoDB tables: 6.3G (Tables: 33793)
[OK] Total fragmented tables: 0
-------- Analysis Performance Metrics --------------------------------------------------------------
[--] innodb_stats_on_metadata: ON
[!!] Stat are updated during querying INFORMATION_SCHEMA.
-------- Security Recommendations ------------------------------------------------------------------
[OK] There are no anonymous accounts for any database users
[OK] All database users have passwords assigned
-------- CVE Security Recommendations --------------------------------------------------------------
[OK] NO SECURITY CVE FOUND FOR YOUR VERSION
-------- Performance Metrics -----------------------------------------------------------------------
[--] Up for: 10d 13h 51m 8s (89M q [97.936 qps], 313K conn, TX: 627G, RX: 68G)
[--] Reads / Writes: 95% / 5%
[--] Binary logging is disabled
[--] Physical Memory : 29.3G
[--] Max MySQL memory : 24.6G
[--] Other process memory: 0B
[--] Total buffers: 13.7G global + 22.4M per thread (500 max threads)
[--] P_S Max memory usage: 0B
[--] Galera GCache Max memory usage: 0B
[OK] Maximum reached memory usage: 14.0G (47.84% of installed RAM)
[OK] Maximum possible memory usage: 24.6G (84.00% of installed RAM)
[OK] Overall possible memory usage with other process is compatible with memory available
[OK] Slow queries: 0% (1/89M)
[OK] Highest usage of available connections: 3% (16/500)
[OK] Aborted connections: 0.00% (1/313391)
[!!] name resolution is active : a reverse name resolution is made for each new connection and can reduce performance
[!!] Query cache may be disabled by default due to mutex contention.
[OK] Query cache efficiency: 47.7% (76M cached / 161M selects)
[!!] Query cache prunes per day: 172476
[OK] Sorts requiring temporary tables: 0% (40 temp sorts / 658K sorts)
[!!] Joins performed without indexes: 6650
[!!] Temporary tables created on disk: 32% (71K on disk / 220K total)
[OK] Thread cache hit rate: 99% (16 created / 313K connections)
[!!] Table cache hit rate: 0% (400 open / 509K opened)
[!!] table_definition_cache(400) is lower than number of tables(33899)
[OK] Open file limit used: 0% (4/2K)
[OK] Table locks acquired immediately: 99% (12M immediate / 12M locks)
-------- Performance schema ------------------------------------------------------------------------
[--] Performance schema is disabled.
[--] Memory used by P_S: 0B
[--] Sys schema isn't installed.
-------- ThreadPool Metrics ------------------------------------------------------------------------
[--] ThreadPool stat is enabled.
[--] Thread Pool Size: 16 thread(s).
[--] Using default value is good enough for your version (5.5.66-MariaDB)
-------- MyISAM Metrics ----------------------------------------------------------------------------
[!!] Key buffer used: 18.7% (588M used / 3B cache)
[OK] Key buffer size / total MyISAM indexes: 2.9G/370.0K
[!!] Read Key buffer hit rate: 63.9% (366 cached / 132 reads)
-------- InnoDB Metrics ----------------------------------------------------------------------------
[--] InnoDB is enabled.
[--] InnoDB Thread Concurrency: 0
[OK] InnoDB File per table is activated
[OK] InnoDB buffer pool / data size: 10.0G/6.3G
[OK] Ratio InnoDB log file size / InnoDB Buffer pool size: 1.0G * 2/10.0G should be equal to 25%
[!!] InnoDB buffer pool instances: 1
[--] InnoDB Buffer Pool Chunk Size not used or defined in your version
[OK] InnoDB Read buffer efficiency: 99.92% (548727530 hits/ 549178090 total)
[OK] InnoDB Write log efficiency: 96.50% (76869307 hits/ 79658631 total)
[OK] InnoDB log waits: 0.00% (0 waits / 2789324 writes)
-------- AriaDB Metrics ----------------------------------------------------------------------------
[--] AriaDB is enabled.
[OK] Aria pagecache size / total Aria indexes: 128.0M/1B
[OK] Aria pagecache hit rate: 100.0% (295K cached / 124 reads)
-------- TokuDB Metrics ----------------------------------------------------------------------------
[--] TokuDB is disabled.
-------- XtraDB Metrics ----------------------------------------------------------------------------
[--] XtraDB is disabled.
-------- Galera Metrics ----------------------------------------------------------------------------
[--] Galera is disabled.
-------- Replication Metrics -----------------------------------------------------------------------
[--] Galera Synchronous replication: NO
[--] No replication slave(s) for this server.
[--] Binlog format: STATEMENT
[--] XA support enabled: ON
[--] Semi synchronous replication Master: Not Activated
[--] Semi synchronous replication Slave: Not Activated
[--] This is a standalone server
-------- Recommendations ---------------------------------------------------------------------------
General recommendations:
Control warning line(s) into /var/lib/mysql/localhost.localdomain.err file
Control error line(s) into /var/lib/mysql/localhost.localdomain.err file
Configure your accounts with ip or subnets only, then update your configuration with skip-name-resolve=1
Increasing the query_cache size over 128M may reduce performance
We will suggest raising the 'join_buffer_size' until JOINs not using indexes are found.
See https://dev.mysql.com/doc/internals/en/join-buffer-size.html
(specially the conclusions at the bottom of the page).
When making adjustments, make tmp_table_size/max_heap_table_size equal
Reduce your SELECT DISTINCT queries which have no LIMIT clause
Increase table_open_cache gradually to avoid file descriptor limits
Read this before increasing table_open_cache over 64: https://bit.ly/1mi7c4C
Read this before increasing for MariaDB https://mariadb.com/kb/en/library/optimizing-table_open_cache/
This is MyISAM only table_cache scalability problem, InnoDB not affected.
See more details here: https://bugs.mysql.com/bug.php?id=49177
This bug already fixed in MySQL 5.7.9 and newer MySQL versions.
Beware that open_files_limit (2505) variable
should be greater than table_open_cache (400)
Variables to adjust:
SET innodb_stats_on_metadata = OFF
query_cache_size (=0)
query_cache_type (=0)
query_cache_size (> 500M) [see warning above]
join_buffer_size (> 128.0K, or always use indexes with JOINs)
tmp_table_size (> 16M)
max_heap_table_size (> 16M)
table_open_cache (> 400)
table_definition_cache(400) > 33899 or -1 (autosizing if supported)
innodb_buffer_pool_instances(=10)
CPU and memory overview for the month.
CPU and memory overview for a day.
