I've a server with ~200 wordpress websites hosted. When Google Favicon bot pays us a visit it searches all the sites indexed on our IP, that means it's requesting ~200 favicons. Most of the sites doesn't have one, so the request is passed to wordpress which has to render a 404 page. This means in a matter of seconds dozens of php processes and mysql connections (with different users), which causes a server overload.
I mitigated this by limiting the number of MySQL connection and it partially works, because the server is not crashing anymore, but when this happens it still gets overloaded and unresponsive for some time.
Given that I could either
- create an empty favicon.ico file for every site
- add a custom htaccess rule for favicon.ico
- enable some caching plugin on wordpress
is there a better way of controlling this issue, which doesn't involve touching websites?
Indeed I understand I didn't explain very well why I posted here and not on stackexchange.
I believe the MySQL server can be more fine-tuned in order to avoid overloading the server. As said I already limited the maximum number of connections, but I was wondering what can be done on the databases side, if there's some way to queue or semaphore them, so that when there are 200 users connected and doing basic queries the server can somehow sustain them.
The server is a quad core with 8G of ram, running Ubuntu 12.04. My custom MySQL config:
[mysqld] table_cache=1024 query_cache_size=64M join_buffer_size=512K tmp_table_size=64M max_heap_table_size=64M open_files_limit=3200 log_slow_queries = /var/log/mysql/mysql-slow.log long_query_time = 2 innodb_buffer_pool_size=600M innodb_thread_concurrency=8 thread_concurrency=8 max_user_connections = 15 max_connections = 80