I'm suddenly having intermittent 2003 connection errors and I can't find anything that has changed that can explain it. These servers have been running smoothly for years and suddenly two days ago these errors started showing up in my logs. The application successfully connects to and queries the database almost all the time but sporadically hangs when trying to connect after several queries in quick succession. The connection timeouts always happen at random intervals. It worked just fine before two days ago and the code hasn't changed in 4 years.
Things I've tried that didn't work:
- Checked all InnoDB tables for corruption
- Increased the
back_logvalue in my.cnf from 50 to 1000 (and restarted MySQL) - Increased the
max_connectionsvalue in my.cnf from 100 to 200 (and restarted MySQL) - Increased
net.core.somaxconnfrom 128 to 1024 - Increased
net.ipv4.tcp_max_syn_backlogfrom 1024 to 8192 - Verified there are no firewall rules blocking or throttling connections
- Flushed the MySQL query cache
- Cleared the MySQL query cache
- Restarted the application server and database server (there are two separate servers)
- Asked it nicely to simply start working again
My servers are EC2 instances in AWS and nothing about their configuration has been changed. There's no unusual traffic and the CPU, memory, and disk usage are well below 80%. Please help!
