I'm hosting a few sites on linode and have encountered a strange problem. For the last two years the site has been running perfectly fine, and now randomly all the websites on the server go to the "site offline check the settings.php" page. No changes have been made to the website at all recently.
when i try go do mysqld stop it says busy for a while then finally says failed, while doing start results in an instant fail.
Many of the threads I googled suggest the hostname is off in the settings.php but since the website has been up for two years I don't think this can be the case.
I haven't had to troubleshoot mySQL before, but if i remember correctly the log file I should be looking at is Hostname.err. In there, it has a large chain of errors concerning InnoDB. The one listed as fatal error says:
InnoDB: Fatal error: cannot allocate memory for the buffer pool
I would appreciate any suggestions, and if there are log files that would help let me know.
Edit: Requested information. 1) When I examine the CNF, it appears that all lines involving innoDB are commented with #. This means that it has always been this way as I have not modified it.
2)mysql Ver 14.14 Distrib 5.5.14, for Linux (i686) using readline 5.1
3) It looks like there is a giant jump in the IO rate every time I try to start the database / before the websites die. How do i go about clearing those cache tables if they have become too unmanageable and is there any risk of losing anything?
Third Edit (Now with datadir):
How much RAM is on the VM ? 512MB
How much space does datadir have ?
Size: 20G Used: 19G Available 0 Use Percent: 100%
What is the size of ibdata1 ( ls -lh /var/lib/mysql/ibdata1)? 114M
FYI: If anyone looks here with the same problem in the future, My issue was the the binary log files had consumed my entire disk space. See here http://systembash.com/content/mysql-binary-log-file-size-huge/