I'm hosting a few sites on linode and have encountered a strange problem. For the last two years the site has been running perfectly fine, and now randomly all the websites on the server go to the "site offline check the settings.php" page. No changes have been made to the website at all recently.

when i try go do mysqld stop it says busy for a while then finally says failed, while doing start results in an instant fail.

Many of the threads I googled suggest the hostname is off in the settings.php but since the website has been up for two years I don't think this can be the case.

I haven't had to troubleshoot mySQL before, but if i remember correctly the log file I should be looking at is Hostname.err. In there, it has a large chain of errors concerning InnoDB. The one listed as fatal error says:

InnoDB: Fatal error: cannot allocate memory for the buffer pool

I would appreciate any suggestions, and if there are log files that would help let me know.

Edit: Requested information. 1) When I examine the CNF, it appears that all lines involving innoDB are commented with #. This means that it has always been this way as I have not modified it.

2)mysql Ver 14.14 Distrib 5.5.14, for Linux (i686) using readline 5.1

3) It looks like there is a giant jump in the IO rate every time I try to start the database / before the websites die. How do i go about clearing those cache tables if they have become too unmanageable and is there any risk of losing anything?

Third Edit (Now with datadir):

How much RAM is on the VM ? 512MB

How much space does datadir have ?

Size: 20G Used: 19G Available 0 Use Percent: 100%

What is the size of ibdata1 ( ls -lh /var/lib/mysql/ibdata1)? 114M

FYI: If anyone looks here with the same problem in the future, My issue was the the binary log files had consumed my entire disk space. See here http://systembash.com/content/mysql-binary-log-file-size-huge/

  • Since its running from 2 years, probably all cache tables have grown in size and there might be a heavy load on mysql. Check your Disk read/writes for mysql. When it reached 100% usually mysql will go down after few minutes
    – GoodSp33d
    Commented Apr 5, 2013 at 15:35
  • 2
    Please post the following: 1) All the InnoDB settings in my.cnf, 2) What version of MySQL are you running (Run mysql --version in Linux), 3) All hardware limits in the VM. Commented Apr 5, 2013 at 15:38
  • How much RAM is on the VM ? How much space does datadir have ? What is the size of ibdata1 ( ls -lh /var/lib/mysql/ibdata1)? Commented Apr 5, 2013 at 17:03
  • 1
    Please run df -h /var/lib/mysql Commented Apr 5, 2013 at 17:44

1 Answer 1


OMG !!! Look at the output of df -h

Size: 20G Used: 19G Available 0 Use Percent: 100%


Get more diskspace freed up or have linode increase diskspace.

According to MySQL 5.0 Certification Study Guide

enter image description here Page 408,409 Section 29.2 Bulletpoint 11 says:

If you run out of disk space while adding rows to a MyISAM table, no error occurs. The server suspends the operation until space becomes available, and then completes the operation.

In reality, mysqld is not dead. It is waiting for free diskspace to continue whatever it normally does. Even during startup, if mysqld makes a temp table, it is a MyISAM table. No space left on the disk? No problem. MyISAM just patiently waits for you to make free space.

If you have an error file like /var/log/mysql/log, clear its contents like this:

echo -n > /var/log/mysqld.log

If the error file in /var/lib/mysql/my-machine.err then do this:

echo -n > /var/lib/mysql/my-machine.err

Give it a Try !!!

  • Thank you, I've got the websites back online by allocating a few extra MB to the disk image that I had in case of a problem like this. Now that its back up I'll see if I can delete a few things to free up disk space. Commented Apr 5, 2013 at 23:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.