it happended the second time, could be the same reason cause it.

mysql tried to restart itself 3 or more times, but it failed finally.

here is mysql log

130706 09:43:27 mysqld_safe Number of processes running now: 0
130706 09:43:27 mysqld_safe mysqld restarted
130706  9:43:28  InnoDB: Initializing buffer pool, size = 8.0M
130706  9:43:28  InnoDB: Completed initialization of buffer pool
130706  9:43:29  InnoDB: Started; log sequence number 0 2552936
130706  9:43:29 [Note] Event Scheduler: Loaded 0 events
130706  9:43:29 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.69-log'  socket: '/var/www/data/mysql.sock'  port: 3306  Source distribution
130706 09:43:34 mysqld_safe Number of processes running now: 0
130706 09:43:34 mysqld_safe mysqld restarted130706  9:43:36  InnoDB: Initializing buffer pool, size = 8.0M
130706  9:43:36  InnoDB: Completed initialization of buffer pool
130706  9:43:36  InnoDB: Started; log sequence number 0 2552936
130706  9:43:36 [Note] Event Scheduler: Loaded 0 events
130706  9:43:36 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.69-log'  socket: '/var/www/data/mysql.sock'  port: 3306  Source distribution
130706 09:43:38 mysqld_safe Number of processes running now: 0
130706 09:43:39 mysqld_safe mysqld restarted
130706  9:43:41  InnoDB: Initializing buffer pool, size = 8.0M
130706  9:43:41  InnoDB: Completed initialization of buffer pool
130706  9:43:41  InnoDB: Started; log sequence number 0 2552936
130706 09:43:42 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended

and following is content of my.cnf

there are 1.5G mem on this server



but i can easily start it using "service mysqld start"

anyone knows what was happened? please help!

  • seems strange that your log, socket file and data are in /var/www/ .. is there a reason for that? Also, the slow-query-log and slow-query-log-file are on the same line.. is that a copy/paste error?
    – msturdy
    Jul 6, 2013 at 3:52
  • yes, it's a copy/paste error. I've modified it.
    – worldask
    Jul 6, 2013 at 3:56
  • ok, cool... but why are those files in the /var/www/ directory? Conventionally that's for files to be served by the web server
    – msturdy
    Jul 6, 2013 at 3:57
  • because there are two hard disks, one for centos, the other one for all of data. unfortunately, it was mounted to /var/www, I cannot change it now.
    – worldask
    Jul 6, 2013 at 3:59
  • what about using mysqld --print-defaults (from: dev.mysql.com/doc/refman/5.0/en/debugging-server.html) and comparing that with your conf file?
    – msturdy
    Jul 6, 2013 at 4:01

1 Answer 1


Check the syslogs; on CentOS, I think the file you want is /var/log/messages.

The symptoms in your MySQL error log file suggest that something is doing the equivalent of a kill -9 on the MySQL process. It's not impossible but it's extremely unlikely for MySQL to just fail to start and leave not a trace in the log files.

130706 09:43:34 mysqld_safe Number of processes running now: 0

That message is not from mysqld but rather from mysqld_safe, which is a wrapper for mysqld and is responsible for trying to restart mysqld if it stops without removing its pidfile (which it would do in the event of a clean shutdown).

I expect you'll find your system is crunched for memory and there will be references to oom-killer in the syslog.

# egrep -i "oom|kill|mysql" /var/log/messages

See if that gets you anything. If not, manually review the syslogs for the time around which the crashes occurred.

This is not the precise scenario that I identified in this answer or this one ... but the end result similar enough that it bears investigating. If it is the same ultimate cause, the delayed onset may be related to the fact that your InnoDB Buffer Pool is tiny compared to those mentioned in the other questions, so they ran into the problem at a different point during startup.

update: diagnosis confirmed, so the next question is how to avoid the problem.

Looking at the fact that you have 1.5 GB of RAM (I assume you mean "total") and then looking at your my.cnf, there are definitely problems, aside from the fact that it's generally not a good idea to run your web server and your MySQL server on the same machine for the simple reason that neither MySQL nor Apache is designed to be tightly constrained to a fixed allocation of memory.

But a couple of aspects of your configuration seem particularly unreasonable:


These two are fixed allocations of memory that you probably can't afford... right here, at startup, MySQL is allocating 128MB for the query cache and 512MB for the MySQL key cache (key_buffer). Reducing these dramatically will of course reduce your performance, but you don't have the memory to spare for these two things. Set them to small values, such as 8M.

Of course, I don't think 8M would ordinarily be a reasonable value on a dedicated server, but again, these are fixed allocations at startup, and we're dealing with system that is very constrained for memory.

Those two are the big ones, and shrinking them down to size will probably get you going again depending on how severely constrained for memory you are.

There are a few others that I would say are less important but still seem out-of-range given the size of the server and the fact that it isn't dedicated to MySQL only.


Surely not. Reducing this isn't going to save much if anything at startup because the allocations don't happen until those connections occur... but if you really have a need for this, then you're really, really dealing with a server that's far too small.


The default for max_allowed_packet is 1M. Unless you changed this for a reason, it's higher than necessary. Reducing this won't likely save anything unless you have a workload that requires this, but this amount of memory could theoretically be allocated by every client thread at the same time which of course wouldn't fly very far.


I'm not finding thread_cache in the documentation, so what is this? thread_cache_size I assume, so this is also too large and should probably be lowered to a very small number (such as "8"). No real harm done except that the threads used by recently-disconnected clients linger in memory after they're not needed, taking up space that you can't really spare.

See also: How MySQL Uses Memory

  • thnks very much, i found that system killed mysqld and httpd both for OOM. But what can I do now? Is it any problem in my.cnf?
    – worldask
    Jul 6, 2013 at 8:32
  • i don't have innoDB, all of tables are using MyISAM
    – worldask
    Jul 6, 2013 at 8:50
  • my.cnf did have problem, but I've found that it didn't directly cause the disaster after watching several days. The spiders crawled constantly made httpd cannot endure the pressure, so system killed httpd and mysqld. But httpd can restart itself succefully everytime, why mysqld restart failed sometime?
    – worldask
    Jul 10, 2013 at 7:04
  • and I've reduce those higher options in my.cnf, it still cannot restart itself
    – worldask
    Jul 10, 2013 at 7:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.