Mysql Memory table getting many locks

Question

On my site I log every pageview (date, ip, referrer, page, etc) in a simple mysql table.

This table has the following activity:

• 1 SELECT per minute
• 1 DELETE per minute
• no UPDATE queries
• Lots of INSERTS (about 300 per second).

This table is never bigger than 200MB.

Today, I changed this table from an InnoDB table to a MEMORY table, this made sense to me to prevent unnecessary hard disk IO. I also prune this table once per minute, to make sure it never get's too big. And this information is not very important, so on restart of MySQL it does not matter that it is removed.

--

Performance wise, things are running fine. But I noticed that while running tuning-primer, that my Current Lock Wait ratio is quite high using MEMORY table.

Current Lock Wait ratio = 1 : 561

If I change this table to InnoDB the ratio is:

Current Lock Wait ratio = 0 : 78600946

My question: Should I worry about this Lock Wait Ratio? And is there something I can change in my configuration to improve things so that the lock wait ratio isn't so high using MEMORY tables?

Answer 1

You have to keep in mind two major aspects about using the MEMORY Storage Engine

ASPECT #1

The MEMORY table behaves like a MyISAM table in that it performs a full table lock when doing INSERTs, UPDATEs, and DELETEs (MySQL Documentation says Locking granularity : Table). I would be very concerned with a high lock ratio if it bottlenecks your I/O performance. If 300 INSERTs per second is coming at this table, you can be fully confident that there will be 300 full table locks per second.

ASPECT #2

No matter how much data you load into a MEMORY table, mysqld will always contact the .frm file of the MEMORY table to verify the table's existence before query parsing. Although the .frm is not a large file, it will incur a very tiny amount of disk I/O. Thousands, or even just hundreds, of queries against a MEMORY table can reveal a noticeable, even a significant, amount of disk I/O.

Since this is the case with the MEMORY storage engine, you must also avoid MyISAM because it has the same locking granularity (Full Table Locking).

Here are my recommedations

Recommedation #1

Cleanup InnoDB so that it has innodb_file_per_table enabled.

Recommedation #2

Use a large InnoDB Buffer Pool

Since the table is 200MB, you can make InnoDB Buffer Pool 5 times as large (restart required)

[mysqld]
innodb_buffer_pool_size=1G

Recommedation #3

Upgrade to MySQL 5.5 in order to have multithreaded reads and write that engages multiple CPUs for InnoDB. If you have InnoDB 5.1.38+ and are using the InnoDB Plugin, the same options that engages multiple CPUs for InnoDB already exists.

Recommedation #4

Common sense would tell me to recommend turning the table back to InnoDB