Take the 2-minute tour ×
Database Administrators Stack Exchange is a question and answer site for database professionals who wish to improve their database skills and learn from others in the community. It's 100% free, no registration required.

On my site I log every pageview (date, ip, referrer, page, etc) in a simple mysql table.

This table has the following activity:

  • 1 SELECT per minute
  • 1 DELETE per minute
  • no UPDATE queries
  • Lots of INSERTS (about 300 per second).

This table is never bigger than 200MB.

Today, I changed this table from an InnoDB table to a MEMORY table, this made sense to me to prevent unnecessary hard disk IO. I also prune this table once per minute, to make sure it never get's too big. And this information is not very important, so on restart of MySQL it does not matter that it is removed.

--

Performance wise, things are running fine. But I noticed that while running tuning-primer, that my Current Lock Wait ratio is quite high using MEMORY table.

Current Lock Wait ratio = 1 : 561

If I change this table to InnoDB the ratio is:

Current Lock Wait ratio = 0 : 78600946

My question: Should I worry about this Lock Wait Ratio? And is there something I can change in my configuration to improve things so that the lock wait ratio isn't so high using MEMORY tables?

share|improve this question
add comment

1 Answer

up vote 2 down vote accepted

You have to keep in mind two major aspects about using the MEMORY Storage Engine

ASPECT #1

The MEMORY table behaves like a MyISAM table in that it performs a full table lock when doing INSERTs, UPDATEs, and DELETEs (MySQL Documentation says Locking granularity : Table). I would be very concerned with a high lock ratio if it bottlenecks your I/O performance. If 300 INSERTs per second is coming at this table, you can be fully confident that there will be 300 full table locks per second.

ASPECT #2

No matter how much data you load into a MEMORY table, mysqld will always contact the .frm file of the MEMORY table to verify the table's existence before query parsing. Although the .frm is not a large file, it will incur a very tiny amount of disk I/O. Thousands, or even just hundreds, of queries against a MEMORY table can reveal a noticeable, even a significant, amount of disk I/O.

Since this is the case with the MEMORY storage engine, you must also avoid MyISAM because it has the same locking granularity (Full Table Locking).

Here are my recommedations

Recommedation #1

Cleanup InnoDB so that it has innodb_file_per_table enabled.

Recommedation #2

Use a large InnoDB Buffer Pool

Since the table is 200MB, you can make InnoDB Buffer Pool 5 times as large (restart required)

[mysqld]
innodb_buffer_pool_size=1G

Recommedation #3

Upgrade to MySQL 5.5 in order to have multithreaded reads and write that engages multiple CPUs for InnoDB. If you have InnoDB 5.1.38+ and are using the InnoDB Plugin, the same options that engages multiple CPUs for InnoDB already exists.

Recommedation #4

Common sense would tell me to recommend turning the table back to InnoDB

share|improve this answer
    
Hi RolandoMySQLDBA, thank you for your very detailed answer. I think you are right in saying that InnoDB is the better option here. I have since moved back. That link in your 3rd point was quite useful. I have made some changes to my config. I've changed innodb_thread_concurrency from 32 to 0. And set both innodb_read_io_threads and innodb_write_io_threads from 16 to 64. And lastly changed innodb_io_capacity from 500 to 1000. This is my overall config now: pastebin.com/MaU3h36x It's a machine with 4x sas drives in Raid 10 with 24gb ram. Total db size is about 10 gb. –  Mr.Boon Jan 18 '12 at 8:47
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.