MySQL INSERTs waiting for UPDATEs (MyISAM)?

Question

In show processlist;, MySQL 5.5, we see that INSERTs tend to wait for UPDATEs to complete before they can proceed. This is on MyISAM tables.

We tried INSERT HIGH_PRIORITY and UPDATE LOW_PRIORITY. However, this did not change the order. Whenever there is an UPDATE in the queue, INSERTs wait for them to finish.

Is this by design? I could not find any write-ups detailing how this ordering works, any references we can read on this?

UPDATE:
I'm posting how a typical show processlist; looks. It might clarify the question.
SELECTs are working just fine. LOW_PRIORITY on the UPDATE pushes it to the back of the list. But INSERTs are waiting as well. This is database has concurrent_insert set to ALWAYS. Therefore I expect SELECTS and INSERTS to work at the same time and UPDATE LOW_PRIORITYs to take affect only when there are no more INSERTs or SELECTs waiting around. When there are no UPDATEs, concurrent_insert kicks in just fine.

Answer 1

What is the lock level for a MyISAM table ? Table Level Locking

This is by design : (See http://dev.mysql.com/doc/refman/5.1/en/myisam-storage-engine.html)

I wrote about before : Benefits of table level locking

You may want to take a shot at INSERT DELAYED. Note what page 410 bulletpoint 5 of "MySQL 5.0 Certification Study Guide" says on INSERT DELAYED:

DELAYED may be used with INSERT (and REPLACE). The server buffers the rows in memory and inserts them when the table is not being used. Delayed inserts increase efficiency because they are done in batches rather than individually. While inserting the rows, the server checks periodically to see whether other requests to use the table have arrived. If so, the server suspends insertion of delayed rows until the table becomes free again. Using DELAYED allows the client to proceed immediately after issuing the INSERT statementrather than waiting until it completes.

There are risks to using INSERT DELAYED. Page 411 says

If you use DELAYED, keep the following points in mind:

• Delayed rows tends to held for a longer time on a very busy server than on a lightly loaded one.
• If a crash occurs while the server is buffering delayed rows in memory, those rows are lost.

Looking over the whole section of the book, HIGH_PRIORITY is only applied to SELECT queries.

For more information, "MySQL 5.0 Certification Study Guide" pages 408-412 Section 29.2.

CAVEAT : When it comes to MySQL Replication, INSERT DELAYED executed on a Master is treated as INSERT on a Slave. If a master crashes, missing rows are logged in the binary logs of the Master. It would be very possible that the missing rows on the Master replicated successfully to the Slave.

UPDATE 2012-08-20 11:18 EDT

Please note what page 410 bulletpoint 3 of "MySQL 5.0 Certification Study Guide" says on LOW_PRIORITY:

The LOW_PRIORITY modifier may be applied to statements that update tables (INSERT, DELETE, REPLACE, or UPDATE). A low-priority write request waits not only until all current readers have finished, but for any pending read requests that arrive while the write request itself is waiting. That is, it waits until there are no pending read requests at all. It is therefore possible for a low-priority write request never to be performed, if read requests keep arriving while the write request is waiting.

Given this description, low-priority UPDATEs definitely take a backseat to INSERTs that have normal priority. Notwithstanding, when a SELECT occurs on a MyISAM table, all write requests take a backseat regardless of priority. If the read traffic is high enough, the precedence of INSERT over UPDATE LOW_PRIOIRTY would be negligible, nominal at best.

UPDATE 2012-08-20 11:36 EDT

If the read traffic is hindering your INSERT over UPDATE LOW_PRIOIRTY situation, I would have to recommend switching your MyISAM table to InnoDB. In fact, InnoDB can be tuned for performing heavy INSERTs. Please see my posts about tuning InnoDB for this purpose:

• Feb 12, 2011 : How do you tune MySQL for a heavy InnoDB workload?
• Jan 18, 2012 : Insert-heavy InnoDB table won't use all my CPU
• Jul 04, 2012 : Capabilities of InnoDB INSERT Performance
• Aug 17, 2012 : When to switch from MyISAM to InnoDB?