I am trying to optimize a query that using IN clause in WHERE to avoid file sorting. To make it easy , I created the following sample which shows the problem. Here is my query:

FROM `test` 
WHERE user_id = 9898 
AND status IN (1,3,4) 
order by id 
limit 30;

Here is the result of explain, as you can see the query is filesort

id  select_type     table   type    possible_keys   key     key_len     ref     rows    Extra
1   SIMPLE  test    range   user_id     user_id     8   NULL    3   Using where; Using index; Using filesort

Here is my table structure

  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `user_id` int(10) unsigned NOT NULL,
  `status` int(3) unsigned NOT NULL,
  PRIMARY KEY (`id`),
  KEY `user_id` (`user_id`,`status`)

-- Dumping data for table `test`

INSERT INTO `test` (`id`, `user_id`, `status`) VALUES
(5, 9797, 2),
(6, 9797, 3),
(4, 9898, 0),
(1, 9898, 2),
(2, 9898, 3),
(3, 9898, 4);

How can I optimize the query? In my real table I can see the following information in error log: # Query_time: 26.498180 Lock_time: 0.000175 Rows_sent: 100 Rows_examined: 4926

  • Is order by id necessary, or might be avoided?
    – BlitZ
    Commented Oct 1, 2013 at 16:02
  • Yes, I have to use it.
    – usef_ksa
    Commented Oct 1, 2013 at 16:06
  • Have you tried adding id to user_id. I.e. KEY user_id (user_id,status, id) ? Commented Aug 21, 2016 at 9:22

2 Answers 2


I was looking at a very similar problem today. After doing a ton of searching online, I found this great article by Percona.

Using "UNION ALL", you can join together a list of queries, like so:

(SELECT * FROM `test` WHERE user_id = 9898 AND status = 1 ORDER BY id LIMIT 30) 
(SELECT * FROM `test` WHERE user_id = 9898 AND status = 3 ORDER BY id LIMIT 30)
(SELECT * FROM `test` WHERE user_id = 9898 AND status = 4 ORDER BY id LIMIT 30)

It looks gnarly compared to select ... status IN (1,3,4), but it's effective in avoiding the filesort.

As long as the inner "SELECT..." statements are efficient, the performance is good.

  • i believe the Oracle database optimizer does OR into UNION ALL rewrite trick automatic, Time the Oracle devs also implement this optimizer rewritting into the MySQL optimizer also.. But the MySQL optimizer would still need to use a filesort and most likely also a temporary table on the outer SQL with this query.. The combination filesort and a temporary table can be even more worse then the filesort alone with a high number of records but with a max UNION RESULT limit of 90 records you are most likely okay here. Commented Mar 27, 2019 at 14:26

Put the list of values into a temporary table, and then perform an INNER JOIN to it. Most SQL optimizers and engines can handle joins much better than they can handle an IN operation. If the list is long, this also allows you to define an index on the temporary table to further assist the optimizer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.