3

I have an older system built on drupal, and there is a particular (generated) query that takes a long time - never completing in some cases.

The query:

SELECT
  node.nid,
  node.created,
  node.changed,
  node.title
FROM node node
  LEFT JOIN (SELECT td.*,
                    tn.nid AS nid
             FROM taxonomy_term_data td
             LEFT JOIN taxonomy_vocabulary tv ON td.vid = tv.vid
             LEFT JOIN taxonomy_index tn ON tn.tid = td.tid
             WHERE (tv.machine_name IN ('publications'))
             ) taxonomy_term_data_node ON node.nid = taxonomy_term_data_node.nid
WHERE node.type in ('story')

Explain:

+----+-------------+-------+------------+--------+---------------+-----------+---------+----------------------+--------+----------+-------------+
| id | select_type | table | partitions | type   | possible_keys | key       | key_len | ref                  | rows   | filtered | Extra       |
+----+-------------+-------+------------+--------+---------------+-----------+---------+----------------------+--------+----------+-------------+
|  1 | SIMPLE      | node  | NULL       | ref    | node_type     | node_type | 14      | const                | 427872 |   100.00 | Using where |
|  1 | SIMPLE      | td    | NULL       | index  | NULL          | vid_name  | 771     | NULL                 |    191 |   100.00 | Using index |
|  1 | SIMPLE      | tv    | NULL       | eq_ref | PRIMARY       | PRIMARY   | 4       | dbname.td.vid        |      1 |   100.00 | Using where |
|  1 | SIMPLE      | tn    | NULL       | ref    | term_node     | term_node | 4       | dbname.td.tid        |   5873 |   100.00 | Using where |
+----+-------------+-------+------------+--------+---------------+-----------+---------+----------------------+--------+----------+-------------+

Indexes:

MySQL [dbname]> show indexes from node;
+-------+------------+------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name         | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-------+------------+------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| node  |          0 | PRIMARY          |            1 | nid         | A         |      956206 |     NULL | NULL   |      | BTREE      |         |               |
| node  |          0 | vid              |            1 | vid         | A         |      956206 |     NULL | NULL   | YES  | BTREE      |         |               |
| node  |          1 | node_changed     |            1 | changed     | A         |      749265 |     NULL | NULL   |      | BTREE      |         |               |
| node  |          1 | node_created     |            1 | created     | A         |      759215 |     NULL | NULL   |      | BTREE      |         |               |
| node  |          1 | node_frontpage   |            1 | promote     | A         |           1 |     NULL | NULL   |      | BTREE      |         |               |
| node  |          1 | node_frontpage   |            2 | status      | A         |           1 |     NULL | NULL   |      | BTREE      |         |               |
| node  |          1 | node_frontpage   |            3 | sticky      | A         |           1 |     NULL | NULL   |      | BTREE      |         |               |
| node  |          1 | node_frontpage   |            4 | created     | A         |      714565 |     NULL | NULL   |      | BTREE      |         |               |
| node  |          1 | node_status_type |            1 | status      | A         |           1 |     NULL | NULL   |      | BTREE      |         |               |
| node  |          1 | node_status_type |            2 | type        | A         |           6 |     NULL | NULL   |      | BTREE      |         |               |
| node  |          1 | node_status_type |            3 | nid         | A         |      956206 |     NULL | NULL   |      | BTREE      |         |               |
| node  |          1 | node_title_type  |            1 | title       | A         |      503053 |     NULL | NULL   |      | BTREE      |         |               |
| node  |          1 | node_title_type  |            2 | type        | A         |      793978 |        4 | NULL   |      | BTREE      |         |               |
| node  |          1 | node_type        |            1 | type        | A         |           7 |        4 | NULL   |      | BTREE      |         |               |
| node  |          1 | uid              |            1 | uid         | A         |         183 |     NULL | NULL   |      | BTREE      |         |               |
| node  |          1 | tnid             |            1 | tnid        | A         |           1 |     NULL | NULL   |      | BTREE      |         |               |
| node  |          1 | translate        |            1 | translate   | A         |           1 |     NULL | NULL   |      | BTREE      |         |               |
| node  |          1 | language         |            1 | language    | A         |           1 |     NULL | NULL   |      | BTREE      |         |               |
| node  |          1 | uuid             |            1 | uuid        | A         |      956206 |     NULL | NULL   |      | BTREE      |         |               |
+-------+------------+------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
MySQL [dbname]> show indexes from taxonomy_term_data;
+--------------------+------------+---------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table              | Non_unique | Key_name      | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+--------------------+------------+---------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| taxonomy_term_data |          0 | PRIMARY       |            1 | tid         | A         |         191 |     NULL | NULL   |      | BTREE      |         |               |
| taxonomy_term_data |          1 | taxonomy_tree |            1 | vid         | A         |           7 |     NULL | NULL   |      | BTREE      |         |               |
| taxonomy_term_data |          1 | taxonomy_tree |            2 | weight      | A         |         148 |     NULL | NULL   |      | BTREE      |         |               |
| taxonomy_term_data |          1 | taxonomy_tree |            3 | name        | A         |         191 |     NULL | NULL   |      | BTREE      |         |               |
| taxonomy_term_data |          1 | vid_name      |            1 | vid         | A         |           7 |     NULL | NULL   |      | BTREE      |         |               |
| taxonomy_term_data |          1 | vid_name      |            2 | name        | A         |         170 |     NULL | NULL   |      | BTREE      |         |               |
| taxonomy_term_data |          1 | name          |            1 | name        | A         |         154 |     NULL | NULL   |      | BTREE      |         |               |
| taxonomy_term_data |          1 | uuid          |            1 | uuid        | A         |         191 |     NULL | NULL   |      | BTREE      |         |               |
+--------------------+------------+---------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+


MySQL [dbname]> show indexes from taxonomy_vocabulary;
+---------------------+------------+--------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table               | Non_unique | Key_name     | Seq_in_index | Column_name  | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+---------------------+------------+--------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| taxonomy_vocabulary |          0 | PRIMARY      |            1 | vid          | A         |           8 |     NULL | NULL   |      | BTREE      |         |               |
| taxonomy_vocabulary |          0 | machine_name |            1 | machine_name | A         |           8 |     NULL | NULL   |      | BTREE      |         |               |
| taxonomy_vocabulary |          1 | list         |            1 | weight       | A         |           1 |     NULL | NULL   |      | BTREE      |         |               |
| taxonomy_vocabulary |          1 | list         |            2 | name         | A         |           8 |     NULL | NULL   |      | BTREE      |         |               |
+---------------------+------------+--------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+

MySQL [dbname]> show indexes from taxonomy_index;
+----------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table          | Non_unique | Key_name  | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| taxonomy_index |          1 | term_node |            1 | tid         | A         |         107 |     NULL | NULL   |      | BTREE      |         |               |
| taxonomy_index |          1 | term_node |            2 | sticky      | A         |         107 |     NULL | NULL   | YES  | BTREE      |         |               |
| taxonomy_index |          1 | term_node |            3 | created     | A         |      484992 |     NULL | NULL   |      | BTREE      |         |               |
| taxonomy_index |          1 | nid       |            1 | nid         | A         |      175997 |     NULL | NULL   |      | BTREE      |         |               |
+----------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+

We do not have the ability to change the application, so I'd like to see if this can be improved by adding indexes and/or partitions, but I'm not sure where to start. I think the problem is with the join to the subquery (which alone returns 190k rows - the node table has a bit over 1M records).

Is there any way to fix this?

Edit - added additional information. The aurora cluster has 3 nodes (1 writer, 2 readers, all of which are r5.2xlarge)

MySQL [dbname]> SELECT type,COUNT(1) FROM node GROUP BY type WITH ROLLUP;
+-------------------+----------+
| type              | COUNT(1) |
+-------------------+----------+
| feed              |        1 |
| page              |        1 |
| print_child_story |   446852 |
| run_list          |    51097 |
| story             |   269002 |
| web_child_story   |   197906 |
| wire_story        |    60183 |
| zapier_endpoint   |       20 |
| NULL              |  1025062 |
+-------------------+----------+
9 rows in set (1.21 sec)

MySQL [dbname]> SELECT version() mysql_version,@@global.aurora_version aurora_version,@@global.innodb_buffer_pool_size;
+---------------+----------------+----------------------------------+
| mysql_version | aurora_version | @@global.innodb_buffer_pool_size |
+---------------+----------------+----------------------------------+
| 5.7.12-log    | 2.10.2         |                       3210739712 |
+---------------+----------------+----------------------------------+
1 row in set (0.00 sec)
8
  • You should add your table and current index definitions to the post btw.
    – J.D.
    Jul 21, 2022 at 13:04
  • 1
    Are you sure you need outer joins throughout? At least one of them, LEFT JOIN taxonomy_vocabulary tv, becomes an inner join because WHERE (tv.machine_name IN ('publications'))
    – mustaccio
    Jul 21, 2022 at 13:05
  • 1
    Please show output for SELECT type,COUNT(1) FROM node GROUP BY type WITH ROLLUP; Jul 21, 2022 at 16:19
  • 1
    Please run SELECT version() mysql_version,@@global.aurora_version aurora_version,@@global.innodb_buffer_pool_size; Jul 21, 2022 at 16:31
  • 1
    @RolandoMySQLDBA - I've added the results of those queries to the question.
    – chris
    Jul 22, 2022 at 15:04

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.