1

In another question we attempted to update a table in our MySQL8 db. That didn't work out, but since then our database (even after both DB and machine restarts) seems painfully slow at the strangest times.

These things might be unrelated but, for example here is an entry from the slow query log:

--
# Query_time: 15.095238  Lock_time: 0.000127  Rows_sent: 0  Rows_examined: 0  Rows_affected: 0  Bytes_sent: 1594
SET timestamp=1685664330;
SELECT `seller_feedbacks`.* FROM `seller_feedbacks` WHERE (seller_feedbacks.tenant_id = 18251) AND `seller_feedbacks`.`order_id` = 229621225

If I do an explain on this query:

mysql> explain SELECT `seller_feedbacks`.* FROM `seller_feedbacks` WHERE (seller_feedbacks.tenant_id = 18251) AND `seller_feedbacks`.`order_id` = 229621225;
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+--------------------------------+
| id | select_type | table | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra                          |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+--------------------------------+
|  1 | SIMPLE      | NULL  | NULL       | NULL | NULL          | NULL | NULL    | NULL | NULL |     NULL | no matching row in const table |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+--------------------------------+
1 row in set, 1 warning (0.00 sec)

mysql> SELECT `seller_feedbacks`.* FROM `seller_feedbacks` WHERE (seller_feedbacks.tenant_id = 18251) AND `seller_feedbacks`.`order_id` = 229621225;
Empty set (0.00 sec)

The orders table is large: >200 million rows The seller_feedback table not so much: 800,000 rows or so.

This query, with different values, appears over 10,000 times in our slow query log in the last 3 days. And the size of the slow query log has tripled.

I know this kind of vague question is difficult and frowned upon, but I do not even know where to start investigating what has happened.

We are running MySQL 8.0.23-14 Percona Server (GPL), Release 14, Revision 3558242 on a Centos box

EDIT 1:

mysql> ANALYZE TABLE orders;
+-----------------------------+---------+----------+----------+
| Table                       | Op      | Msg_type | Msg_text |
+-----------------------------+---------+----------+----------+
| xxx_production.orders       | analyze | status   | OK       |
+-----------------------------+---------+----------+----------+
1 row in set (0.76 sec)

mysql> ANALYZE TABLE seller_feedbacks;
+---------------------------------------+---------+----------+----------+
| Table                                 | Op      | Msg_type | Msg_text |
+---------------------------------------+---------+----------+----------+
| xxx_production.seller_feedbacks       | analyze | status   | OK       |
+---------------------------------------+---------+----------+----------+
1 row in set (0.18 sec)

MySQL > CREATE TABLE `seller_feedbacks` (
  `id` int NOT NULL AUTO_INCREMENT,
  `tenant_id` int DEFAULT NULL,
  `order_id` int DEFAULT NULL,
  `customer_id` int DEFAULT NULL,
  `rating` int DEFAULT NULL,
  `comment` text,
  `response` text,
  `arrived_on_time` tinyint(1) DEFAULT NULL,
  `item_as_described` tinyint(1) DEFAULT NULL,
  `feedback_date` datetime DEFAULT NULL,
  `role` varchar(255) DEFAULT NULL,
  `customer_service` varchar(255) DEFAULT NULL,
  `created_at` datetime NOT NULL,
  `updated_at` datetime NOT NULL,
  `vendor_order_id` varchar(255) DEFAULT NULL,
  `customer_email` varchar(255) DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `index_seller_feedbacks_on_order_id` (`order_id`),
  KEY `index_seller_feedbacks_on_customer_id` (`customer_id`),
  KEY `index_seller_feedbacks_on_tenant_id_and_customer_id` (`tenant_id`,`customer_id`),
  KEY `index_seller_feedbacks_on_tenant_id_and_order_id` (`tenant_id`,`order_id`),
  KEY `index_seller_feedbacks_on_tenant_id_and_feedback_date` (`tenant_id`,`feedback_date`),
  CONSTRAINT `fk_rails_2e1355dc58` FOREIGN KEY (`tenant_id`) REFERENCES `tenants` (`id`),
  CONSTRAINT `fk_rails_3f223c66ad` FOREIGN KEY (`order_id`) REFERENCES `orders` (`id`),
  CONSTRAINT `fk_rails_c10b580c0c` FOREIGN KEY (`customer_id`) REFERENCES `customers` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=1481045 DEFAULT CHARSET=utf8 ROW_FORMAT=COMPRESSED |

This table is created by Rails migrations.

EDIT 2: This is the JSON version of the EXPLAIN.

{
  "query_block": {
    "select_id": 1,
    "cost_info": {
      "query_cost": "3.85"
    },
    "table": {
      "table_name": "seller_feedbacks",
      "access_type": "ref",
      "possible_keys": [
        "index_seller_feedbacks_on_tenant_id_and_customer_id",
        "index_seller_feedbacks_on_tenant_id_and_order_id",
        "index_seller_feedbacks_on_tenant_id_and_feedback_date"
      ],
      "key": "index_seller_feedbacks_on_tenant_id_and_customer_id",
      "used_key_parts": [
        "tenant_id"
      ],
      "key_length": "5",
      "ref": [
        "const"
      ],
      "rows_examined_per_scan": 11,
      "rows_produced_per_join": 11,
      "filtered": "100.00",
      "cost_info": {
        "read_cost": "2.75",
        "eval_cost": "1.10",
        "prefix_cost": "3.85",
        "data_read_per_join": "33K"
      },
      "used_columns": [
        "id",
        "tenant_id",
        "order_id",
        "customer_id",
        "rating",
        "comment",
        "response",
        "arrived_on_time",
        "item_as_described",
        "feedback_date",
        "role",
        "customer_service",
        "created_at",
        "updated_at",
        "vendor_order_id",
        "customer_email"
      ]
    }
  }
}
7
  • Please provide SHOW CREATE TABLE and do ANALYZE TABLE.
    – Rick James
    Jun 2 at 4:11
  • The plan is saying that there are no possible indexes to use, is that what you would expect? That would be unusual, but it’s certainly possible that they were missed when doing the maintenance in your linked question (it looks like you effectively replaced the table with another one, potentially a syncing of indexes was never done) Jun 2 at 6:34
  • @AndrewSayer the pt-osc never completed so I don't think the table swap happened. But that is an interesting point!
    – phil
    Jun 2 at 6:39
  • @RickJames added to the question
    – phil
    Jun 2 at 6:39
  • Hmmm... Does any row match the SELECT? It seems not. Please provide EXPLAIN FORMAT=JSON SELECT .... It may give a clue of what it was thinking about. (Yeah, I know this seems to be irrelevant to pt-osc.)
    – Rick James
    Jun 2 at 20:49

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.