There is a software hangfire
that uses mysql/mariadb as the backend. When the hangfire database was dropped, some InnoDB tables were left undropped and locked in some unusual way.
- hangfire_hash
- hangfire_job
- hangfire_jobqueue
- hangfire_state
Any DDL/DML like DROP TABLE
, TRUNCATE
, ALTER TABLE
, DELETE
or UPDATE
causes an error:
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction
On the other hand, SELECT * FROM hangfire_job;
or SELECT count(*) FROM hangfire_job;
works as expected.
I had found an article with exactly the same issue and, most surprisingly, locked and undroppable tables are also the same:
https://stackoverflow.com/q/78422902/1200574
What could be causing this strange behavior and is there a way to deal with these immortal tables?
UPD.1
MariaDB [sys]> SELECT * FROM sys.innodb_lock_waits \G
*************************** 1. row ***************************
wait_started: 2024-12-18 20:33:03
wait_age: 00:00:23
wait_age_secs: 23
locked_table: `emails`.`hangfire_jobqueue`
locked_index: NULL
locked_type: TABLE
waiting_trx_id: 7691700269
waiting_trx_started: 2024-12-18 20:33:03
waiting_trx_age: 00:00:23
waiting_trx_rows_locked: 0
waiting_trx_rows_modified: 0
waiting_pid: 171774
waiting_query: drop table emails.hangfire_jobqueue
waiting_lock_id: 7691700269:7892
waiting_lock_mode: X
blocking_trx_id: 7609295307
blocking_pid: 0
blocking_query: NULL
blocking_lock_id: 7609295307:7892
blocking_lock_mode: IX
blocking_trx_started: 2024-12-17 17:33:26
blocking_trx_age: 27:00:00
blocking_trx_rows_locked: 2
blocking_trx_rows_modified: 6
sql_kill_blocking_query: KILL QUERY 0
sql_kill_blocking_connection: KILL 0
1 row in set (0.001 sec)
MariaDB [sys]> KILL QUERY 0;
ERROR 1094 (HY000): Unknown thread id: 0
UPD.2 After reboot locking is still present:
MariaDB [(none)]> select * from information_schema.INNODB_TRX \G
*************************** 1. row ***************************
trx_id: 7698617537
trx_state: LOCK WAIT
trx_started: 2024-12-19 12:04:31
trx_requested_lock_id: 7698617537:7931
trx_wait_started: 2024-12-19 12:04:31
trx_weight: 1
trx_mysql_thread_id: 39
trx_query: drop database emails
trx_operation_state:
trx_tables_in_use: 0
trx_tables_locked: 1
trx_lock_structs: 1
trx_lock_memory_bytes: 1128
trx_rows_locked: 0
trx_rows_modified: 0
trx_concurrency_tickets: 0
trx_isolation_level: REPEATABLE READ
trx_unique_checks: 1
trx_foreign_key_checks: 1
trx_last_foreign_key_error: NULL
trx_is_read_only: 0
trx_autocommit_non_locking: 0
*************************** 2. row ***************************
trx_id: 7609295307
trx_state: RUNNING
trx_started: 2024-12-19 12:01:16
trx_requested_lock_id: NULL
trx_wait_started: NULL
trx_weight: 10
trx_mysql_thread_id: 0
trx_query: NULL
trx_operation_state:
trx_tables_in_use: 0
trx_tables_locked: 4
trx_lock_structs: 4
trx_lock_memory_bytes: 1128
trx_rows_locked: 0
trx_rows_modified: 6
trx_concurrency_tickets: 0
trx_isolation_level: REPEATABLE READ
trx_unique_checks: 1
trx_foreign_key_checks: 1
trx_last_foreign_key_error: NULL
trx_is_read_only: 0
trx_autocommit_non_locking: 0
2 rows in set (0.000 sec)
sys.innodb_lock_waits
but it refers to nonexistent thread.