1

We face a issue that the InnoDB table is locked, when a sql file that is executed is large than a certain file size.

The problem does not appear when the file size is much lower, for example 25MB. But when the file increase to something like 45+ MB the InnoDB table locks, when the sql is on 70% of running.

Is this something that we can prevent by changing our my.cnf or something else?

Our current my.cnf:

[mysqld]
sql_mode=""

bind-address = 127.0.0.1
local-infile=1
innodb_file_per_table=1
innodb_file_format=barracuda

skip-name-resolve
performance_schema=ON

slow_query_log = 1
slow_query_log_file=/var/log/mysql-log-slow-queries.log

# key_buffer_size = 390M
max_allowed_packet = 200M
# sort_buffer_size = 16M
# read_buffer_size = 16M
read_rnd_buffer_size = 2M
# myisam_sort_buffer_size = 64M
tmp_table_size = 300M
query_cache_type = 1
query_cache_size = 650M
query_cache_limit = 2M
query_cache_min_res_unit=512
thread_cache_size = 100
max_connections = 225
wait_timeout = 900
connect_timeout = 120
innodb_buffer_pool_size = 8G
max_heap_table_size =290M
innodb_log_buffer_size = 36M
# join_buffer_size = 32M
innodb_buffer_pool_instances = 8

long_query_time = 5
table_definition_cache = 4K
open_files_limit = 60K
table_open_cache = 50767
innodb_log_file_size= 256M
innodb_lock_wait_timeout = 1000
innodb_lru_scan_depth = 100
innodb_flush_neighbors = 0
innodb_io_capacity_max = 10000
innodb_io_capacity = 5000

innodb_purge_threads = 4
innodb_flushing_avg_loops = 10

Small part of the sql file:

SET NAMES 'utf8';
SET CHARACTER SET utf8;
/************** SYSTEM ******************/
UPDATE `catalog_product_entity` SET  `updated_at` = '2018-09-24 08:45:15',  `updated_by` = '23'   WHERE `entity_id` = '158700';
UPDATE `catalog_product_entity` SET  `updated_at` = '2018-09-24 08:45:15',  `updated_by` = '23'   WHERE `entity_id` = '158701';
3
  • recommend setting log-slow-verbosity=query_plan,explain. The default long_query_time is 10 seconds, I think that should be dropped. The contents of the sql file aren't described at all and are largely the case of this problem. Does the sql file explicitly lock the table? Does the slow query log show anything? What do you mean by locked, slow/readonly/no reads? what error? Have you increased the innodb_log_file_size?
    – danblack
    Commented Sep 22, 2018 at 23:55
  • @danblack Thanks for your reply. Sorry but the contents of the sql file are to large to display here. No the sql file does not explicitly lock the table, that's due to the large file and data. Slow query log is empty, also after we lowered long_query_time to 5. The table got no reads, so it will not be possible to read any data from this table at the same time. innodb_log_file_size is increased to 400M already. Really don't know what I am missing right now.
    – JGeer
    Commented Sep 24, 2018 at 7:44
  • What is the general format of the sql file, what forms of SQL. Are there large amounts of FK constraints or unique indexes? Are session variables assocated with FK/unque keys set? Are index created at the beginning or end of dump? Was it a mysqldump that generated this file? If so how and with what arguments? Does disabling query cache help?
    – danblack
    Commented Sep 24, 2018 at 8:52

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.