Take the 2-minute tour ×
Database Administrators Stack Exchange is a question and answer site for database professionals who wish to improve their database skills and learn from others in the community. It's 100% free, no registration required.

Ever since we've moved to Amazon RDS, we've had some pretty crazy performance issues, and today we started having locking issues. Because of that, I figured it was just a timeout issue, and went to check our used memory. We were swapping about 70MB worth. I went on a memory witch-hunt with mysqltuner, and it said about 400% max memory usage possible. I now have it down to just a little > 100%., thanks to Percona's Configuration Wizard.

However, we still have this lock problem, so I'm assuming it is NOT related to the memory/swapping. Why am I still receiving the lockout? What's going on here?

I'm confident a reboot will solve the problem, but that shouldn't be the solution. What could we do to prevent this in the future? I tried flushing the query cache and the tables -- that worked.

Other kinds of flushes did not work, thanks to RDS :/

Here is a wealth of information I could figure to provide:

QUERY

INSERT INTO `myTable` (`firstName`, `lastName`, `email`) VALUES ('Travis', 'B...', '...@gmail.com')

Error message

Lock wait timeout exceeded; try restarting transaction

Table Schema

CREATE TABLE IF NOT EXISTS `myTable` (
  `id` int(15) NOT NULL AUTO_INCREMENT,
  `firstName` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
  `lastName` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
  `email` varchar(255) COLLATE utf8_unicode_ci NOT NULL,

  PRIMARY KEY (`id`),
  UNIQUE KEY `Unique Emails` (`email`),
  KEY `FullName` (`firstName`,`lastName`),
) ENGINE=InnoDB  DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=504 ;

mysqltuner.pl

 >>  MySQLTuner 1.2.0 - Major Hayden <major@mhtx.net>
 >>  Bug reports, feature requests, and downloads at http://mysqltuner.com/
 >>  Run with '--help' for additional options and output filtering
[--] Performing tests on emp:3306
[OK] Logged in using credentials passed on the command line
[--] Assuming 17511 MB of physical memory
[!!] Assuming 0 MB of swap space (use --forceswap to specify)

-------- General Statistics --------------------------------------------------
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.5.20-log

-------- Storage Engine Statistics -------------------------------------------
[--] Status: +Archive -BDB -Federated +InnoDB -ISAM -NDBCluster 
[--] Data in InnoDB tables: 451M (Tables: 75)
[--] Data in PERFORMANCE_SCHEMA tables: 0B (Tables: 17)
[!!] Total fragmented tables: 8

-------- Security Recommendations  -------------------------------------------
[OK] All database users have passwords assigned

-------- Performance Metrics -------------------------------------------------
[--] Up for: 32d 22h 37m 31s (6M q [2.145 qps], 29K conn, TX: 62B, RX: 2B)
[--] Reads / Writes: 60% / 40%
[--] Total buffers: 13.5G global + 28.6M per thread (150 max threads)
Use of uninitialized value $arch in string eq at ./mysqltuner.pl line 720 (#1)
    (W uninitialized) An undefined value was used as if it were already
    defined.  It was interpreted as a "" or a 0, but maybe it was a mistake.
    To suppress this warning assign a defined value to your variables.

    To help you figure out what was undefined, perl will try to tell you the
    name of the variable (if any) that was undefined. In some cases it cannot
    do this, so it also tells you what operation you used the undefined value
    in.  Note, however, that perl optimizes your program and the operation
    displayed in the warning may not necessarily appear literally in your
    program.  For example, "that $foo" is usually optimized into "that "
    . $foo, and the warning will refer to the concatenation (.) operator,
    even though there is no . in your program.

[!!] Maximum possible memory usage: 17.7G (103% of installed RAM)
[OK] Slow queries: 0% (7K/6M)
[OK] Highest usage of available connections: 61% (92/150)
[OK] Key buffer size / total MyISAM indexes: 32.0M/2.4M
[OK] Query cache efficiency: 82.1% (2M cached / 3M selects)
[OK] Query cache prunes per day: 0
[OK] Sorts requiring temporary tables: 0% (152 temp sorts / 106K sorts)
[OK] Temporary tables created on disk: 18% (17K on disk / 95K total)
[OK] Thread cache hit rate: 99% (92 created / 29K connections)
[!!] Table cache hit rate: 0% (86 open / 11K opened)
[OK] Open file limit used: 0% (9/65K)
[OK] Table locks acquired immediately: 99% (1M immediate / 1M locks)
[OK] InnoDB data size / buffer pool: 451.5M/12.7G

-------- Recommendations -----------------------------------------------------
General recommendations:
    Run OPTIMIZE TABLE to defragment tables for better performance
    Reduce your overall MySQL memory footprint for system stability
    Increase table_cache gradually to avoid file descriptor limits
Variables to adjust:
  *** MySQL's maximum memory usage is dangerously high ***
  *** Add RAM before increasing MySQL buffer variables ***
    table_cache (> 2048)

SHOW INNODB ENGINE STATUS

=====================================
120820 18:33:44 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 6 seconds
-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 330692 1_second, 330448 sleeps, 29989 10_second, 31250 background, 31249 flush
srv_master_thread log flush and writes: 332918
----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 67641, signal count 988874
Mutex spin waits 2257282, rounds 1886106, OS waits 12973
RW-shared spins 178174, rounds 1056448, OS waits 18278
RW-excl spins 606712, rounds 2742530, OS waits 32994
Spin rounds per wait: 0.84 mutex, 5.93 RW-shared, 4.52 RW-excl
------------
TRANSACTIONS
------------
Trx id counter 954707
Purge done for trx's n:o < 9535F2 undo n:o < 0
History list length 1943
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 0, not started
MySQL thread id 29763, OS thread handle 0x49a33940, query id 6086387LOCAL
sHOW ENGINE INNODB STATUS
---TRANSACTION 95454A, not started
MySQL thread id 29753, OS thread handle 0x49cfe940, query id 6085376EC2_1
---TRANSACTION 9543EB, not started
MySQL thread id 29752, OS thread handle 0x4a08c940, query id 6084567EC2_1
---TRANSACTION 95452A, not started
MySQL thread id 29746, OS thread handle 0x49768940, query id 6085452EC2_1
---TRANSACTION 9544BD, not started
MySQL thread id 29740, OS thread handle 0x4949d940, query id 6086384EC2_1
---TRANSACTION 954600, not started
MySQL thread id 29735, OS thread handle 0x4a51e940, query id 6086225EC2_1
---TRANSACTION 9546D1, not started
MySQL thread id 29712, OS thread handle 0x49b78940, query id 6086274 EC2_2
---TRANSACTION 9546AB, not started
MySQL thread id 29711, OS thread handle 0x497a9940, query id 6086169 EC2_2
---TRANSACTION 954587, not started
MySQL thread id 29707, OS thread handle 0x49c3b940, query id 6085510 EC2_2
---TRANSACTION 95453A, not started
MySQL thread id 29706, OS thread handle 0x49bb9940, query id 6085342 EC2_2
---TRANSACTION 9544CD, not started
MySQL thread id 29699, OS thread handle 0x499b1940, query id 6085099 EC2_2
---TRANSACTION 95464B, not started
MySQL thread id 29705, OS thread handle 0x4982b940, query id 6085987 EC2_2
---TRANSACTION 9545D4, not started
MySQL thread id 29704, OS thread handle 0x4172f940, query id 6085675 EC2_2
---TRANSACTION 9545F8, not started
MySQL thread id 29703, OS thread handle 0x4a357940, query id 6085800 EC2_2
---TRANSACTION 954468, not started
MySQL thread id 29702, OS thread handle 0x41a7f940, query id 6084853 EC2_2
---TRANSACTION 954496, not started
MySQL thread id 29700, OS thread handle 0x496e6940, query id 6084976 EC2_2
---TRANSACTION 95451A, not started
MySQL thread id 29701, OS thread handle 0x41a3e940, query id 6085264 EC2_2
---TRANSACTION 952531, not started
MySQL thread id 29690, OS thread handle 0x4165d940, query id 6066385 EC2_3
---TRANSACTION 954608, not started
MySQL thread id 29654, OS thread handle 0x416ee940, query id 6085838EC2_1
---TRANSACTION 952052, not started
MySQL thread id 29620, OS thread handle 0x49b37940, query id 6062029 EC2_3
---TRANSACTION 951EE8, not started
MySQL thread id 29608, OS thread handle 0x49f88940, query id 6060576 EC2_3
---TRANSACTION 951E7A, not started
MySQL thread id 29606, OS thread handle 0x41ac0940, query id 6060179 EC2_3
---TRANSACTION 0, not started
MySQL thread id 29417, OS thread handle 0x498ee940, query id 6081627LOCAL
---TRANSACTION 951EDE, not started
MySQL thread id 29463, OS thread handle 0x49f06940, query id 6060539 EC2_3
---TRANSACTION 954458, not started
MySQL thread id 29169, OS thread handle 0x49bfa940, query id 6084817EC2_1
---TRANSACTION 954403, not started
MySQL thread id 29154, OS thread handle 0x49af6940, query id 6085912EC2_1
---TRANSACTION 954625, not started
MySQL thread id 29173, OS thread handle 0x4a398940, query id 6085896EC2_1
---TRANSACTION 9546C1, not started
MySQL thread id 29170, OS thread handle 0x4a316940, query id 6086239EC2_1
---TRANSACTION 9544A2, not started
MySQL thread id 29168, OS thread handle 0x499f2940, query id 6085002EC2_1
---TRANSACTION 9545AD, not started
MySQL thread id 29153, OS thread handle 0x4945c940, query id 6085594EC2_1
---TRANSACTION 952679, not started
MySQL thread id 29418, OS thread handle 0x49cbd940, query id 6081627LOCAL
---TRANSACTION 9543F3, not started
MySQL thread id 29156, OS thread handle 0x49e02940, query id 6086208EC2_1
---TRANSACTION 954542, not started
MySQL thread id 29161, OS thread handle 0x41770940, query id 6085530EC2_1
---TRANSACTION 9546FE, not started
MySQL thread id 29150, OS thread handle 0x494de940, query id 6086368EC2_1
---TRANSACTION 954480, not started
MySQL thread id 29157, OS thread handle 0x41874940, query id 6085286EC2_1
---TRANSACTION 9546EF, not started
MySQL thread id 29155, OS thread handle 0x49e84940, query id 6086336EC2_1
---TRANSACTION 9544A5, not started
MySQL thread id 29174, OS thread handle 0x418b5940, query id 6085288EC2_1
---TRANSACTION 954522, not started
MySQL thread id 29158, OS thread handle 0x4a00a940, query id 6085281EC2_1
---TRANSACTION 954439, not started
MySQL thread id 29152, OS thread handle 0x4a2d5940, query id 6084755EC2_1
---TRANSACTION 9546D9, not started
MySQL thread id 29159, OS thread handle 0x4992f940, query id 6086289EC2_1
---TRANSACTION 9546E1, not started
MySQL thread id 29162, OS thread handle 0x49d3f940, query id 6086310EC2_1
---TRANSACTION 954706, not started
MySQL thread id 29151, OS thread handle 0x49560940, query id 6086381EC2_1
---TRANSACTION 954558, not started
MySQL thread id 29164, OS thread handle 0x417f2940, query id 6085402EC2_1
---TRANSACTION 95468A, not started
MySQL thread id 29172, OS thread handle 0x49dc1940, query id 6086104EC2_1
---TRANSACTION 95462D, not started
MySQL thread id 29171, OS thread handle 0x4a45b940, query id 6086031EC2_1
---TRANSACTION 9546F7, not started
MySQL thread id 29149, OS thread handle 0x4a14f940, query id 6086362EC2_1
---TRANSACTION 954597, not started
MySQL thread id 29163, OS thread handle 0x41978940, query id 6085714EC2_1
---TRANSACTION 954567, not started
MySQL thread id 29166, OS thread handle 0x49f47940, query id 6085434EC2_1
---TRANSACTION 954577, not started
MySQL thread id 29165, OS thread handle 0x49970940, query id 6086386EC2_1
---TRANSACTION 9545EE, not started
MySQL thread id 29211, OS thread handle 0x41b83940, query id 6085767EC2_4
---TRANSACTION 954532, not started
MySQL thread id 29193, OS thread handle 0x4951f940, query id 6085322EC2_4
---TRANSACTION 95457F, not started
MySQL thread id 29194, OS thread handle 0x41833940, query id 6085490EC2_4
---TRANSACTION 95466F, not started
MySQL thread id 29202, OS thread handle 0x4a3d9940, query id 6086048EC2_4
---TRANSACTION 9544C5, not started
MySQL thread id 29201, OS thread handle 0x497ea940, query id 6085081EC2_4
---TRANSACTION 954512, not started
MySQL thread id 29200, OS thread handle 0x4941b940, query id 6085245EC2_4
---TRANSACTION 9546C9, not started
MySQL thread id 29199, OS thread handle 0x4161c940, query id 6086256EC2_4
---TRANSACTION 95463D, not started
MySQL thread id 29198, OS thread handle 0x496a5940, query id 6085950EC2_4
---TRANSACTION 9545C4, not started
MySQL thread id 29197, OS thread handle 0x4a49c940, query id 6085645EC2_4
---TRANSACTION 954460, not started
MySQL thread id 29196, OS thread handle 0x4986c940, query id 6084834EC2_4
---TRANSACTION 94DEA8, not started
MySQL thread id 21921, OS thread handle 0x415db940, query id 6086180 localhost 127.0.0.1 rdsadmin
---TRANSACTION 94E45F, ACTIVE 12759 sec
20 lock struct(s), heap size 3112, 4 row lock(s), undo log entries 78
MySQL thread id 29195, OS thread handle 0x418f6940, query id 6084915EC2_4
--------
FILE I/O
--------
I/O thread 0 state: waiting for i/o request (insert buffer thread)
I/O thread 1 state: waiting for i/o request (log thread)
I/O thread 2 state: waiting for i/o request (read thread)
I/O thread 3 state: waiting for i/o request (read thread)
I/O thread 4 state: waiting for i/o request (read thread)
I/O thread 5 state: waiting for i/o request (read thread)
I/O thread 6 state: waiting for i/o request (write thread)
I/O thread 7 state: waiting for i/o request (write thread)
I/O thread 8 state: waiting for i/o request (write thread)
I/O thread 9 state: waiting for i/o request (write thread)
Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] ,
 ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0
Pending flushes (fsync) log: 0; buffer pool: 0
61692 OS file reads, 618676 OS file writes, 263209 OS fsyncs
0.00 reads/s, 0 avg bytes/read, 0.17 writes/s, 0.17 fsyncs/s
-------------------------------------
INSERT BUFFER AND ADAPTIVE HASH INDEX
-------------------------------------
Ibuf: size 1, free list len 27, seg size 29, 13470 merges
merged operations:
 insert 16085, delete mark 1, delete 0
discarded operations:
 insert 0, delete mark 0, delete 0
Hash table size 26925443, node heap has 12297 buffer(s)
88.49 hash searches/s, 299.95 non-hash searches/s
---
LOG
---
Log sequence number 26883102702
Log flushed up to   26883102702
Last checkpoint at  26883102692
0 pending log writes, 0 pending chkp writes
147530 log i/o's done, 0.17 log i/o's/second
----------------------
BUFFER POOL AND MEMORY
----------------------
Total memory allocated 13924155392; in additional pool allocated 0
Dictionary memory allocated 584219
Buffer pool size   830399
Free buffers       4746
Database pages     813356
Old database pages 300262
Modified db pages  1
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 325, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 83323, created 730033, written 2387294
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 813356, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
--------------
ROW OPERATIONS
--------------
0 queries inside InnoDB, 0 queries in queue
1 read views open inside InnoDB
Main thread process no. 7755, id 1197041984, state: sleeping
Number of rows inserted 69343335, updated 755918, deleted 6235734, read 5140753915
0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 542.41 reads/s
----------------------------
END OF INNODB MONITOR OUTPUT

SHOW STATUS

Variable_name   Value
Aborted_clients 6941
Aborted_connects    39
Binlog_cache_disk_use   171
Binlog_cache_use    43567
Binlog_stmt_cache_disk_use  2
Binlog_stmt_cache_use   82079
Bytes_received  983
Bytes_sent  5749
Com_admin_commands  11
Com_assign_to_keycache  0
Com_alter_db    0
Com_alter_db_upgrade    0
Com_alter_event 0
Com_alter_function  0
Com_alter_procedure 0
Com_alter_server    0
Com_alter_table 0
Com_alter_tablespace    0
Com_analyze 0
Com_begin   0
Com_binlog  0
Com_call_procedure  0
Com_change_db   1
Com_change_master   0
Com_check   0
Com_checksum    0
Com_commit  0
Com_create_db   0
Com_create_event    0
Com_create_function 0
Com_create_index    0
Com_create_procedure    0
Com_create_server   0
Com_create_table    0
Com_create_trigger  0
Com_create_udf  0
Com_create_user 0
Com_create_view 0
Com_dealloc_sql 0
Com_delete  0
Com_delete_multi    0
Com_do  0
Com_drop_db 0
Com_drop_event  0
Com_drop_function   0
Com_drop_index  0
Com_drop_procedure  0
Com_drop_server 0
Com_drop_table  0
Com_drop_trigger    0
Com_drop_user   0
Com_drop_view   0
Com_empty_query 0
Com_execute_sql 0
Com_flush   2
Com_grant   0
Com_ha_close    0
Com_ha_open 0
Com_ha_read 0
Com_help    0
Com_insert  0
Com_insert_select   0
Com_install_plugin  0
Com_kill    0
Com_load    0
Com_lock_tables 0
Com_optimize    0
Com_preload_keys    0
Com_prepare_sql 0
Com_purge   0
Com_purge_before_date   0
Com_release_savepoint   0
Com_rename_table    0
Com_rename_user 0
Com_repair  0
Com_replace 0
Com_replace_select  0
Com_reset   0
Com_resignal    0
Com_revoke  0
Com_revoke_all  0
Com_rollback    0
Com_rollback_to_savepoint   0
Com_savepoint   0
Com_select  4
Com_set_option  6
Com_signal  0
Com_show_authors    0
Com_show_binlog_events  0
Com_show_binlogs    0
Com_show_charsets   0
Com_show_collations 0
Com_show_contributors   0
Com_show_create_db  0
Com_show_create_event   0
Com_show_create_func    0
Com_show_create_proc    0
Com_show_create_table   0
Com_show_create_trigger 0
Com_show_databases  0
Com_show_engine_logs    0
Com_show_engine_mutex   0
Com_show_engine_status  1
Com_show_events 0
Com_show_errors 0
Com_show_fields 0
Com_show_function_status    0
Com_show_grants 0
Com_show_keys   0
Com_show_master_status  0
Com_show_open_tables    0
Com_show_plugins    0
Com_show_privileges 0
Com_show_procedure_status   0
Com_show_processlist    1
Com_show_profile    0
Com_show_profiles   0
Com_show_relaylog_events    0
Com_show_slave_hosts    0
Com_show_slave_status   0
Com_show_status 1
Com_show_storage_engines    0
Com_show_table_status   0
Com_show_tables 0
Com_show_triggers   0
Com_show_variables  2
Com_show_warnings   2
Com_slave_start 0
Com_slave_stop  0
Com_stmt_close  0
Com_stmt_execute    0
Com_stmt_fetch  0
Com_stmt_prepare    0
Com_stmt_reprepare  0
Com_stmt_reset  0
Com_stmt_send_long_data 0
Com_truncate    0
Com_uninstall_plugin    0
Com_unlock_tables   0
Com_update  0
Com_update_multi    0
Com_xa_commit   0
Com_xa_end  0
Com_xa_prepare  0
Com_xa_recover  0
Com_xa_rollback 0
Com_xa_start    0
Compression ON
Connections 29818
Created_tmp_disk_tables 0
Created_tmp_files   524
Created_tmp_tables  2
Delayed_errors  0
Delayed_insert_threads  0
Delayed_writes  0
Flush_commands  5
Handler_commit  0
Handler_delete  0
Handler_discover    0
Handler_prepare 0
Handler_read_first  0
Handler_read_key    0
Handler_read_last   0
Handler_read_next   0
Handler_read_prev   0
Handler_read_rnd    0
Handler_read_rnd_next   4
Handler_rollback    0
Handler_savepoint   0
Handler_savepoint_rollback  0
Handler_update  0
Handler_write   2
Innodb_buffer_pool_pages_data   813356
Innodb_buffer_pool_pages_dirty  0
Innodb_buffer_pool_pages_flushed    4774898
Innodb_buffer_pool_pages_free   4746
Innodb_buffer_pool_pages_misc   12297
Innodb_buffer_pool_pages_total  830399
Innodb_buffer_pool_read_ahead_rnd   0
Innodb_buffer_pool_read_ahead   20910
Innodb_buffer_pool_read_ahead_evicted   0
Innodb_buffer_pool_read_requests    10304108333
Innodb_buffer_pool_reads    49001
Innodb_buffer_pool_wait_free    0
Innodb_buffer_pool_write_requests   617475366
Innodb_data_fsyncs  263503
Innodb_data_pending_fsyncs  0
Innodb_data_pending_reads   0
Innodb_data_pending_writes  0
Innodb_data_read    1369460736
Innodb_data_reads   61692
Innodb_data_writes  619040
Innodb_data_written 100213573632
Innodb_dblwr_pages_written  2387449
Innodb_dblwr_writes 42845
Innodb_have_atomic_builtins ON
Innodb_log_waits    29
Innodb_log_write_requests   47769295
Innodb_log_writes   134764
Innodb_os_log_fsyncs    144634
Innodb_os_log_pending_fsyncs    0
Innodb_os_log_pending_writes    0
Innodb_os_log_written   21975030784
Innodb_page_size    16384
Innodb_pages_created    730033
Innodb_pages_read   83323
Innodb_pages_written    2387449
Innodb_row_lock_current_waits   0
Innodb_row_lock_time    779558
Innodb_row_lock_time_avg    3765
Innodb_row_lock_time_max    51611
Innodb_row_lock_waits   207
Innodb_rows_deleted 6235739
Innodb_rows_inserted    69343350
Innodb_rows_read    5143551347
Innodb_rows_updated 755920
Innodb_truncated_status_writes  0
Key_blocks_not_flushed  0
Key_blocks_unused   26792
Key_blocks_used 0
Key_read_requests   0
Key_reads   0
Key_write_requests  0
Key_writes  0
Last_query_cost 0.000000
Max_used_connections    92
Not_flushed_delayed_rows    0
Open_files  9
Open_streams    0
Open_table_definitions  77
Open_tables 86
Opened_files    197336
Opened_table_definitions    0
Opened_tables   0
Performance_schema_cond_classes_lost    0
Performance_schema_cond_instances_lost  0
Performance_schema_file_classes_lost    0
Performance_schema_file_handles_lost    0
Performance_schema_file_instances_lost  0
Performance_schema_locker_lost  0
Performance_schema_mutex_classes_lost   0
Performance_schema_mutex_instances_lost 0
Performance_schema_rwlock_classes_lost  0
Performance_schema_rwlock_instances_lost    0
Performance_schema_table_handles_lost   0
Performance_schema_table_instances_lost 0
Performance_schema_thread_classes_lost  0
Performance_schema_thread_instances_lost    0
Prepared_stmt_count 0
Qcache_free_blocks  1
Qcache_free_memory  536615520
Qcache_hits 2766375
Qcache_inserts  56621
Qcache_lowmem_prunes    0
Qcache_not_cached   544557
Qcache_queries_in_cache 128
Qcache_total_blocks 313
Queries 6115939
Questions   23
Rpl_status  AUTH_MASTER
Select_full_join    0
Select_full_range_join  0
Select_range    0
Select_range_check  0
Select_scan 2
Slave_heartbeat_period  0.000
Slave_open_temp_tables  0
Slave_received_heartbeats   0
Slave_retried_transactions  0
Slave_running   OFF
Slow_launch_threads 0
Slow_queries    0
Sort_merge_passes   0
Sort_range  0
Sort_rows   0
Sort_scan   0
Ssl_accept_renegotiates 0
Ssl_accepts 0
Ssl_callback_cache_hits 0
Ssl_cipher  
Ssl_cipher_list 
Ssl_client_connects 0
Ssl_connect_renegotiates    0
Ssl_ctx_verify_depth    0
Ssl_ctx_verify_mode 0
Ssl_default_timeout 0
Ssl_finished_accepts    0
Ssl_finished_connects   0
Ssl_session_cache_hits  0
Ssl_session_cache_misses    0
Ssl_session_cache_mode  Unknown
Ssl_session_cache_overflows 0
Ssl_session_cache_size  0
Ssl_session_cache_timeouts  0
Ssl_sessions_reused 0
Ssl_used_session_cache_entries  0
Ssl_verify_depth    0
Ssl_verify_mode 0
Ssl_version 
Table_locks_immediate   1422155
Table_locks_waited  10
Tc_log_max_pages_used   0
Tc_log_page_size    0
Tc_log_page_waits   7
Threads_cached  6
Threads_connected   63
Threads_created 92
Threads_running 1
Uptime  2845992
Uptime_since_flush_status   2845992

Database sizes

SELECT table_schema AS  "Data Base Name", ROUND( SUM( data_length + index_length ) /1024 /1024, 2 ) AS  "Data Base Size in MB"
FROM information_schema.TABLES
GROUP BY table_schema
# information_schema.TABLES
-
  Data Base Name: "Production"
  Data Base Size in MB: 848.22
-
  Data Base Name: "information_schema"
  Data Base Size in MB: 0.01
-
  Data Base Name: "mysql"
  Data Base Size in MB: 5.40
-
  Data Base Name: "performance_schema"
  Data Base Size in MB: 0.00
...

On an m2.xlarge instance

RDS Pictures

The table myTable exists in the 'SHOW OPEN TABLES' query, but it does not say it is locked.

share|improve this question
    
Just curious, what RDS Server Model are you using ? t1.micro or m1.small ?? –  RolandoMySQLDBA Aug 20 '12 at 19:42
    
It's an m2.xlarge model. I'll update the question. Thanks for reminding me! –  Travis Aug 20 '12 at 19:43
    
Can we see the output from information_schema.innodb_trx? –  JM Hicks Jan 9 '13 at 7:36
    
We have since recreated the instance and have not had any problems so far. I'll update here if it ever happens again. –  Travis Jan 14 '13 at 18:18
    
I am not sure this is an application level problem. We got the same behaviour for no apparent reason. It corrected itself when it ran out of space and reverted when it hit the swap. What could consume 15 GB of disc space and free it in 10 minutes. –  user36492 Apr 1 at 13:20

1 Answer 1

Since you are running MySQL 5.5, have you considering tuning InnoDB for better performance?

There are several variables that have been added to the MySQL 5.1 InnoDB Plugin that are now native to MySQL 5.5. They usually help increase hyperthreading and take advantage of more IOPS if the environment can handled it. In your case, you should be able to.

I have spun RDS models before (See my post : Local database vs Amazon RDS) and can tell you that RDS is not very helpful in tuning InnoDB. You will have to take the bull by the horns on this one.

When you spun up the RDS instance, you probably used the default DB Parameter Group

You should be able to create a new DB Parameter Group for yourself. When you get to this, set the following:

I suggest these because they are default in RDS for innodb_read_io_threads and innodb_write_io_threads are just too low. In addition, your SHOW INNODB ENGINE STATUS\G

--------
FILE I/O
--------
I/O thread 0 state: waiting for i/o request (insert buffer thread)
I/O thread 1 state: waiting for i/o request (log thread)
I/O thread 2 state: waiting for i/o request (read thread)
I/O thread 3 state: waiting for i/o request (read thread)
I/O thread 4 state: waiting for i/o request (read thread)
I/O thread 5 state: waiting for i/o request (read thread)
I/O thread 6 state: waiting for i/o request (write thread)
I/O thread 7 state: waiting for i/o request (write thread)
I/O thread 8 state: waiting for i/o request (write thread)
I/O thread 9 state: waiting for i/o request (write thread)

only shows the default of 4 for each class of io threads.

Here is the Bad News: To implement config changes, once you create your own DB Parameter Group, you must export the data from the old RDS instance, create a new instance using your new DB Parameter Group, and reload. In more detail...

  • Create a Custom DB Parameter Group (call it MySettings)
  • Download RDS CLI and setup a config file with your AWS Credentials
  • Execute the following : ./rds-modify-db-parameter-group MySettings --parameters "name=innodb_read_io_threads,value=16,method=immediate"
  • Modify using DB Parameter Option List MySettings
  • Restart the MySQL RDS Instance

I hope this helps !!!

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.