1

After running the following queries:

CREATE TABLE mem_test (i int NOT NULL AUTO_INCREMENT PRIMARY KEY) ENGINE=MEMORY;
INSERT INTO mem_test () VALUES ();
SELECT table_rows,avg_row_length,data_length,index_length FROM INFORMATION_SCHEMA.TABLES WHERE table_name='mem_test';

I get this in MySQL 5.7.28:

+------------+----------------+-------------+--------------+
| table_rows | avg_row_length | data_length | index_length |
+------------+----------------+-------------+--------------+
|          1 |              8 |      126992 |       126984 |
+------------+----------------+-------------+--------------+

and this in MariaDB 10.4.10:

+------------+----------------+-------------+--------------+
| table_rows | avg_row_length | data_length | index_length |
+------------+----------------+-------------+--------------+
|          1 |              5 |     1677712 |      2093064 |
+------------+----------------+-------------+--------------+

My table will never grow anywhere close to the size of these pre-allocated buffers, so how can I get MySQL and MariaDB to waste less RAM? Or aren't they actually using what this query indicates??

Additional comparisons, for reference:

+-------+--------+------------+----------------+-------------+--------------+
| DB    | Engine | table_rows | avg_row_length | data_length | index_length |
+-------+--------+------------+----------------+-------------+--------------+
| MySQL | MEMORY |          1 |              8 |      126992 |       126984 |
| Maria | MEMORY |          1 |              5 |     1677712 |      2093064 |
| Both  | InnoDB |          1 |          16384 |       16384 |            0 |
| Both  | MyISAM |          1 |              7 |           7 |         2048 |
+-------+--------+------------+----------------+-------------+--------------+

I found a note on the MySQL forums that "memory tables grow by 'read_buffer_size' bytes when they need extending" (https://forums.mysql.com/read.php?92,405702,417606#msg-417606), but my numbers don't agree: I have read_buffer_size = 131072 in MySQL and 2097152 in MariaDB.

8
  • Please provide SHOW VARIABLES LIKE '%heap%';
    – Rick James
    Jan 9, 2020 at 5:42
  • Hmmm... I got 1 | 8 | 2093072 | 2093072 | on 5.6.22.
    – Rick James
    Jan 9, 2020 at 5:50
  • I get 1 | 5 | 126992 | 126984 | with 10.4.11 using default settings, so similar to what is reported in the question for 5.7.28.
    – dbdemon
    Jan 9, 2020 at 10:19
  • SHOW VARIABLES LIKE '%heap%'; gives me max_heap_table_size = 16777216 (MySQL) or 33554432 (MariaDB), and no other variables. Jan 9, 2020 at 15:55
  • 1
    @RogerDueck It looks like your MariaDB 'read_buffer_size' should be removed from the configuration to allow defaults to work for you, shutdown; restart and redo your tests to see if you achieve your goal of lowering the RAM required for the MEMORY table. Mar 6, 2020 at 16:20

1 Answer 1

1

Although data_length and index_length aren't exact multiples of read_buffer_size, they are based on it. So adjusting read_buffer_size in my.cnf is the solution (thanks @Wilson Hauck), as Shane Bester pointed out on the MySQL forums:

memory tables grow by 'read_buffer_size' bytes when they need extending (https://forums.mysql.com/read.php?92,405702,417606#msg-417606)

Based on performance tests by Peter Zaitsev in 2007 (https://www.percona.com/blog/2007/09/17/mysql-what-read_buffer_size-value-is-optimal/), the default 128K setting is generally best, and using smaller values brings very little penalty.

4
  • 1
    Roger, thanks for digging deeper than most of us would for the CAUSE of this use of RAM. Caution, if Handler_read_next Rate Per Second increases significantly, you may have an edge case that can benefit from larger read_buffer_size (double it, and monitor RPS and general response time). Mar 7, 2020 at 15:31
  • How do I get handler_read_next rate per second? By running SHOW GLOBAL STATUS LIKE 'handler_read_next' regularly and tracking the rate of change? Is there a general range of values that would be considered ideal? Mar 16, 2020 at 15:21
  • Yes, divide the result by uptime (seconds) for RPS. View my profile, Network profile for contact info and consider our free download from Utility Scripts the globalstatusage10sec.sql that will calculate the rate at the end of 10 seconds for you. Mar 16, 2020 at 15:28
  • Additional information request. RAM size, # cores, any SSD or NVME devices on MySQL Host server? Post on pastebin.com and share the links. From your SSH login root, Text results of: B) SHOW GLOBAL STATUS; after minimum 24 hours UPTIME C) SHOW GLOBAL VARIABLES; D) SHOW FULL PROCESSLIST; E) complete MySQLTuner report AND Optional helpful information, if available includes - htop OR top for most active apps, ulimit -a for a Linux/Unix list of limits, iostat -xm 5 3 for IOPS by device and core/cpu count, for server workload tuning analysis to provide suggestions. Mar 19, 2020 at 14:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.