We're running MySQL on a machine with 8GB of memory. The innodb_buffer_pool_size is set to 1GB. Recently, MySQL crashed due to a full memory condition. Our monitoring tool showed 100% memory utilization by MySQL when a specific process was running. This process involved inserting data into a table with many TEXT fields.
Here's the interesting part:
Our monitoring tool's historical data suggests this process has caused 100% memory usage before, but fortunately, MySQL didn't crash then. In those instances, memory usage likely returned to near the innodb_buffer_pool_size level after the process completed. I confirmed that MySQL was solely responsible for the high memory utilization by analyzing the output of the top command. I thought MySQL wouldn't use much more memory than the innodb_buffer_pool_size, but I guess I was wrong.
Additional Information:
MySQL server version: 8.0.39 . All parameters are default except max_allowed_packet (set to 500MB). I tried reducing max_allowed_packet to 64MB but it caused our insertion process to fail, requiring the value to be increased back to 500MB.
Question:
Based on this experience, why is MySQL using all available memory instead of staying within the innodb_buffer_pool_size limit or close to that level?
Although a memory leak bug in MySQL 8.0.39 seems improbable, the release notes for MySQL 8.0.40 indicate a bug fix that may be relevant 'Fixed a memory leak in the mysql client. (Bug #36600203)' https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-40.html , I am uncertain whether this behavior is expected, a configuration issue, or a bug. Please advise.
