3

I have a scaling issue with an application, it uses Aurora as the DB and uses Slick as the ORM. I am not sure where the issue lies but I have a question releated to MySQL / Aurora that I hope will shed some light on my issue.

The application needs a significant number of connections in the pool otherwise the application starts failing. However the when I look at INFORMATION_SCHEMA.PROCESSLIST I can see a significant number of threads in the 'cleaned up' state.

I can't see 'cleaned up' on the list of general states - https://dev.mysql.com/doc/refman/5.7/en/general-thread-states.html

What does 'cleaned up' mean exactly? ( I can guess but I can't find any documentation )

below is a count of the number of threads total vs the number in 'cleaned up'

mysql> select count(*) from INFORMATION_SCHEMA.PROCESSLIST where STATE = 'cleaned up' order by time desc ;
+----------+
| count(*) |
+----------+
|    11982 |
+----------+
1 row in set (0.08 sec)

mysql> select count(*) from INFORMATION_SCHEMA.PROCESSLIST order by time desc ;
+----------+
| count(*) |
+----------+
|    12007 |
+----------+
1 row in set (0.08 sec)
2

Finally found the answer, its an Aurora specific state that has the same meaning as the blank state in MySQL.

Cleaned Up state means it has completed and cleaned up and waiting for the connection to close.

https://forums.aws.amazon.com/thread.jspa?messageID=708499

0

(Not an Answer, but a course of action...)

What is the value of max_connections? It sounds like it is terribly large. It is a very rare system that needs more than 1000 for that setting.

So.. Lower the setting (in my.cnf) to 500 and restart mysqld. This will either fix the problem of push it back to the client, which will then complaint that it cannot get connection to the server.

So, then, ... figure out why there are so many clients clamoring for database activity.

And, find out what queries are so slow that they can't get in and out quickly. It may be as simple as a missing composite index.

  • max_connections is very large. 15,000. I have control over the clients but had a miss configuration. The result was to reduce the number of connections each client kept in its connection pool. Performance much better now. – Sam Sep 21 '16 at 16:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.