MySQL - [Note] Got an error reading communication packets

Question

I have deployed MySQL on OpenShift (kubernetes) with GlusterFS as persistent storage and i am getting following error after every 2 seconds in MySQL Container logs

2020-01-17T18:00:04.787586Z 1541 [Note] Got an error reading communication packets
2020-01-17T18:00:06.787610Z 1543 [Note] Got an error reading communication packets
2020-01-17T18:00:08.787548Z 1544 [Note] Got an error reading communication packets
2020-01-17T18:00:10.787586Z 1546 [Note] Got an error reading communication packets
2020-01-17T18:00:12.787575Z 1547 [Note] Got an error reading communication packets

I tried by changing below variables but no success

• set @@global.max_connections = 400;
• max_allowed_packet=256M
• innodb_log_buffer_size = 32M
• innodb_log_file_size = 2047M
• SET GLOBAL thread_cache_size=50;

i tried to set SET GLOBAL log_warnings=3; for details logging but nothing is showing

$ mysql --version
mysql  Ver 14.14 Distrib 5.7.28, for Linux (x86_64) using  EditLine wrapper

Internet is full of tips about how to solve this error message, but the usual solutions are tweaking the various timeout and max_allowed_packet settings. However, changing these settings made no difference to me, the error kept happening infact now i shut all application and only MySql is running but still error is repeating. Although my services are not hampering but i have implemented Centralized logging and this log is flooding it.

I also run mysqltuner but its suggestion are for query cache only and tunning it has no effect

https://www.linode.com/docs/databases/mysql/how-to-optimize-mysql-performance-using-mysqltuner/

So if there is not client why MySQL is showing this message as no reading or writing happening??

Extra Details:

• MySql Starting logs as they start popping as soon as mysql start https://gist.github.com/imranrazakhan/d5ccca11c840b3229bf2cedb1a2deebb

• SHOW GLOBAL STATUS https://gist.github.com/imranrazakhan/5b0f06529916435e5df71aebdb8062c0

• SHOW GLOBAL VARIABLES; https://gist.github.com/imranrazakhan/2bf9cadd5d1ae0e350feeecd3a984cef

• SHOW FULL PROCESSLIST;

• mysqltuner output; https://gist.github.com/imranrazakhan/71e4638b638364a939ac1410765188ee

• ulimit -a;

  https://gist.github.com/imranrazakhan/cadf78df1d09e4ad11676d51ec02839b

• MySQL Conf https://gist.github.com/imranrazakhan/5ec96905048758eb49cf7ac6140bdfff

Answer 1

After spending lot of time I am able to resolve my issue and reason is that in Openshitf/kubernetes we have section to verify readiness of mysql like

  readinessProbe:
    tcpSocket:
      port: 3306
    initialDelaySeconds: 20
    periodSeconds: 2
    timeoutSeconds: 1

So it was pinging every two seconds on 3306 port, i changed it to below and its resolved the issue,

 readinessProbe:
        exec:
          command:
          - sh
          - -c
          - "mysqladmin ping -u root -p${MYSQL_ROOT_PASSWORD}"
        initialDelaySeconds: 5
        periodSeconds: 2
        timeoutSeconds: 1