2

Suddenly I notice my replication stopped and first I got some primary key duplication error when I checked using the show slave status\G. Then I remove those lines and restarted and ending up now with this error. So I first increase both the master and slave max_allowed_packet to 1G yet not working and now I changed to 2G yet is the same. I have run stop slave and change master to master_host='192.168.**',master_user='repu1',master_password='***', master_log_file='mysql-bin.000039', master_log_pos=640109146; start slave; yet I keep getting the same error. "Got fatal error 1236 from master when reading data from binary log: 'log event entry exceeded max_allowed_packet; Increase max_allowed_packet on master'"

I have checked the repu1 password using this command mysql -h 192.168.* -u repu1 -p its working fine too.

show slave status \G;
*************************** 1. row ***************************
               Slave_IO_State:
                  Master_Host: 192.168.100.12
                  Master_User: repu1
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000039
          Read_Master_Log_Pos: 640355070
               Relay_Log_File: mysqld-relay-bin.000002
                Relay_Log_Pos: 251
        Relay_Master_Log_File: mysql-bin.000039
             Slave_IO_Running: No
            Slave_SQL_Running: No
              Replicate_Do_DB: dbn
          Replicate_Ignore_DB:
           Replicate_Do_Table:
       Replicate_Ignore_Table:
      Replicate_Wild_Do_Table:
  Replicate_Wild_Ignore_Table:
                   Last_Errno: 1594
                   Last_Error: Relay log read failure: Could not parse relay log event entry. The possible reasons are: the master's binary log is corrupted (you can check this by running 'mysqlbinlog' on the binary log), the slave's relay log is corrupted (you can check this by running 'mysqlbinlog' on the relay log), a network problem, or a bug in the master's or slave's MySQL code. If you want to check the master's binary log or slave's relay log, you will be able to know their names by issuing 'SHOW SLAVE STATUS' on this slave.
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 640109146
              Relay_Log_Space: 247058
              Until_Condition: None
               Until_Log_File:
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File:
           Master_SSL_CA_Path:
              Master_SSL_Cert:
            Master_SSL_Cipher:
               Master_SSL_Key:
        Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 1236
                Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'log event entry exceeded max_allowed_packet; Increase max_allowed_packet on master'
               Last_SQL_Errno: 1594
               Last_SQL_Error: Relay log read failure: Could not parse relay log event entry. The possible reasons are: the master's binary lo

3 Answers 3

1

Try these steps:

1) Take a mysqldump from master

2) Restore it on a slave, whether it gets restored successfully?

3) You should not keep max_allowed_packet to a high value like 1G or 2G keep it minimal like 16M or 32M and restart Master server.

4) Apply same max_allowed_packet size in slave as well.3

5) Now, stop the slave, execute Change Master .... query accourdingly and start slave

I hope this will work for you.

4
  • the problem is my data is coming in live and I got no chance to down the master. Any resolution to up the slave without down doing the the mysqldump ?
    – biz14
    Dec 5, 2013 at 5:02
  • 1
    a) take a master's binlog position and filename b) stop slave; c) Change master to ... current master's position and filename d) Start slave; Incase you might have missed some records in Slave you can get back those by syncing table-by-table using Percona tools or SQLyog Dec 5, 2013 at 5:28
  • I have done this change master to master_host='192.168.**',master_user='repu1',master_password='***', master_log_file='mysql-bin.000039', master_log_pos=640109146; start slave; see still not working. So can I change the master_log_pos to something else is that advisable ?
    – biz14
    Dec 5, 2013 at 6:01
  • I dont mind loosing one or two lines which I can recover later.
    – biz14
    Dec 5, 2013 at 14:15
1

This error is usually caused by a hardware glitch or a bug so obscure that it hasn't been found in over a decade. The optimal solution that will definitely fix it without any risk of anomalies creeping in is to fully re-initialize the replica. You can do this without downtime or locking by using either xtrabackup or mysqldump --single-transaction.

Having said that, this can be prohibitively expensive if you have terabytes of data, so there may be a way you can restore replication without rebuilding the slave completely. Use mysqlbinlog on the master to identify where the transactions begin, and look for the adjacent transactions near position 640109146 (your Exec_Master_Log_Pos). There is a very good chance that the position listed is actually erroneous and no transaction in the binlog starts at that position. You are also very likely to find that the coordinate distances between these transactions are nowhere near your max_allowed_packet value.

If that is the case, you can reset your replication coordinates on the slave to the last transaction mysqlbinlog lists that is before the position the slave thinks it is at. You can find more details on exactly what to do in this mysql troubleshooting article. There is risk associated with doing this (if the coordinates got corrupted, there is no telling what else got corrupted), but if doing the full re-initialisation immediately is prohibitively expensive, you might be able to buy yourself some time using this approach, at least until you can verify (e.g. using pt-table-checksum whether you have to rebuild the entire replica data set.

HTH

Disclosure: I wrote the referenced article.

0
0

I believe I know how to reproduce this bug. I ran in to it when updating a large record set. I'm on mariaDB 10.3.32

I have a table with a TEXT column. It tends to have a few kilobytes of data in each row. I was clearing that column out to anonymize a clone of the table. I was doing it in chunks of 500k.

UPDATE mytable SET mycol='' WHERE id BETWEEN BETWEEN 6500000 AND 6999999;

The binlog would have the SET @4='' ... WHERE @4='3k of old text here' for each row. 500k rows of 3k of data would be over 1G.

I had to search the bin log file for the next statement, change master to that position. Then start slave again.

New contributor
David is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.