I have run into some issues with replication
After an in-place upgrade of mySQL from 5.0.77 -> 5.1.67 rel 14.4 Percona
replication between a master and multiple slaves is causing issues. slaves are at the same level as master.
The replication mode is mixed (SBR and RBR)
The master seems to be truncating the log incorrectly, seemingly at random, and slaves subsequently stop replicating due to corrupt log being sent through.
mysqlbinlog also segfaults when trying to read the log.
This is the error being thrown by the slaves (table names/server names obfuscated):
Last_Error: Could not execute Update_rows event on table ozdb1.tableA; Incorrect string >value: '\x8EIQ\x13)\x00...' for column 'Description1' at row 2, Error_code: 1366; >Incorrect string value: '\x8E\x08\x0F\x0F\x0F\x0F...' for column 'Description2' at row >1, Error_code: 1366; Incorrect string value: '\xBD\x06DINI...' for column 'Description3' >at row 1, Error_code: 1366; Corrupted replication event was detected, Error_code: 1610; >handler error HA_ERR_CORRUPT_EVENT; the event's master log apdb041-bin.005603, >end_log_pos 1070537659
currently the only way to recover is via full refresh of the slaves.
Would there be an issue on the master server? No disk / network / logging indicates a reason why the mysql master would incorrectly truncate the log.
What could be causing the master to incorrectly truncate or not close off the log properly?
I'd be happy to hear about any extra detail which might be required to help answer this question!