Take the 2-minute tour ×
Database Administrators Stack Exchange is a question and answer site for database professionals who wish to improve their database skills and learn from others in the community. It's 100% free, no registration required.

My question is: how to fix broken mysql replication on slave server (error 1236)?

server 1 (192.168.1.105) server 2 (192.168.1.106)

I ran these steps to re-sync:

At the master - server 1 (192.168.1.105):

mysql -u root -p
  1. RESET MASTER;
  2. FLUSH TABLES WITH READ LOCK;
  3. SHOW MASTER STATUS;
  4. Note > File: mysql-bin.000001 and Position: 107
  5. mysqldump -uroot -p --events --ignore-table=mysql.events --all-databases > /var/backups/mysqldump.sql
  6. UNLOCK TABLES;
  7. Copy mysql dump file to server 2
    (scp -p /var/backups/mysqldump.sql root@192.168.1.106:/tmp)

At the slave server 1 (192.168.1.106):

  1. STOP SLAVE;
  2. mysql -uroot -p < /tmp/mysqldump.sql
  3. RESET SLAVE;
  4. CHANGE MASTER TO MASTER_HOST='192.168.1.105', MASTER_USER='slaveuser', MASTER_PASSWORD='mypassword';
  5. CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000001', MASTER_LOG_POS=107;
  6. START SLAVE;

I still got this error on server 1

Last_IO_Errno: 1236 Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'Could not find first log file name in binary log index file'

share|improve this question

migrated from stackoverflow.com Jul 12 '13 at 9:21

This question came from our site for professional and enthusiast programmers.

4 Answers 4

I believe the log file and position need to be specified with the host.

Try:

CHANGE MASTER TO MASTER_HOST='192.168.1.105', MASTER_USER='slaveuser', MASTER_PASSWORD='mypassword', MASTER_LOG_FILE='mysql-bin.000001', MASTER_LOG_POS=107;
share|improve this answer
    
This worked! Thank You myninjaname! –  onastvar Jul 14 '13 at 5:39
    
It worked, but now I see error 1050: Last_SQL_Error: Error 'Table 'attempts_login' already exists' on query. Default database: 'dbispconfig2'. Query: 'CREATE TABLE attempts_login ( ip varchar(39) NOT NULL, times int(11) DEFAULT NULL, login_time timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP) ENGINE=MyISAM DEFAULT CHARSET=utf8' How to fix this? –  onastvar Jul 15 '13 at 6:15
    
Your slave and master server's databases are out of sync. I'd start by creating another fresh copy on the slave server, basically: Lock and mysqldump the databases (on the master). Write down the MASTER_LOG_POS, then unlock the database. Restore the mysqldump file on the slave. Then perform the CHANGE MASTER TO with the new MASTER_LOG_POS. –  myninjaname Jul 23 '13 at 3:33

If restoring the database on the slave takes too long, the log file mysql-bin.000001 may be purged from the master by the time you START SLAVE. When you get the error on the slave, run SHOW BINARY LOGS on the master and see if mysql-bin.000001 is gone.

You should also look at the setting for expire_logs_days to make sure the binlog files aren't being expired too quickly.

Another thing you could do to compensate is the following, on the slave, as soon as you UNLOCK TABLES on the master:

  1. STOP SLAVE;
  2. RESET SLAVE;
  3. CHANGE MASTER TO MASTER_HOST='192.168.1.105', MASTER_USER='slaveuser', MASTER_PASSWORD='mypassword';
  4. CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000001', MASTER_LOG_POS=107;
  5. START SLAVE IO_THREAD;
  6. copy mysql dump from master
  7. mysql -uroot -p < /tmp/mysqldump.sql
  8. START SLAVE SQL_THREAD;

That should start the downloading of binary logs while you're doing the restore, so you don't miss out on any.

share|improve this answer
    
I tried these steps, same issue, appreciate your time. Thanks Bill! –  onastvar Jul 14 '13 at 5:40

If you need to do this with minimal downtime and have 100% confidence that you have consistent slave here are step by step direction how to rebuild the slave using Linux LVM.

Preparation

Set innodb max dirty pages percent to zero on the master. This will force MySQL to write all the pages to the disk which will significantly speed up the restart.

set global innodb_max_dirty_pages_pct = 0;

To monitor the number of dirty pages run the command

mysqladmin ext -i10 | grep dirty

Once the number stop decreasing you have reach the point to continue. Next reset the master to clear the old bin logs / relay logs:

RESET MASTER;

Rebuilding the slave using LVM

Execute lvdisplay to get LV Path

lvdisplay

Output will look like this

--- Logical volume ---
LV Path                /dev/vg_mysql/lv_data
LV Name                lv_data
VG Name                vg_mysql

Shutdown the master database with command

service mysql stop

Next take a snaphot, mysql_snapshot will be the new logical volume name.

lvcreate --size 10G --snapshot --name mysql_snapshot /dev/vg_mysql/lv_data

Start master again with command

service mysql start

Restore dirty pages setting to the default

set global innodb_max_dirty_pages_pct = 75;

Run lvdisplay again to make sure the snapshot is there and visible

lvdisplay

Output:

--- Logical volume ---
LV Path                /dev/vg_mysql/mysql_snapshot
LV Name                mysql_snapshot
VG Name                vg_mysql

Mount the snapshot

mkdir /mnt/mysql_snapshot
mount /dev/vg_mysql/mysql_snapshot /mnt/mysql_snapshot

If you have an existing MySQL slave running you need to stop it

service mysql stop

Next you need to clear MySQL data folder

cd /var/lib/mysql
rm -fr *

Back to master. Now rsync the snapshot to the MySQL slave

rsync --progress -harz /mnt/mysql_snapshot/ targethostname:/var/lib/mysql/

Once rsync has completed you may unmount and remove the snapshot

umount /mnt/mysql_snapshot
lvremove -f /dev/vg_mysql/mysql_snapshot

Create replication user on the master if the old replication user doesn't exist or password is unknown

GRANT REPLICATION SLAVE on *.* to 'replication'@'[SLAVE IP]' identified by 'YourPass';

Verify that /var/lib/mysql data files are owned by the mysql user, if so you can omit the following command:

chown -R mysql:mysql /var/lib/mysql

Next record the binlog position

ls -laF | grep mysql-bin

You will see something like

..
-rw-rw----     1 mysql mysql  1073750329 Aug 28 03:33 mysql-bin.000017
-rw-rw----     1 mysql mysql  1073741932 Aug 28 08:32 mysql-bin.000018
-rw-rw----     1 mysql mysql   963333441 Aug 28 15:37 mysql-bin.000019
-rw-rw----     1 mysql mysql    65657162 Aug 28 16:44 mysql-bin.000020

Here the master log file is the highest file number in sequence and bin log position is the file size. Record these values:

master_log_file=mysql-bin.000020
master_log_post=65657162

Next start the slave MySQL

service mysql start

Execute change master command on the slave by executing the following:

CHANGE MASTER TO 
master_host="10.0.0.12", 
master_user="replication", 
master_password="YourPass", 
master_log_file="mysql-bin.000020", 
master_log_pos=65657162; 

Finally start the slave

SLAVE START;

Check slave status:

SHOW SLAVE STATUS\G

Make sure Slave IO is running and there are no connection errors. Good luck!

My blog has some additional info on this subject but the core story is the same!

http://www.juhavehnia.com/2015/05/rebuilding-mysql-slave-using-linux-lvm.html

share|improve this answer
    
Please add some explanation of the link you posted. That way the post can still be helpful even after the link disappears. –  RLF May 21 at 0:49

Fix For GTID base replication (5.6) ERROR: 'Got fatal error 1236

mysql 5.6 GTID replication 'Got fatal error 1236 from master when reading data from binary log'

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.