Skip to main content
Database Administrators

Return to Question

Tweeted twitter.com/#!/StackDBAs/status/128049750506344449
formatting
Source Link
Derek Downey
Derek Downey
  • 23.5k
  • 11
  • 78
  • 104

How to handle the My SQlSQL DB Master to Master replication when there was a disk space outage

We had master to master replication with two node nodes A & B, both are in virtual environment. Initially there was an outage (diskspace issue) in node A and stopped replication. Application level traffic has been diverted to node B and relocated the node A data store and made available of space. 

Node A has started successfully & up.Coming Coming to the replication has started and got andan error in node B while syncing with node A and bin log in node bB got corrupted. What is theThe root cause is still a mystery. But analyzing the logs could found some duplicate entries and below is the error log info :

[ERROR] Error reading packet from server: Client requested master to start replication from impossible position ( server_errno=1236).[ERROR] Slave I/O: Got fatal error 1236 from master when reading data from binary log: 'Client requested master to start replication from impossible position', Error_code: 1236 111014 20:25:48 [Note] Slave I/O thread exiting, read up to log 'mysql-bin.001067', position 183468345.

[ERROR] Error reading packet from server: Client requested master to start replication from impossible position ( server_errno=1236).
[ERROR] Slave I/O: Got fatal error 1236 from master when reading data from binary log: 'Client requested master to start replication from impossible position', Error_code: 1236
111014 20:25:48 [Note] Slave I/O thread exiting, read up to log 'mysql-bin.001067', position 183468345.

How todo we handle the replication atin this situation  ? Can we skip the current bin-log and start the replication with the next available bin  - loglog and its posistionposition. isIs it a good idea to sync node B with node A  :

CHANGE MASTER TO MASTER_HOST='XX.XX.XXX.XXX', MASTER_USER='replicate', MASTER_PASSWORD='slave', MASTER_PORT=3306, MASTER_LOG_FILE='mysql-bin.001025', MASTER_LOG_POS=4, MASTER_CONNECT_RETRY=10;

CHANGE MASTER TO
  MASTER_HOST='XX.XX.XXX.XXX',
  MASTER_USER='replicate',
  MASTER_PASSWORD='slave',
  MASTER_PORT=3306,
  MASTER_LOG_FILE='mysql-bin.001025',
  MASTER_LOG_POS=4,
  MASTER_CONNECT_RETRY=10;

The main reason iI am posing this scenario is, that iI don't want to restore the DB and build it from scratch,where where DB backup is around 80 GB in node B. How can iI repair the replication.?

How to handle the My SQl DB Master to Master replication when there was a disk space outage

We had master to master replication with two node nodes A & B, both are in virtual environment. Initially there was an outage (diskspace issue) in node A and stopped replication. Application level traffic has been diverted to node B and relocated the node A data store and made available of space. Node A has started successfully & up.Coming to the replication has started and got and error in node B while syncing with node A and bin log in node b got corrupted. What is the root cause still mystery. But analyzing the logs could found some duplicate entries and below is the error log info :

[ERROR] Error reading packet from server: Client requested master to start replication from impossible position ( server_errno=1236).[ERROR] Slave I/O: Got fatal error 1236 from master when reading data from binary log: 'Client requested master to start replication from impossible position', Error_code: 1236 111014 20:25:48 [Note] Slave I/O thread exiting, read up to log 'mysql-bin.001067', position 183468345.

How to handle the replication at this situation  ? Can we skip the current bin-log and start the replication with the next available bin  - log and its posistion. is it a good idea to sync node B with node A  :

CHANGE MASTER TO MASTER_HOST='XX.XX.XXX.XXX', MASTER_USER='replicate', MASTER_PASSWORD='slave', MASTER_PORT=3306, MASTER_LOG_FILE='mysql-bin.001025', MASTER_LOG_POS=4, MASTER_CONNECT_RETRY=10;

The main reason i am posing this scenario is, that i don't want to restore the DB and build it from scratch,where DB backup is around 80 GB in node B. How can i repair the replication.

How to handle the My SQL DB Master to Master replication when there was a disk space outage

We had master to master replication with two node nodes A & B, both are in virtual environment. Initially there was an outage (diskspace issue) in node A and stopped replication. Application level traffic has been diverted to node B and relocated the node A data store and made available of space. 

Node A has started successfully & up. Coming to the replication has started and got an error in node B while syncing with node A and bin log in node B got corrupted. The root cause is still a mystery. But analyzing the logs could found some duplicate entries and below is the error log info :

[ERROR] Error reading packet from server: Client requested master to start replication from impossible position ( server_errno=1236).
[ERROR] Slave I/O: Got fatal error 1236 from master when reading data from binary log: 'Client requested master to start replication from impossible position', Error_code: 1236
111014 20:25:48 [Note] Slave I/O thread exiting, read up to log 'mysql-bin.001067', position 183468345.

How do we handle the replication in this situation? Can we skip the current bin-log and start the replication with the next available bin-log and its position. Is it a good idea to sync node B with node A:

CHANGE MASTER TO
  MASTER_HOST='XX.XX.XXX.XXX',
  MASTER_USER='replicate',
  MASTER_PASSWORD='slave',
  MASTER_PORT=3306,
  MASTER_LOG_FILE='mysql-bin.001025',
  MASTER_LOG_POS=4,
  MASTER_CONNECT_RETRY=10;

The main reason I am posing this scenario is, that I don't want to restore the DB and build it from scratch, where DB backup is around 80 GB in node B. How can I repair the replication?

Source Link
Gopinath Karangula
Gopinath Karangula
  • 933
  • 2
  • 10
  • 21

How to handle the My SQl DB Master to Master replication when there was a disk space outage

We had master to master replication with two node nodes A & B, both are in virtual environment. Initially there was an outage (diskspace issue) in node A and stopped replication. Application level traffic has been diverted to node B and relocated the node A data store and made available of space. Node A has started successfully & up.Coming to the replication has started and got and error in node B while syncing with node A and bin log in node b got corrupted. What is the root cause still mystery. But analyzing the logs could found some duplicate entries and below is the error log info :

[ERROR] Error reading packet from server: Client requested master to start replication from impossible position ( server_errno=1236).[ERROR] Slave I/O: Got fatal error 1236 from master when reading data from binary log: 'Client requested master to start replication from impossible position', Error_code: 1236 111014 20:25:48 [Note] Slave I/O thread exiting, read up to log 'mysql-bin.001067', position 183468345.

How to handle the replication at this situation ? Can we skip the current bin-log and start the replication with the next available bin - log and its posistion. is it a good idea to sync node B with node A :

CHANGE MASTER TO MASTER_HOST='XX.XX.XXX.XXX', MASTER_USER='replicate', MASTER_PASSWORD='slave', MASTER_PORT=3306, MASTER_LOG_FILE='mysql-bin.001025', MASTER_LOG_POS=4, MASTER_CONNECT_RETRY=10;

The main reason i am posing this scenario is, that i don't want to restore the DB and build it from scratch,where DB backup is around 80 GB in node B. How can i repair the replication.