Take the 2-minute tour ×
Database Administrators Stack Exchange is a question and answer site for database professionals who wish to improve their database skills and learn from others in the community. It's 100% free, no registration required.

Preamble:

We use RDS at AWS to manage our MySQL database. I know that RDS creates binary logs used for replication.. I want to harvest these binary logs and store them on S3, so that at any time, i can restore to an off-site database and replay the actions.

The Problem:

I span up a new RDS instance, and tried the following.

root@aws_test_server:/root# mysqlbinlog -h testing.xxxxxxxx.eu-west-1.rds.amazonaws.com -u myuser -p --read-from-remote-server -t mysql-bin-changelog.000002
Enter password:
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!40019 SET @@session.max_insert_delayed_threads=0*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
ERROR: Got error reading packet from server: Could not find first log file name in binary log index file
DELIMITER ;
# End of log file
ROLLBACK /* added by mysqlbinlog */;
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
root@aws_test_server:/root#

So the binary log retrieves fine, however see the Error within the binary log.. What does this mean?

I googled for about 2 days and could find no mention of this error [anywhere].

My questions:

  1. Is this a good idea? (backing up binary logs to S3..)
  2. Can it be done in a better way? (~infinite point-in-time db recovery)
  3. What can I do about the error?

PS: this is happening for the production DB and this test DB

PS2: I note that this is not directly a programming question, however is related to it. Please do not move/close from SO to somewhere like serverfault. Thanks


Update 2/4/2014:

SHOW BINARY LOGS;

mysql> show binary logs;
+----------------------------+-----------+
| Log_name                   | File_size |
+----------------------------+-----------+
| mysql-bin-changelog.000002 |      1097 |
| mysql-bin-changelog.000003 |       469 |
| mysql-bin-changelog.000004 |       469 |
| mysql-bin-changelog.000005 |       594 |
| mysql-bin-changelog.000006 |       715 |
| mysql-bin-changelog.000007 |       412 |
+----------------------------+-----------+
6 rows in set (0.00 sec)

SHOW MASTER STATUS;

mysql> show master status;
+----------------------------+----------+--------------+------------------+-------------------+
| File                       | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+----------------------------+----------+--------------+------------------+-------------------+
| mysql-bin-changelog.000007 |      412 |              |                  |                   |
+----------------------------+----------+--------------+------------------+-------------------+
1 row in set (0.01 sec)

Command: mysqlbinlog -R -h $HOST -u ttc -p -j 0 --stop-position=412 -r file

Result:

Enter password:
Warning: option 'start-position': unsigned value 0 adjusted to 4
mysqlbinlog Ver 3.3 for debian-linux-gnu at x86_64
Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Dumps a MySQL binary log in a format usable for viewing or for piping to
the mysql command line client.

Usage: mysqlbinlog [options] log-files
  -?, --help          Display this help and exit.
  --base64-output[=name]
                      Determine when the output statements should be
                      base64-encoded BINLOG statements: 'never' disables it and
                      works only for binlogs without row-based events;
                      'decode-rows' decodes row events into commented SQL
                      statements if the --verbose option is also given; 'auto'
                      prints base64 only when necessary (i.e., for row-based
                      events and format description events); 'always' prints
                      base64 whenever possible. 'always' is deprecated, will be
                      removed in a future version, and should not be used in a
                      production system.  --base64-output with no 'name'
                      argument is equivalent to --base64-output=always and is
                      also deprecated.  If no --base64-output[=name] option is
                      given at all, the default is 'auto'.
  --character-sets-dir=name
                      Directory for character set files.
  -d, --database=name List entries for just this database (local log only).
  --debug-check       Check memory and open file usage at exit .
  --debug-info        Print some debug info at exit.
  --default-auth=name Default authentication client-side plugin to use.
  -D, --disable-log-bin
                      Disable binary log. This is useful, if you enabled
                      --to-last-log and are sending the output to the same
                      MySQL server. This way you could avoid an endless loop.
                      You would also like to use it when restoring after a
                      crash to avoid duplication of the statements you already
                      have. NOTE: you will need a SUPER privilege to use this
                      option.
  -F, --force-if-open Force if binlog was not closed properly.
                      (Defaults to on; use --skip-force-if-open to disable.)
  -f, --force-read    Force reading unknown binlog events.
  -H, --hexdump       Augment output with hexadecimal and ASCII event dump.
  -h, --host=name     Get the binlog from server.
  -l, --local-load=name
                      Prepare local temporary files for LOAD DATA INFILE in the
                      specified directory.
  -o, --offset=#      Skip the first N entries.
  -p, --password[=name]
                      Password to connect to remote server.
  --plugin-dir=name   Directory for client-side plugins.
  -P, --port=#        Port number to use for connection or 0 for default to, in
                      order of preference, my.cnf, $MYSQL_TCP_PORT,
                      /etc/services, built-in default (3306).
  --protocol=name     The protocol to use for connection (tcp, socket, pipe,
                      memory).
  -R, --read-from-remote-server
                      Read binary logs from a MySQL server.
  -r, --result-file=name
                      Direct output to a given file.
  --server-id=#       Extract only binlog entries created by the server having
                      the given id.
  --set-charset=name  Add 'SET NAMES character_set' to the output.
  -s, --short-form    Just show regular queries: no extra info and no row-based
                      events. This is for testing only, and should not be used
                      in production systems. If you want to suppress
                      base64-output, consider using --base64-output=never
                      instead.
  -S, --socket=name   The socket file to use for connection.
  --start-datetime=name
                      Start reading the binlog at first event having a datetime
                      equal or posterior to the argument; the argument must be
                      a date and time in the local time zone, in any format
                      accepted by the MySQL server for DATETIME and TIMESTAMP
                      types, for example: 2004-12-25 11:25:56 (you should
                      probably use quotes for your shell to set it properly).
  -j, --start-position=#
                      Start reading the binlog at position N. Applies to the
                      first binlog passed on the command line.
  --stop-datetime=name
                      Stop reading the binlog at first event having a datetime
                      equal or posterior to the argument; the argument must be
                      a date and time in the local time zone, in any format
                      accepted by the MySQL server for DATETIME and TIMESTAMP
                      types, for example: 2004-12-25 11:25:56 (you should
                      probably use quotes for your shell to set it properly).
  --stop-position=#   Stop reading the binlog at position N. Applies to the
                      last binlog passed on the command line.
  -t, --to-last-log   Requires -R. Will not stop at the end of the requested
                      binlog but rather continue printing until the end of the
                      last binlog of the MySQL server. If you send the output
                      to the same MySQL server, that may lead to an endless
                      loop.
  -u, --user=name     Connect to the remote server as username.
  -v, --verbose       Reconstruct SQL statements out of row events. -v -v adds
                      comments on column data types.
  -V, --version       Print version and exit.
  --open-files-limit=#
                      Used to reserve file descriptors for use by this program.

Variables (--variable-name=value)
and boolean options {FALSE|TRUE}  Value (after reading options)
--------------------------------- ----------------------------------------
base64-output                     (No default value)
character-sets-dir                (No default value)
database                          (No default value)
debug-check                       FALSE
debug-info                        FALSE
default-auth                      (No default value)
disable-log-bin                   FALSE
force-if-open                     TRUE
force-read                        FALSE
hexdump                           FALSE
host                              ttc-testing.cz6c92dfxwoh.eu-west-1.rds.amazonaws.com
local-load                        (No default value)
offset                            0
plugin-dir                        (No default value)
port                              3306
read-from-remote-server           TRUE
server-id                         0
set-charset                       (No default value)
short-form                        FALSE
socket                            /var/run/mysqld/mysqld.sock
start-datetime                    (No default value)
start-position                    4
stop-datetime                     (No default value)
stop-position                     412
to-last-log                       FALSE
user                              ttc
open-files-limit                  64
share|improve this question
    
Please don't post the same question here and on SO. stackoverflow.com/questions/22797278 –  Michael - sqlbot Apr 2 at 1:04

1 Answer 1

up vote 1 down vote accepted

You are mistaken in your assertion that "the binary log retrieves fine." The output you are seeing is boilerplate output from mysqlbinlog... there's no evidence here that you are actually fetching the binlog. There's no such log file on the server. That's the meaning of this error.

SHOW BINARY LOGS;

...to find the available logs on the server.

You can confirm that you are not in fact actually fetching a file, by passing a completely bogus binlog file name and I suspect the output will be the same.

Backing up binlogs is an excellent plan, but you should look into the --stop-never option that, if I recall correctly, was introduced in the version of mysqlbinlog that shipped with MySQL Server 5.6 (backwards compatible with all 5.x servers) and maintains a connection to the server and fetches the files, saving each one under its own name.

share|improve this answer
    
Hmm, my mysqlbinlog states mysqlbinlog: unknown option '--stop-never'. Please see updated question for more details –  matejkramny Apr 2 at 9:04
    
When specifying the correct log file, it now reports ERROR: Got error reading packet from server: Slave can not handle replication events with the checksum that master is configured to log; the first event 'mysql-bin-changelog.000889' at 4, the last event read from '/rdsdbdata/log/binlog/mysql-bin-changelog.000889' at 120, the last byte read from '/rdsdbdata/log/binlog/mysql-bin-changelog.000889' at 120. –  matejkramny Apr 2 at 9:23
    
You need to download the correct version of mysqlbinlog. It looks like you are using a version that ships with MySQL 5.5, not 5.6. dev.mysql.com/doc/refman/5.6/en/mysqlbinlog.html –  Michael - sqlbot Apr 2 at 10:42
    
thanks, thats the problem :(. Thanks for help! –  matejkramny Apr 2 at 11:16
    
If you can update your answer, i'll mark it as the accepted answer.. Thanks –  matejkramny Apr 2 at 11:16

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.