My first one on DBA...
After successful upgrading a master MySQL server 5.5 to 5.6.17 (CentOS 6 VPS serer), I done and moved my binary backups to a slave server (same version 5.6.17 but different environment on Jelastic). The binary backup was done using Percona Xtrabackup 2.1.7 with preparation. Both systems are x86_64 GNU/Linux
After restoring the binary backups to /var/lib/mysql and starting MySQL I was welcomed with the following error from MySQL log:
140725 15:02:11 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
2014-07-25 15:02:13 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2014-07-25 15:02:13 11854 [Note] Plugin 'FEDERATED' is disabled.
2014-07-25 15:02:13 11854 [Note] InnoDB: Using atomics to ref count buffer pool pages
2014-07-25 15:02:13 11854 [Note] InnoDB: The InnoDB memory heap is disabled
2014-07-25 15:02:13 11854 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2014-07-25 15:02:13 11854 [Note] InnoDB: Compressed tables use zlib 1.2.3
2014-07-25 15:02:13 11854 [Note] InnoDB: Using Linux native AIO
2014-07-25 15:02:13 11854 [Note] InnoDB: Using CPU crc32 instructions
2014-07-25 15:02:13 11854 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2014-07-25 15:02:13 11854 [Note] InnoDB: Completed initialization of buffer pool
2014-07-25 15:02:13 11854 [ERROR] InnoDB: Only one log file found.
2014-07-25 15:02:13 11854 [ERROR] Plugin 'InnoDB' init function returned error.
2014-07-25 15:02:13 11854 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2014-07-25 15:02:13 11854 [ERROR] Unknown/unsupported storage engine: InnoDB
2014-07-25 15:02:13 11854 [ERROR] Aborting
2014-07-25 15:02:13 11854 [Note] Binlog end
2014-07-25 15:02:13 11854 [Note] Shutting down plugin 'partition'
This is ls /var/lib/mysql
backup-my.cnf
ibdata1
ib_logfile0
xtrabackup_binary
xtrabackup_binlog_info
xtrabackup_binlog_pos_innodb
xtrabackup_checkpoints
mysql
.... OTHER DATABASE FOLDERS .....
This is very strange, and it used to work perfectly fine when I was setting up 5.5 > 5.5 replication exactly the same was.
Why is it failing with 5.6 now?
What does it mean: [ERROR] InnoDB: Only one log file found"
Additionally on the master I can see ib_logfile0 and ib_logfile1 but Xtrabackup gives only ib_logfile0
- I tried restarting the server.
- I tried doing the backup/restore prcedure twice to make sure I didn't miss something.
- I tried removing ib_logfile0 file as suggested somewhere on the official MySQL forum.
- I tried explicitly enabling InnoDB in my.cnf with
Innodb=ON
EDIT: Answering for questions:
The actual line I'm using for the backup?
ulimit -n 10240 && sudo -u mysql \
/bin/sh -c "cd /var/lib/mysql && /usr/bin/innobackupex \
--user=${MYSQL_BACKUP_USER} --password=${MYSQL_BACKUP_PASS} \
--lock-wait-query-type=update --lock-wait-timeout=300 \
--kill-long-query-type=select --kill-long-queries-timeout=10 \
--rsync --slave-info --safe-slave-backup ${XB_TARGET_DIR}"
/usr/bin/innobackupex --version
InnoDB Backup Utility v1.5.1-xtrabackup;
cat xtrabackup_binary
xtrabackup_56
cat backup-my.cnf
# This MySQL options file was generated by innobackupex.
# The MySQL server
[mysqld]
innodb_checksum_algorithm=innodb
innodb_log_checksum_algorithm=innodb
innodb_data_file_path=ibdata1:10M:autoextend
innodb_log_files_in_group=2
innodb_log_file_size=50331648
innodb_fast_checksum=0
innodb_page_size=16384
innodb_log_block_size=512
innodb_undo_tablespaces=0
Do you have the log files outside of the datadir?
No, log files are within the datadir
backup-my.cnfhave a innodb_log_files_in_group = 1? Do you have the log files outside of the datadir? All these are common mistakes.innobackupex --apply-log-that should run xtrabackup twice, the second run regenerates the (empty) logs. You can run that anywhere, but before starting your slave.