0

OS: CentOS 7

MariaDB: mariadb Ver 15.1 Distrib 10.4.12-MariaDB, for Linux (x86_64) using readline 5.1


I am creating a full backup every 7 days on a replication slave, and an incremental backup every hour on top of that.

This script is used to create backups: run-mariabackup.sh

To restore the backup I am using this process.

However, after restoring the backup MariaDB does not start.

systemd: Starting MariaDB 10.4.12 database server...
mysqld: [Note] /usr/sbin/mysqld (mysqld 10.4.12-MariaDB) starting as process 11363 ...
mysqld: [ERROR] mysqld: File '/var/lib/mysql/aria_log_control' not found (Errcode: 13 "Permission denied")
mysqld: [ERROR] mysqld: Got error 'Can't open file' when trying to use aria control file '/var/lib/mysql/aria_log_control'
mysqld: [ERROR] Plugin 'Aria' init function returned error.
mysqld: [ERROR] Plugin 'Aria' registration as a STORAGE ENGINE failed.
mysqld: [Note] InnoDB: Using Linux native AIO
mysqld: [ERROR] InnoDB: Operating system error number 13 in a file operation.
mysqld: [ERROR] InnoDB: The error means mysqld does not have the access rights to the directory.
mysqld: [ERROR] InnoDB: os_file_get_status() failed on './ibdata1'. Can't determine file permissions
mysqld: [ERROR] Plugin 'InnoDB' init function returned error.
mysqld: [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
mysqld: [Note] Plugin 'FEEDBACK' is disabled.
mysqld: [ERROR] Could not open mysql.plugin table. Some plugins may be not loaded
mysqld: [ERROR] Failed to initialize plugins.
mysqld: [ERROR] Aborting
systemd: mariadb.service: main process exited, code=exited, status=1/FAILURE

Of course I have already done chown -R mysql:mysql /var/lib/mysql prior to that, so this error puzzled me. After this error I tried chmod -R 777 /var/lib/mysql to see if there is some permission that was missing, but starting MariaDB again provided me with the same error.

Here are the exact steps I took to restore the backup:

cd /home/mysqlbackup/
for i in $(find . -name backup.stream.gz | grep '2020-03-12_12-08-44' | xargs dirname); do
        mkdir -p $i/backup;
        zcat $i/backup.stream.gz | mbstream -x -C $i/backup/;
done
mariabackup --prepare --target-dir base/2020-03-12_12-08-44/backup/ --user backup --password mypass
mariabackup --prepare --target-dir base/2020-03-12_12-08-44/backup/ --user backup --password mypass --incremental-dir incr/2020-03-12_12-08-44/2020-03-12_13-24-20/backup/
mariabackup --prepare --target-dir base/2020-03-12_12-08-44/backup/ --user backup --password mypass --incremental-dir incr/2020-03-12_12-08-44/2020-03-12_13-54-25/backup/
mariabackup --prepare --target-dir base/2020-03-12_12-08-44/backup/ --user backup --password mypass --incremental-dir incr/2020-03-12_12-08-44/2020-03-12_14-00-02/backup/
mariabackup --prepare --target-dir base/2020-03-12_12-08-44/backup/ --user backup --password mypass --incremental-dir incr/2020-03-12_12-08-44/2020-03-12_15-00-01/backup/
mariabackup --prepare --target-dir base/2020-03-12_12-08-44/backup/ --user backup --password mypass --incremental-dir incr/2020-03-12_12-08-44/2020-03-12_16-00-01/backup/
systemctl stop mariadb
mv /var/lib/mysql/ /var/lib/mysql_bak/
mariabackup --copy-back --target-dir base/2020-03-12_12-08-44/backup/ --user backup --password mypass --datadir /var/lib/mysql/
chown -R mysql:mysql /var/lib/mysql/
systemctl start mariadb
4
  • Is SELinux enabled and running in enforcing mode? (sestatus should reveal the answer to this.)
    – dbdemon
    Mar 12, 2020 at 15:27
  • @dbdemon Yes it is enforcing Mar 12, 2020 at 15:29
  • Your hunch was right, doing setenforce 0 fixed the problem. But how can I fix it without disabling selinux? Also, after starting the server it is no longer replicating SHOW SLAVE STATUS\G Empty set (0.000 sec) Mar 12, 2020 at 15:32
  • Now that I think about it I guess it wouldn't even make sense for the server to start replicating again, since you would restore the backup on the master anyway. Mar 12, 2020 at 15:44

1 Answer 1

2

When the permissions look correct, but you're still getting Errcode: 13 "Permission denied", then the next thing to check is whether a security system is causing the problem. On CentOS that system is SELinux. To check if it's running in enforcing mode, execute this:

sestatus

You can do setenforce 0 to temporarily change it to permissive mode.

To avoid the problem:

When you initially install MariaDB, the SELinux security context should be set correctly for all the files. However, when you do mv /var/lib/mysql/ /var/lib/mysql_bak/, then you remove it. I think you can get it back by simply executing restorecon -Rv /var/lib/mysql.

For reference, read this page in the MariaDB Knowledgebase: Security-Enhanced Linux with MariaDB

2
  • I should add that I don't know if this will work for any new files that came from the backup and weren't in this datadir originally.
    – dbdemon
    Mar 12, 2020 at 15:53
  • restorecon -Rv /var/lib/mysql fixed my problem without having to disable selinux. Thank you. Mar 12, 2020 at 16:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.