0

I upgraded mariadb on a Centos 7 box from 5.5 to 10.5 and after updating the tables, mariadb will no longer start. In /var/log/messages, I'm getting the following errors. Any help would be much appreciated!

Mar 23 10:22:20 theo systemd: Starting MariaDB 10.5.19 database server...
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [Note] Starting MariaDB 10.5.19-MariaDB source revision f8a85af8ca1c937b8d4f847477bd282f80251cde as process 15269
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [Note] InnoDB: Uses event mutexes
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [Note] InnoDB: Compressed tables use zlib 1.2.7
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [Note] InnoDB: Number of pools: 1
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [Note] InnoDB: Using Linux native AIO
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [Note] InnoDB: Completed initialization of buffer pool
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [Note] InnoDB: 128 rollback segments are active.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: Operating system error number 13 in a file operation.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: The error means mysqld does not have the access rights to the directory.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: Cannot open datafile for read-only: './mysql/gtid_slave_pos.ibd' OS error: 81
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: Operating system error number 13 in a file operation.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: The error means mysqld does not have the access rights to the directory.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: Could not find a valid tablespace file for ``mysql`.`gtid_slave_pos``. Please refer to https://mariadb.com/kb/en/innodb-data-dictionary-troubleshooting/ for how to resolve the issue.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [Warning] InnoDB: Ignoring tablespace for `mysql`.`gtid_slave_pos` because it could not be opened.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: Operating system error number 13 in a file operation.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: The error means mysqld does not have the access rights to the directory.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: Cannot open datafile for read-only: './mysql/innodb_index_stats.ibd' OS error: 81
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: Operating system error number 13 in a file operation.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: The error means mysqld does not have the access rights to the directory.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: Could not find a valid tablespace file for ``mysql`.`innodb_index_stats``. Please refer to https://mariadb.com/kb/en/innodb-data-dictionary-troubleshooting/ for how to resolve the issue.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [Warning] InnoDB: Ignoring tablespace for `mysql`.`innodb_index_stats` because it could not be opened.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: Operating system error number 13 in a file operation.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: The error means mysqld does not have the access rights to the directory.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: Cannot open datafile for read-only: './mysql/innodb_table_stats.ibd' OS error: 81
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: Operating system error number 13 in a file operation.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: The error means mysqld does not have the access rights to the directory.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: Could not find a valid tablespace file for ``mysql`.`innodb_table_stats``. Please refer to https://mariadb.com/kb/en/innodb-data-dictionary-troubleshooting/ for how to resolve the issue.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [Warning] InnoDB: Ignoring tablespace for `mysql`.`innodb_table_stats` because it could not be opened.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: Operating system error number 13 in a file operation.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: The error means mysqld does not have the access rights to the directory.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: Cannot open datafile for read-only: './mysql/transaction_registry.ibd' OS error: 81
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: Operating system error number 13 in a file operation.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: The error means mysqld does not have the access rights to the directory.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] InnoDB: Could not find a valid tablespace file for ``mysql`.`transaction_registry``. Please refer to https://mariadb.com/kb/en/innodb-data-dictionary-troubleshooting/ for how to resolve the issue.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [Warning] InnoDB: Ignoring tablespace for `mysql`.`transaction_registry` because it could not be opened.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [Note] InnoDB: Creating shared tablespace for temporary tables
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [Note] InnoDB: 10.5.19 started; log sequence number 63339738232; transaction id 664540336
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [Note] Plugin 'FEEDBACK' is disabled.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] Could not open mysql.plugin table: "Table 'mysql.plugin' doesn't exist". Some plugins may be not loaded
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [Note] InnoDB: Buffer pool(s) load completed at 230323 10:22:20
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] Can't open and lock privilege tables: Table 'mysql.servers' doesn't exist
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [Note] Server socket created on IP: '::'.
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] Fatal error: Can't open and lock privilege tables: Table 'mysql.db' doesn't exist
Mar 23 10:22:20 theo mariadbd: 2023-03-23 10:22:20 0 [ERROR] Aborting
Mar 23 10:22:20 theo systemd: mariadb.service: main process exited, code=exited, status=1/FAILURE
Mar 23 10:22:20 theo systemd: Failed to start MariaDB 10.5.19 database server.
Mar 23 10:22:20 theo systemd: Unit mariadb.service entered failed state.
Mar 23 10:22:20 theo systemd: mariadb.service failed.
Improve this question
1
  • It is telling you that it doesnt have access rights. Please post what you have done to rectify that.
    – Rohit Gupta
    Mar 23 at 19:29

1 Answer 1

Reset to default
1

and after updating the tables, mariadb will no longer start

It sounds as if you may have run the update with the wrong user or otherwise somehow made the data directory unreadable and/or unwritable for the relevant system user (default system user is mysql). Perhaps you have lost the SELinux context for (some of) the files?

You can probably fix this by running a couple of commands to recursively set the ownership, permissions and SELinux context:

sudo chown -R mysql:mysql /var/lib/mysql
sudo find /var/lib/mysql -type d -exec chmod u=rwx,go= {} \;
sudo find /var/lib/mysql -type f -exec chmod ug=rw,o= {} \;
sudo semanage fcontext -a -t mysqld_db_t "/var/lib/mysql(/.*)?"
sudo restorecon -Rv /var/lib/mysql

The above assumes your datadir has the default location /var/lib/mysql/. (You do seem to be using this, judging by the error log output in your question.)

Please refer to the SELinux article in the MariaDB KB documentation for more details about SELinux.

Improve this answer
3
  • perror 13 --> OS error code 13: Permission denied
    – Rick James
    Apr 5 at 20:18
  • @RickJames Yes - and I believe that can be the result of wrong ownership, wrong permissions and/or wrong / missing SELinux context.
    – dbdemon
    Apr 6 at 7:46
  • Yes, I believe I did the update as the root user. It was not obvious since all the folders had the correct owner. When I did chown -R mysql:mysql /var/lib/mysql it worked again.
    – zoof
    Apr 9 at 15:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.