After carefully checking again, I've found transaction_registry.ibd on the file system. So I manually deleted the file after stopping mariadb. Here you can see what I did:
sudo systemctl stop mariadb
sudo rm /var/lib/mysql/mysql/transaction_registry.ibd
sudo systemctl startb mariadb
sudo mysql_upgrade -u root --force -pxxxxx
...