-1

I am trying to get mysql-cluster going on Ubuntu 22 from the official MySQL apt repo.

I keep running into complications.

Certain ndb-related daemons are required ... like ndbd. But there are no service files for these daemons! If I have to create these service files, that is NUTS. If things like commandline options change on a mysql package upgrade, those service files will not be updated. There's also a good chance that I might create the service file incorrectly.

Are there complete packages and instructions available anywhere for setting this up? I've found some things with google, but none of them cover creating service files in systemd for the ndb daemons.

11
  • I finally found a page that had the service files. Created those. Now mysql won't start. Configuration error: Error: Could not alloc node id at gandalf.elyograg.org:1186: Connection done from wrong host ip 192.168.217.202. That IS the correct IP address for this host.
    – elyograg
    Commented May 21, 2023 at 2:57
  • The existing database server is smeagol.elyograg.org at 192.168.217.200. I want to add a third node to the cluster that will NOT have the data, as a tiebreaker. I am beginning on gandalf.elyograg.org at 192.168.217.202. After I get it running woth all data, I will add sauron, and then add the data. Finally I will bring smeagol into the cluster.
    – elyograg
    Commented May 21, 2023 at 2:59
  • This is VERY frustrating! I have ndb_mgmd and ndbd running. But mysql will NOT start. ``` 2023-05-20 21:59:37 [NdbApi] INFO -- Configuration error: Error: Could not alloc node id at gandalf.elyograg.org:1186: Connection done from wrong host ip 192.168.217.202. ```
    – elyograg
    Commented May 21, 2023 at 4:00
  • Also, the ndb cluster log says this: ``` 2023-05-20 22:01:11 [MgmtSrvr] WARNING -- Unable to allocate nodeid for API at 192.168.217.202. Returned error: 'Connection done from wrong host ip 192.168.217.202.' 2023-05-20 22:01:13 [MgmtSrvr] INFO -- Node 2: Initial start, waiting for 3 to connect, nodes [ all: 2 and 3 connected: 2 no-wait: ] ``` Which is highly frustrating as there is no node 3 in config.ini! It ONLY contains info for this one single host, and that ndbd config is node 2.
    – elyograg
    Commented May 21, 2023 at 4:03
  • I ran into a problem where the ndb daemons would not start if ipv6 is disabled systemwide. So I had to re-enable a protocol that I am not using.
    – elyograg
    Commented May 21, 2023 at 4:04

3 Answers 3

0

The complete packages are available on the apt repo and instructions for installing are Installing MySQL NDB Cluster Using the APT Repository and NDB Cluster Installation.

Still, no service files for systemd are provided for management server and data nodes. The systemd model does not map nicely to MySQL Cluster.

Below follow an example of configuration and service files that should work for many cases, but not all.

One limitation is the clusters must have exactly one management server, and at most one data node per host.

If there are several management servers one will need to distinguish between restarting management server with or without --reload. And one can not use ExecReload in an easy way for the reload configuration cases since management server needs to be restarted also in that case.

Systemd Service Files Example

Note, using these systemd service files is not the recommended way to manage cluster since there are several short comings with them.

Example uses paths as they appear on Ubuntu 22.10 using the MySQL APT repo.

The example depends on these extra directories to be created

# mkdir /etc/mysql/cluster
# mkdir -m 700 /var/lib/mysql-cluster
# chown mysql:mysql /var/lib/mysql-cluster

And these configuration files to be added

/etc/mysql/conf.d/cluster.cnf:

[mysql_cluster]
ndb-connectstring=<hostname-for-managementserver>

[mysqld]
ndbcluster

[ndb_mgmd]
config-file=/etc/mysql/cluster/config.ini
config-dir=/var/lib/mysql-cluster

/etc/mysql/cluster/config.ini (the sections also has alias names [MGM] same as [ndb_mgmd], [DB] same as [ndbd], [API] same as [mysqld]):

[MGM]
DataDir=/var/lib/mysql-cluster
HostName=<hostname-for-managentserver>

[DB default]
DataDir=/var/lib/mysql-cluster
NoOfReplicas=3
LockPagesInMainMemory=1
DataMemory=100M

[DB]
HostName=<host#1-for-data-node>

[DB]
HostName=<host#2-for-data-node>

[DB]
HostName=<host#3-for-data-node>

[API]
HostName=<host#1-for-mysql-server>

[API]
HostName=<host#2-for-mysql-server>

[API]
HostName=<host#3-for-mysql-server>

[API]
# Empty section to be able to run other ndb client programs from anywhere

Service file for management server

/lib/systemd/system/ndb_mgmd.service:

[Unit]
Description=MySQL Cluster Management Node
Documentation=man:ndb_mgmd(8)
After=network-online.target
Wants=network-online.target

[Install]
WantedBy=multi-user.target

[Service]
User=mysql
Group=mysql
WorkingDirectory=/var/lib/mysql-cluster
# Starting with --reload forces the config-file to be reread, but it works if
# there is only one managment server in the cluster.
# If there are more management servers one need to ensure that at most one
# management server at restarts with --reload.
ExecStart=/usr/sbin/ndb_mgmd --nodaemon --reload
Restart=on-failure

Service file for data node

/lib/systemd/system/ndbd.service:

[Unit]
Description=MySQL Cluster Data Node
Documentation=man:ndbmtd(8)
After=network-online.target
Wants=network-online.target

[Install]
WantedBy=multi-user.target

[Service]
User=mysql
Group=mysql
WorkingDirectory=/var/lib/mysql-cluster
CapabilityBoundingSet=CAP_IPC_LOCK CAP_SYS_NICE
ExecStart=/usr/sbin/ndbmtd --nodaemon
Restart=on-failure

System Start

When bringing up the cluster, so called system (re)start, one first starts the management server

> systemctl start ndb_mgmd

and then all configured data nodes must be started, on each host with data nodes do:

> systemctl start ndbd

and then start all your MySQL servers:

> systemctl start mysql

Configuration Change

After updating the cluster configuration file, config.ini, one first need to restart the management server with --reload

> systemctl restart ndb_mgmd

And then restart all data nodes and MySQL server such that they are aware of the new configuration. For data nodes one can use ndb_mgm for restarts.

> ndb_mgm
ndb_mgm> 2 RESTART
Node 2: Node shutdown initiated
Node 2: Node shutdown completed, restarting, no start.
Node 2 is being restarted
Node 2: Start initiated (version 8.0.33)
Node 2: Started (version 8.0.33)
ndb_mgm> 3 RESTART
...
ndb_mgm> 4 RESTART
...

For configuration changes that needs initial node restarts one need to use 'RESTART -i'.

Configuration changes that need initial system restart will be trickier. That can not be handled using ndb_mgm but one need to stop all data nodes and then start them with an extra --initial option to the start command.

Systemd do not have alternative start command, and the ndbd service does not and should not include --initial in the ExecStart command. Instead one need to manually remove all the data from disk

# rm -rf /var/lib/mysql-cluster/ndb_<number>_fs/

And then start service as usual

# systemctl start ndbd

Warning: ExecReload=kill -HUP does not work!

I searched internet for systemd service files for Ndb, and I found several all suggesting using kill -HUP for ExecReload to reload configuration - that does not work and have never worked, ndb_mgmd ignores HUP signal. Also for data nodes it is meaningless.

For ndb_mgmd to reread config.ini it needs to be restarted with --reload command line option (except for the first time when there is no previous configuration read).

Comment on Comments on Question

The below is not really part of the answer but addresses some of the OP comments on question.

Comment#1

I want to add a third node to the cluster that will NOT have the data, as a tiebreaker.

If this about already having two data nodes in a nodegroup that have data, and then later add a third data node to the same nodegroup, that is not supported online by cluster.

All nodes in a nodegroup always must have the same data.

Changing the number of replicas requires so called initial system restart, start with an empty cluster and restore data from backup.

Adding nodes when adding a nodegroup can be done online, in this case the new nodegroup(s) will not have the same data as the original nodegroup(s), for this see Adding NDB Cluster Data Nodes Online

Comment#2

Which is highly frustrating as there is no node 3 in config.ini! It ONLY contains info for this one single host, and that ndbd config is node 2.

This sounds that the management server has not reread config.ini, make sure that you restart management server using ndb_mgmd ... --reload option after changing config.ini.

Running on a different configuration than expected will bring much confusion.

0

The git tree logicalclocks/ndb-chef contains a number of scripts to use in connection with systemd for RonDB (fork of NDB Cluster), most of them will work also with NDB Cluster.

1
  • The fix of the IPv6 issue is solved in the latest RonDB 21.04 version. A fix for it is in the works for NDB Cluster as well. Commented Jun 29, 2023 at 17:19
-3

I was able to figure out how to convert my database install to a multi-primary InnoDB Cluster. High availability achieved! A combination of the following URL, other documentation, and ChatGPT:

https://dev.mysql.com/blog-archive/mysql-innodb-cluster-8-0-a-hands-on-tutorial/

1
  • Unfortunately, the Zabbix database is incompatible with InnoDB Cluster. That database makes extensive use of foreign keys that cascade deletes. Which is not supported with InnoDB cluster.
    – elyograg
    Commented May 23, 2023 at 1:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.