mysql data node not connecting to node manager on AWS EC2

Question

I am trying to setup mySql manager + data node + sql server (Server A) and another data node cluster on (Server B). Both running on Ubuntu 16.04 amd_64

Server A

/etc/hosts also on Server B

Both private ip
172.xx.xx.xx ip-172-xx-xx-xx-eu-west-1.compute.internal
172.xx.xx.xx ip-172-xx-xx-xx.eu-east-1.compute.internal

/config.ini

[ndbd default]
 NoOfReplicas=2  # Number of replicas

[ndb_mgmd]
hostname=ip-172-xx-xx-xx.eu-west-1.compute.internal <- Server A
datadir=/var/lib/mysql-cluster
NodeId=1

[ndbd]
hostname=ip-172-xx-xx-xx.eu-west-1.compute.internal <- Server A
NodeId=3
datadir=/usr/local/mysql/data

[ndbd]
hostname=ip-172-xx-xx-xx.eu-east-1.compute.internal <- Server B
NodeId=2
datadir=/usr/local/mysql/data

[mysqld]
hostname=ip-172-xx-xx-xx.eu-west-1.compute.internal <- Server A
/etc/my.cnf

[mysqld]
ndbcluster

[mysq_cluster]
ndb-connectstring = ip-172-xx-xx-xx.eu-west-1.compute.internal <- Server A

Server B

/etc/my.cnf

[mysqld]
ndbcluster

[mysql_cluster]
ndb-connectstring=172.xx.xx.xx <- Server A private ip

Server A error

running ndbd gives me this error

[ndbd] INFO     -- Angel connected to '172.xx.xx.xx:1186'
[ndbd] ERROR    -- Failed to allocate nodeid, error: 'Error: Could not alloc node id at 172.xx.xx.xx port 1186: Connection done from wrong host ip 172.xx.xx.xx.'

Server B error

running ndbd gives me this error

Unable to connect with connect string: nodeid=0,172.xx.xx.xx:1186
Retrying every 5 seconds. Attempts left: 12

ndb_mgm

-- NDB Cluster -- Management Client --
ndb_mgm> show
Connected to Management Server at: 172.xx.xx.xx:1186 <- A
Cluster Configuration
---------------------
[ndbd(NDB)] 1 node(s)
id=2 (not connected, accepting connect from 172.xx.xx.xx) <- B

[ndb_mgmd(MGM)] 1 node(s)
id=1    @172.x.x.x  (mysql-5.7.22 ndb-7.6.6)

[mysqld(API)]   1 node(s)
id=3 (not connected, accepting connect from 172.xx.xx.xx) <- A

Not sure how would i connect to mysqld(API) that should work fine after i installed the server/client cluster.

Also i got inbound on both Instances A & B to accept all traffic between all port ranges 1000-60000 just in case i am missing something, and outbound is open to everything.

I wonder what is it i am missing, i got so tired today trying to find out what is going on with the setup.

Any help will be appreciated !

Answer 1

From the output of ndb_mgm> show it looks like your active configuration is a single data node configuration not the two node configuration you show in config.ini.

When changing configuration one need to restart ndb_mgmd with --initial option, otherwise it will use the cached configuration on disk.

And after that restart all other nodes to make the use the new configuration.