1

I'm running MySQL Cluster 7.4.6 on Azure VM's with the following infrastructure:

2 x Mgmt Nodes: Ubuntu 14.10LTS, 2 cores, 3.5GB ram 2 x MySQL API Nodes: Ubuntu 14.10LTS, 2 cores, 3.5GB ram 2 x Data Nodes: Ubuntu 14.10LTS, 4 cores, 7GB ram

Config.ini

[TCP DEFAULT]
SendBufferMemory=256K
ReceiveBufferMemory=256K

[ndbd default]
# Options affecting ndbd processes on all data nodes:
ServerPort=14555
NoOfReplicas=2    # Number of replicas
DataMemory=3G
SharedGlobalMemory=256M
DiskPageBufferMemory=1G
IndexMemory=1G
LockPagesInMainMemory=0

MaxNoOfConcurrentOperations=50000
MaxNoOfTriggers=4096

#StringMemory=25
MaxNoOfTables=512
MaxNoOfOrderedIndexes=256
MaxNoOfUniqueHashIndexes=256
MaxNoOfAttributes=5000
#DiskCheckpointSpeedInRestart=100M
#FragmentLogFileSize=256M
#NoOfFragmentLogFiles=3
#RedoBuffer=8M

#HeartbeatIntervalDbDb=15000
#HeartbeatIntervalDbApi=15000
#TimeBetweenLocalCheckpoints=20
#TimeBetweenGlobalCheckpoints=1000
#TimeBetweenEpochs=100

#TimeBetweenEpochsTimeout=32000

#MemReportFrequency=30
#BackupReportFrequency=30

[ndb_mgmd default]
datadir=/var/lib/mysql-cluster  # Directory for MGM node log files

[ndb_mgmd]
nodeid=1
hostname=192.168.2.4            # Hostname or IP address of MGM node

[ndb_mgmd]
nodeid=2
hostname=192.168.2.9            # Hostname or IP address of MGM node

[ndbd]
                                # (one [ndbd] section per data node)
hostname=192.168.2.5            # Hostname or IP address
datadir=/usr/local/mysql/data   # Directory for this data node's data files

[ndbd]
hostname=192.168.2.7            # Hostname or IP address
datadir=/usr/local/mysql/data   # Directory for this data node's data files

[mysqld]
hostname=192.168.2.6            # Hostname or IP address
                                # (additional mysqld connections can be
                                # specified for this node for various
                                # purposes such as running ndb_restore)

[mysqld]
hostname=192.168.2.8            # Hostname or IP address
                                # (additional mysqld connections can be
                                # specified for this node for various
                                # purposes such as running ndb_restore)

Disk space varies, but currently using about 11GB/30GB across each machine.

I have a table as below with ~15m records. Nothing special, just a join table essentially.

+------------+---------+------+-----+---------+----------------+
| Field      | Type    | Null | Key | Default | Extra          |
+------------+---------+------+-----+---------+----------------+
| id         | int(11) | NO   | PRI | NULL    | auto_increment |
| product_id | int(11) | NO   | MUL | 0       |                |
| branch_id  | int(11) | NO   | MUL | 0       |                |
| range_id   | int(11) | NO   | MUL | 0       |                |
+------------+---------+------+-----+---------+----------------+

 Indices:  product_id, branch_id, range_id, branch_id+product_id

All of my other tables ANALYZE quickly and correctly, returning 'OK'.

When attempting to ANALYZE this table, it always fails prematurely as below after about 60-240s.

+----------------------------------------+---------+----------+-----------------------------------------------+
| Table                                  | Op      | Msg_type | Msg_text                                      |
+----------------------------------------+---------+----------+-----------------------------------------------+
| db1.cat_products_ranges                | analyze | error    | Unknown - internal error 702 during operation |
+----------------------------------------+---------+----------+-----------------------------------------------+
1 row in set (2 min 19.26 sec)

Looking on the interwebs for this error is like finding a needle in a haystack and I'm out of ideas. The manual reference error 702 tells me that there was an attempted request to a non-master. In light of this, I attempted shutting down the non-master data node. The process is still running after 2 hours and I cannot tell whether it's hung or not. The cluster is operational, however.

Can I find out if it's still running or hung?

I have attempted rebuilding the cluster from scratch, literally rebuilding it last night with new VM's on the latest 7.4.6 version and importing the data.

I have attempted dropping the table and re-importing it via SQL, creating manually and importing via < dump.

Interestingly, trying to drop the indices results in SQL Error (1205): Lock wait timeout exceeded; try restarting transaction so I cannot attempt that.

Can anyone please shed some light on this or provide some suggestions as to debug? I have done everything I can think of and now at day 3 with no light at the end of the tunnel.

If you need additional details, please let me know.

Thanks in advance.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.