Hot answers tagged

16

Disclosure: I am a MySQL employee, working on MySQL Cluster. I would say that MySQL Cluster could achieve higher throughput / host than sharded MySQL+InnoDB provided that : Queries are simple All data fits in-memory In terms of latency, MySQL Cluster should have more stable latency than sharded MySQL. Actual latency for purely in-memory data could be ...


4

How about a GROUP BY count on foobar from scratch ??? First, insert any new data into foobar Then, do a fresh GROUP BY count on foobar into the temp table: CREATE TABLE foo_amount_new LIKE foo_amount; INSERT INTO foo_amount_new SELECT foo_id,COUNT(1) FROM foobar WHERE bar_id = ... GROUP BY foo_id; Finally, swap the temp table in and drop the old ...


4

MySQL Cluster supports storing non-indexed columns on disk-only with an LRU cache of recently accessed data. However indexed columns are always held in-memory. MySQL Cluster preallocates all memory, according to the DataMemory and IndexMemory parameters. It will not ask the underlying OS for more memory dynamically. This means that you need to have ...


4

MySQL Cluster is designed around 3 core principles: High availability (shared nothing, local and geographic replication, integrated heart-beating, automated failover and self-healing recovery, etc.) Write-scalability and real time performance (auto-sharding, in-memory optimizations, etc) Multiple database interfaces (SQL and NoSQL) These are discussed in ...


4

The data node marked as 'Master' in the output from the ndb_mgm show command is the coordinator of some cluster internal management tasks. For example, it coordinates changes to the distributed dictionary due to DDL commands, and manages node join and leave transactions. The 'Master' role is dynamic, and is generally assigned to the longest running data ...


4

You should go through some more articles regarding cluster to make better understanding. I have put here the steps of taking cluster backup and restore with 4 data node scenario. 1.MySQL Cluster Native Backup Tool (ONLINE Backup) and Restore 1.1 ndb_mgm> START BACKUP <Option> <Option>:NOWAIT, WAIT STARTED, WAIT COMPLETED NOWAIT - This ...


4

Each data node's redo log has a number of 'parts' - currently 4 by default. Each redo log part is like a mini redo log, with 1 / Num_parts of the capacity defined for 'Redo log'. This design increases file system and lower layers parallelism when writing the redo log, but requires balance across the parts to be reasonable. When a table is created, table ...


3

MySQL cluster is built around an index structure that is optimized for memory fit; a T-tree. This is different from your regular storage engines in MySQL which use a B-tree or B+tree structure, which can survive quite well out of memory fit assuming that you have some hot-spots / non uniform access (this is normally a safe assumption). If you want to build ...


3

To answer your specific question: master in this context means that it is this specific node that coordinates checkpoints and schema modification (create/drop/alter table) responsibility for data is shared equally on all nodes. responsibility for transaction coordination is also shared equally on all nodes. In terms of where it is used, MySQL Cluster is ...


3

When I wanted to get information about MySQL Cluster, MySQL (before Sun bought it) would send out a brief questionnaire, which I wish I still had. One of the questions asked if your tables had integer keys only. Although MySQL Cluster is ACID-complaint, it does not provide a suitable storage engine for data with compound keys. In some respects, you have to ...


3

currently the maximum data set size is around 3TB, if you are using all in-memory data. You can increase that if you use disk-based tables MySQL Cluster also has very flexible replication to external storage engines, ie to InnoDB, so many users manage their "hot" data in cluster, then replicate "colder / aged" data to innodb. There is a blog describing how ...


2

The basic docs for using the Memcached API for MySQL Cluster 7.2 start at http://dev.mysql.com/doc/refman/5.5/en/mysql-cluster-ndbmemcache.html On the memcached command line, you need to supply an option like "-E ndb_engine.so". Memcached has a "pluggable engine" architecture, and the MySQL Cluster component is implemented as an NDB Engine. When memcached ...


2

You can decrease the size of redo log files by decreasing parameter NoOfFragmentLogFiles which defines no. of fragment files or by reducing size of each fragment file by parameter FragmentLogFileSize. By default NoOfFragmentLogFiles=16 and FragmentLogFileSize=16M. Hence total redo log files size will be 256MB and there are 4 of these if you will look into ...


2

You're correct that the on-line repartitioning of data temporarily uses extra memory on the data nodes. In most cases the application data is split over many tables and so relatively little memory is needed when repartitioning each table. If the data is in one big table and you don't have the available RAM then there are a number of options... Perform the ...


2

You can view what is going on with "ndb_mgm -e 'N report eventlog'" where N is the data node's nodeid. This won't necessarily give you an estimate of when it will be finished but it can be used to verify that stuff is indeed going on and that the ndbd isn't just spinning indefinitely. "ndb_mgm -e 'all status'" is also useful, as it will show you the current ...


2

Ah ha. This seems to be the error you get when you try to replicate from a server that uses statement binlogs. I initially overlooked that this scenario was not supported. I was able to work around this by adding an intermediate mysqld that just converts the statement replication binlogs into mixed replication binlogs (using log-slave-updates and ...


2

So the solution was the conflicting myslqd. Apparently mysql has a mysqld and mysql cluster has a mysqld and they were conflicting or I was running the wrong one. In the end I uninstalled mysql and reinstalled mysql cluster and it worked perfectly. Big thanks to RolandoMySQLDBA for helping me troubleshoot this issue.


2

there are not Masters/Slaves in Cluster, the nodes are all active. That Master label is an internal thing and you don't have to take care about it. @Tom: are you sure you know mysql cluster? To be a specific use case doesn't mean to be "strange". To be more precise, MySQL Cluster gives the best when you have a huge amount of small transactions, since is a ...


2

Solution is rather simple: Run mysqld multiple times on the same DB Server !!!. Assign each mysqld to run on a different port. The OS will spread the multple mysqld processes across the cores. UPDATE You may have to jerry rig some port forwarding using mysql-proxy Assuming 10.240.35.180 is the SQL node startup mysql on port 3307 in the SQL node ...


2

In light of this, what happens if there's NOT enough memory Probably what you'd expect if there's not enough memory. Files will be made on your drives and you will be running as fast as your drives allow. Be sure to limit MySQL properly so that it doesn't consume all your memory needed for other system processes.


2

You may want to consider if you really need the MySQL Cluster, it may be overkill for your purposes and may not justify the complexity. A pair of MySQL Master-Master or Master-Slave instances with replication are good enough for many shops and will deliver better performance in this scenario. If you have to stay with your current architecture, you can add ...


2

DISCLAIMER : Not SCMCDBA Certified If you plan to hold data in memory, you need to increase RAM on all data nodes to 16GB increase the DataMemory to 6GB increase the number of SQL nodes to 3 (Makes SQLNodes to DataNodes 1:1) If you cannot make these changes, then you should offload data to local storage MySQL Cluster -- Storing table data on data ...


1

With noofreplicas=2 and two nodes, you will have 2/2 = 1 nodegroup, meaning that you have no sharding at all (or one shard, as you prefer). All your partitions will be in both nodes. Your cluster will provide you high availability and better read throughup, but not write scaling. What you show are the statistics for the table users, if you have not defined ...


1

First of all, are you creating your tables with CREATE TABLE table_name TABLESPACE hhmefep_tbs STORAGE DISK ENGINE NDBCLUSTER; or just CREATE TABLE table_name ENGINE=NDBCLUSTER; First one is to create table to stores table data on disk, the second, which is default mysql cluster theme and the reason why mysql cluster exist, stores data in memory. So ...


1

The problem lies in enabling the slaves on your SQL Node 2 in each datacenter. MySQL Cluster does support Multi-master replication as seen in the first diagram on the documentation page. Notice how only one SQL node in each cluster participate in the circular replication topology. And the duplicate issue is referenced in the quote from this page on setting ...


1

We should start the first node without below parameter in my.cnf : wsrep_cluster_address="gcomm://192.168.120.107,192.168.120.111,192.168.120.117" Then start the other nodes normally : /etc/init.d/mysql start The first node should start normally like the others.


1

Ok. The issue is related to SELinux. I put SELinux in permissive mode with setenforce 0 and restarted MySQL, and it connected fine - this gave it away pretty quickly. So I loaded audit2why and figured it out from that. I ran the following command then setenforce 1 to re-enable SELinux, restarted MySQL and it all works as expected! The magic command: ...


1

Why you can't change ndbcluster to InnoDB? Clustering with MySQL is supported only by the NDB storage engine. ndbcluster/ndb storage engine is meant for clustering if you specify a table with ENGINE=ndbcluster; this means table is being shared with all nodes and simply you can't change it to other ENGINE types, either you should take a backup of this table ...


1

After reading deeper the manual, storing the model in an InnoDB engine and running ndb_size.pl I have been able to find that these parameters (which had their default values) were in fact too low: MaxNoOfOrderedIndexes MaxNoOfUniqueHashIndexes MaxNoOfAttributes


1

It seems that for solving your problem you chose the wrong tool. MySQL Cluster is good when you do mostly key based lookups from multiple threads from memory based tables. Disk based tables may be useful when you have rarely read data. Or your work dataset is small portion of the table which should fit into memory cache whose size is defined by ...



Only top voted, non community-wiki answers of a minimum length are eligible