Tag Info

Hot answers tagged

12

Disclosure : I am a MySQL employee, working on MySQL Cluster. I would say that MySQL Cluster could achieve higher throughput / host than sharded MySQL+InnoDB provided that : Queries are simple All data fits in-memory In terms of latency, MySQL Cluster should have more stable latency than sharded MySQL. Actual latencies for purely in-memory data could ...


4

How about a GROUP BY count on foobar from scratch ??? First, insert any new data into foobar Then, do a fresh GROUP BY count on foobar into the temp table: CREATE TABLE foo_amount_new LIKE foo_amount; INSERT INTO foo_amount_new SELECT foo_id,COUNT(1) FROM foobar WHERE bar_id = ... GROUP BY foo_id; Finally, swap the temp table in and drop the old ...


4

MySQL Cluster is designed around 3 core principles: High availability (shared nothing, local and geographic replication, integrated heart-beating, automated failover and self-healing recovery, etc.) Write-scalability and real time performance (auto-sharding, in-memory optimizations, etc) Multiple database interfaces (SQL and NoSQL) These are discussed in ...


4

The data node marked as 'Master' in the output from the ndb_mgm show command is the coordinator of some cluster internal management tasks. For example, it coordinates changes to the distributed dictionary due to DDL commands, and manages node join and leave transactions. The 'Master' role is dynamic, and is generally assigned to the longest running data ...


4

MySQL Cluster supports storing non-indexed columns on disk-only with an LRU cache of recently accessed data. However indexed columns are always held in-memory. MySQL Cluster preallocates all memory, according to the DataMemory and IndexMemory parameters. It will not ask the underlying OS for more memory dynamically. This means that you need to have ...


3

MySQL cluster is built around an index structure that is optimized for memory fit; a T-tree. This is different from your regular storage engines in MySQL which use a B-tree or B+tree structure, which can survive quite well out of memory fit assuming that you have some hot-spots / non uniform access (this is normally a safe assumption). If you want to build ...


3

To answer your specific question: master in this context means that it is this specific node that coordinates checkpoints and schema modification (create/drop/alter table) responsibility for data is shared equally on all nodes. responsibility for transaction coordination is also shared equally on all nodes. In terms of where it is used, MySQL Cluster is ...


3

Each data node's redo log has a number of 'parts' - currently 4 by default. Each redo log part is like a mini redo log, with 1 / Num_parts of the capacity defined for 'Redo log'. This design increases file system and lower layers parallelism when writing the redo log, but requires balance across the parts to be reasonable. When a table is created, table ...


2

there are not Masters/Slaves in Cluster, the nodes are all active. That Master label is an internal thing and you don't have to take care about it. @Tom: are you sure you know mysql cluster? To be a specific use case doesn't mean to be "strange". To be more precise, MySQL Cluster gives the best when you have a huge amount of small transactions, since is a ...


2

When I wanted to get information about MySQL Cluster, MySQL (before Sun bought it) would send out a brief questionnaire, which I wish I still had. One of the questions asked if your tables had integer keys only. Although MySQL Cluster is ACID-complaint, it does not provide a suitable storage engine for data with compound keys. In some respects, you have to ...


2

In light of this, what happens if there's NOT enough memory Probably what you'd expect if there's not enough memory. Files will be made on your drives and you will be running as fast as your drives allow. Be sure to limit MySQL properly so that it doesn't consume all your memory needed for other system processes.


2

The basic docs for using the Memcached API for MySQL Cluster 7.2 start at http://dev.mysql.com/doc/refman/5.5/en/mysql-cluster-ndbmemcache.html On the memcached command line, you need to supply an option like "-E ndb_engine.so". Memcached has a "pluggable engine" architecture, and the MySQL Cluster component is implemented as an NDB Engine. When memcached ...


2

You may want to consider if you really need the MySQL Cluster, it may be overkill for your purposes and may not justify the complexity. A pair of MySQL Master-Master or Master-Slave instances with replication are good enough for many shops and will deliver better performance in this scenario. If you have to stay with your current architecture, you can add ...


2

currently the maximum data set size is around 3TB, if you are using all in-memory data. You can increase that if you use disk-based tables MySQL Cluster also has very flexible replication to external storage engines, ie to InnoDB, so many users manage their "hot" data in cluster, then replicate "colder / aged" data to innodb. There is a blog describing how ...


2

Ah ha. This seems to be the error you get when you try to replicate from a server that uses statement binlogs. I initially overlooked that this scenario was not supported. I was able to work around this by adding an intermediate mysqld that just converts the statement replication binlogs into mixed replication binlogs (using log-slave-updates and ...


2

You're correct that the on-line repartitioning of data temporarily uses extra memory on the data nodes. In most cases the application data is split over many tables and so relatively little memory is needed when repartitioning each table. If the data is in one big table and you don't have the available RAM then there are a number of options... Perform the ...


1

You could run RESET QUERY CACHE to eliminate collected result sets in the query cache. You could also be more aggressive and drop the query cache and recreate it a different size. For example, to set the query_cache_size to 512M, do the following SET GLOBAL query_cache_size = 0; SELECT SLEEP(30); SET GLOBAL query_cache_size = 1024 * 1024 * 512; ...


1

You should read these posts and analyze what options to choose: High Availability and Scalability A Summary of Scaling Options for MySQL Where Would I Use MySQL Cluster? Also there are similar questions asked here: http://serverfault.com/questions/305024/is-it-possible-to-managing-20-tb-data-using-mysql ...


1

So the solution was the conflicting myslqd. Apparently mysql has a mysqld and mysql cluster has a mysqld and they were conflicting or I was running the wrong one. In the end I uninstalled mysql and reinstalled mysql cluster and it worked perfectly. Big thanks to RolandoMySQLDBA for helping me troubleshoot this issue.


1

You can view what is going on with "ndb_mgm -e 'N report eventlog'" where N is the data node's nodeid. This won't necessarily give you an estimate of when it will be finished but it can be used to verify that stuff is indeed going on and that the ndbd isn't just spinning indefinitely. "ndb_mgm -e 'all status'" is also useful, as it will show you the current ...


1

Hmm, the two data nodes on one server means that if you lose 192.168.0.1 (node 3/node 4) you lose the whole cluster (possibly why you are experiencing the failure). In fact the fact that you have effectively a single data node (unless I'm missing something) means you don't really have a cluster - you only have one copy of data.


1

I don't think I've ever come across any description of how ndbcluster affects row layout. (I haven't looked hard, though. Also haven't read the source code.) But, for example, innodb tables have a well-documented row layout. (Innodb row structure.) You can find quite a few things that affect the overall length of a row.


1

It looks like your MySQLD is running at port 5000, from the working mysql command line. The server listening on port 1186 is probably the cluster management server (ndb_mgmd). It has its own client (ndb_mgm). You cannot connect to it using the mysql client. Similarly, the MySQL JDBC driver cannot connect to it. Frazer


1

To follow up on JD's response, you can view a worked example of using the Memcached API to MySQL Cluster at http://www.clusterdb.com/mysql-cluster/scalabale-persistent-ha-nosql-memcache-storage-using-mysql-cluster/ You can choose whether certain key-value pairs are stored in Memcached, MySQL Cluster or both by setting up meta-data in the database.


1

I don't believe this is possible. From the FAQ on the MySQL and Memcache section: memcached plays no role in database writes, it is a method of caching data already read from the database in RAM. So inserting a value into memcached should not be expected to update the MySQL database. One major problem is that memcached has no knowledge of how and where ...


1

Maybe you could try the following: ALTER TABLE piwik_log_profiling RENAME piwik_log_profiling_old; CREATE TABLE piwik_log_profiling SELECT * FROM piwik_log_profiling_old WHERE 1=2; ALTER TABLE piwik_log_profiling ADD COLUMN query100 CHAR(100) NOT NULL AFTER query; ALTER TABLE piwik_log_profiling ADD UNIQUE INDEX (query100); ALTER TABLE piwik_log_profiling ...


1

MySQL Cluster automatically automatically replicates data nodes, based on the number of replicas and the number of data nodes you specify. Your tables are automatically sharded across these nodes, and updates are replicated between them for HA Worth watching the short demo to see more: http://www.oracle.com/pls/ebn/swf_viewer.load?p_shows_id=11464419 For ...


1

Basically, a chain is only as strong as its weakest link. When private virtual servers are setup (using VMWare to setup ESX Clusters) all bare-metal servers in the Cluster are supposed to have the same amount of RAM. MySQL Cluster should be treated with the same kind of kid gloves. If any servers (even just 1) have less memory that others in a Cluster, ...


1

Disclosure: I'm part of the MySQL Cluster product team The 255 node limit is for a single cluster. You could have multiples of these supporting your application, with replication between them. It is also important to consider per-node performance - in recent tests, MySQL Cluster achieved 6.8m QPS across 8 x physical Intel servers, configured with 2 sockets ...


1

AFAIK MySQL Cluster has the ability to scale, but it is really a manual process to do so. In fact, it takes more planning and more hardware to do such scaling. Here is why: MySQL Cluster According to "Guide to Scaling Web Databases with MySQL Cluster" (October 2011 Whitepaper), page 5 paragraphs 4,5 under the subheading 'Auto-Sharding': MySQL Cluster ...



Only top voted, non community-wiki answers of a minimum length are eligible