I was reading article from : http://www.datastax.com/docs/1.0/cluster_architecture/replication#networktopologystrategy and I have a question about - EC2MultiRegionSnitch. I did not understand it well. According to article: if application requests data from node, but that node does not have data, but data is in another node in other data center, will application be able to get it?


Lets understand data distribution in multiple data center first. If you have two data-centers -- you basically have complete data in each data-center. And if you have set replication factor, say, 2 for each data-center -- this means each data-center will have 2 copies of the data. This is where token assignment to nodes comes into the picture as important factor to make sure no node is overburdened. See this image from datastax website:

enter image description here

You see T0 and T3 are fatter than the rest. If you see whole cluster, it seems that data must be equidistributed... but that's not true. Each data-center can be seen as virtual ring within the ring. :) You see the images below? Now you realize that T0 is carrying data range (T2, T0] and T3 is loaded with (T5, T3]! But the fact that all is within the data-center, on any sunny day, you would not need to go fetch from other data-center unless you have ALL or EACH_QUORUM (or a CL more than number of replica on the data-center) as your consistency level.

So, to answer your question: In a multiple data-center setup, you will always have data in each data-center (assuming proper replication factor is setup). If you have a consistency level such that it require to read/write from other data-center to satisfy the consistency level, the coordinating node (the one your client is connected to) will talk to a node in other data-center. In EC2, each region is a data-center, each availability zone is treated as rack.

| improve this answer | |
  • Hmm, that is not what I need, because network speed between data centers are slow. So I do not want replication occur between data centers. Can I configure Cassandra so that local data is stored local only and be accessed from other data centers? – Abzal Kalimbetov Jun 21 '13 at 4:55
  • cross-data center requests occur very rarely – Abzal Kalimbetov Jun 21 '13 at 4:57
  • 1
    what do you mean by "local data"? Look at consistency level. If you have set consistency level "LOCAL_QUORUM" the request will never wait for data to be returned from other data center. And, if you write with "LOCAL_QUORUM" data will be written to the data-center the client is connected to and then the replication will take place in background. So, connection speed between data-centers should not matter. You can tune it to performance requirement. – Nishant Jun 21 '13 at 5:00
  • local data is local to data center that is in one region. In other words, I do not want to replicate to all data centers, but I want one data center to have all replica of others. – Abzal Kalimbetov Jun 21 '13 at 10:06

In regard to a standard implementation of Cassandra the answer is Yes. So everything in the cluster regardless of the data center would have access.

From the Apache Cassandra site:

Cassandra's support for replicating across multiple datacenters is best-in-class, providing lower latency for your users and the peace of mind of knowing that you can survive regional outages.

The site you're referencing is talking about a service they perform called partitioning where they're actually distributing data in a different way based on their partitioning scheme, while still using Cassandra as the database. They do replication based on NetworkTopologyStrategy which depends on the network and the number of datacenters.

In regard to Amazon EC2 They're mixing lingo a little bit which is where the confusion is coming in.

... on Amazon EC2 where all nodes in the cluster are within a single region. The region is treated as the data center and the availability zones are treated as racks within the data center.

So on Amazon's network a region will contain multiple data centers, but they're distributing info like it were local to a data center (probably because their network has plenty of resources and nodes). This document here goes more into the availability zones concept for their network. Basically their Network Topology is a little larger than most networks.

The data should remain in what they're considering a data center, but whether this is the same physical data center or not would depend on the number of racks, the types of machines they're running and the backbones (OC-3,OC-12,OC-48, etc lines) going between them. If you're in a region where they can't have replication across multiple data centers as quickly they'll likely have more redundancy in the physical data centers themselves.

| improve this answer | |
  • So the data will remain locally for each data center, right? – Abzal Kalimbetov Jun 21 '13 at 4:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.