Cassandra in multiple data center - EC2MultiRegionSnitch

Question

I was reading article from : http://www.datastax.com/docs/1.0/cluster_architecture/replication#networktopologystrategy and I have a question about - EC2MultiRegionSnitch. I did not understand it well. According to article: if application requests data from node, but that node does not have data, but data is in another node in other data center, will application be able to get it?

Answer 1

Lets understand data distribution in multiple data center first. If you have two data-centers -- you basically have complete data in each data-center. And if you have set replication factor, say, 2 for each data-center -- this means each data-center will have 2 copies of the data. This is where token assignment to nodes comes into the picture as important factor to make sure no node is overburdened. See this image from datastax website:

You see T0 and T3 are fatter than the rest. If you see whole cluster, it seems that data must be equidistributed... but that's not true. Each data-center can be seen as virtual ring within the ring. :) You see the images below? Now you realize that T0 is carrying data range (T2, T0] and T3 is loaded with (T5, T3]! But the fact that all is within the data-center, on any sunny day, you would not need to go fetch from other data-center unless you have ALL or EACH_QUORUM (or a CL more than number of replica on the data-center) as your consistency level.

So, to answer your question: In a multiple data-center setup, you will always have data in each data-center (assuming proper replication factor is setup). If you have a consistency level such that it require to read/write from other data-center to satisfy the consistency level, the coordinating node (the one your client is connected to) will talk to a node in other data-center. In EC2, each region is a data-center, each availability zone is treated as rack.

Answer 2

In regard to a standard implementation of Cassandra the answer is Yes. So everything in the cluster regardless of the data center would have access.

From the Apache Cassandra site:

Cassandra's support for replicating across multiple datacenters is best-in-class, providing lower latency for your users and the peace of mind of knowing that you can survive regional outages.

The site you're referencing is talking about a service they perform called partitioning where they're actually distributing data in a different way based on their partitioning scheme, while still using Cassandra as the database. They do replication based on NetworkTopologyStrategy which depends on the network and the number of datacenters.

In regard to Amazon EC2 They're mixing lingo a little bit which is where the confusion is coming in.

... on Amazon EC2 where all nodes in the cluster are within a single region. The region is treated as the data center and the availability zones are treated as racks within the data center.

So on Amazon's network a region will contain multiple data centers, but they're distributing info like it were local to a data center (probably because their network has plenty of resources and nodes). This document here goes more into the availability zones concept for their network. Basically their Network Topology is a little larger than most networks.

The data should remain in what they're considering a data center, but whether this is the same physical data center or not would depend on the number of racks, the types of machines they're running and the backbones (OC-3,OC-12,OC-48, etc lines) going between them. If you're in a region where they can't have replication across multiple data centers as quickly they'll likely have more redundancy in the physical data centers themselves.