0

We have two separate Cassandra clusters, each with six nodes: Cluster A and Cluster B. Our requirement is to merge these two clusters to save costs. Because this is a critical application, we need to plan this with no downtime or a very minimal cutover window.

We can create tables from Cluster B to Cluster A and copy data using snapshots, but how can we ensure continuous writes for live data into the new Cluster A and minimize downtime? Any suggestions on how to proceed with this activity?

1 Answer 1

0

To migrate the data there are several options available, depending on the volume of data. For smaller datasets, data can be migrated via CSV using either:

  • cqlsh COPY TO (export) and COPY FROM (import) - "sort of" reliable with datasets under 10GB, but may need some fine tuning using the COPY options to work
  • dsbulk - more reliable than cqlsh COPY, but not practical for exporting datasets in the order of 100s of GB or greater

For larger datasets, the Spark migrator built by DataStax is probably a better solution to migrate data.

Another "viable" option to migrate data is the native tool sstableloader. However, sstableloader was built to export from offline nodes, and is infamously tricky to get right.

Then you have the problem of performing the migration without downtime. To achieve that, Cluster B must be the reference to read from until the migration is complete, while also receiving new writes. To ensure data integrity during the migration, new write requests should route to both cluster A and B.

DataStax developed a tool to achieve precisely this - the Zero Downtime Migration (ZDM) proxy

The ZDM proxy will sit between your drivers and the Cassandra clusters. You need to configure the app drivers to point requests to the ZDM proxy instead of cluster B, while the ZDM proxy will connect to both clusters, routing writes to both, and reads only to cluster B.

My suggestions:

  • Back up your data before attempting any migration operations.
  • Start by testing migration tools on a lower environment with a dataset of similar size, and ideally an identical schema. Establish which migration tool to use.
  • Once a migration tool is validated and established for the intended migration, test the ZDM proxy on a lower environment as well.
  • Test the full migration on a lower environment, by deploying the ZDM proxy first, to only then start the data migration. Validate data migration from the application side once the process is over, remove the ZDM proxy, and point drivers to the target cluster.
  • Repeat process on the intended clusters B->A after it is validated on a lower environment.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.