Bulk-loading snapshot, sstableloader returns "[table] doesn't exist"

Question

I have about 60GB of data on a simple scylla cluster and I would like to migrate them to Cassandra 4.1.5. I have taken a snapshot on scylla with command

nodetool snapshot -t scylla-20240524 --table <table> <keystore>

I am now trying to import it into Cassandra running

bin/sstableloader -d localhost -k <keystore> /mnt/scylla-20240524

But I get the following output:

Established connection to initial hosts
Opening sstables and calculating sections to stream
Skipping file me-4332-big-Data.db: table <keystore>.scylla doesn't exist
Skipping file me-4333-big-Data.db: table <keystore>.scylla doesn't exist
Skipping file me-4334-big-Data.db: table <keystore>.scylla doesn't exist
Skipping file me-4335-big-Data.db: table <keystore>.scylla doesn't exist
Skipping file me-4400-big-Data.db: table <keystore>.scylla doesn't exist
Skipping file me-4401-big-Data.db: table <keystore>.scylla doesn't exist
Skipping file me-4402-big-Data.db: table <keystore>.scylla doesn't exist
Skipping file me-4403-big-Data.db: table <keystore>.scylla doesn't exist

Summary statistics: 
   Connections per host    : 1         
   Total files transferred : 0         
   Total bytes transferred : 0.000KiB  
   Total duration          : 3954 ms   
   Average transfer rate   : 0.000KiB/s
   Peak transfer rate      : 0.000KiB/s

I can't find 'scylla' as a table in scylla schema... is there anything I should do to prepare cassandra schema? or what else?

Answer 1

The problem has to do with the directory you are passing to sstableloader. The format for running the command is:

$ sstableloader -d host [options] /path/to/ks_name/table_name/

where table_name/ must be the name of the table you are cloning and the subdirectory which contains the data files. In your case, the path you supplied is /mnt/scylla-20240524 so the loader command thinks the table name is scylla.

You need to copy the snapshot's contents to a directory structure that follows the format ks_name/table_name/.

If you're interested, I've previously written a step-by-step procedure for cloning data from one cluster to another in How do I migrate data in tables to a new Cassandra cluster?. Cheers!

Answer 2

Alternate option is to leverage this fully OSS DSBulk tool as it can very well handle 60GB seamlessly for your use case here.

It can be as simple as this command,

# './dsbulk --version' should be 1.11.0

./dsbulk unload -k origin_keyspace_name -t origin_table_name -u <USERNAME> -p <PASSWORD> | ./dsbulk load -k target_keyspace_name -t target_table_name -u <USERNAME> -p <TARGET_PASSWORD>

Note: This doesn't use local storage as we're using pipe function to leverage the unloaded rows from origin cluster directly and loading them to our target cluster.

Resources

• Documentation in case if you need to use other advanced configurations
• Technical blogs explaining DSBulk usages, tips & tricks