1

I have 3 node cluster and upon checking the nodetool status; Load is just less than 100 GB on all three nodes. The replication factor is two and percentage own is 65-70% for all three.

However when I inspected the /data directory it is having index.db files for size more than 400 GB and the total size of the keyspace directory is more than 700GB.

Any idea on why a huge gap??

Let me know if any extra details are required :)

PS: nodetool listsnapshots command shows an empty list (No snapshots)

2
  • have you checked snapshots?
    – Alex Ott
    Commented Jan 11, 2023 at 13:44
  • Yes and the list of snapshots is empty Commented Jan 12, 2023 at 4:52

1 Answer 1

1

There are several reasons why there is a discrepancy between the data size reported by nodetool status and the output of Linux utilities such as du and df.

In your case where you think there is about a 600GB difference, the most likely cause is that there are files on disk which are not used by Cassandra usually because someone has placed it there. For example, someone copied files into one or more directories as an attempt to restore some data.

My suggestion is to review each table directory, comparing it's size on disk with the size as reported by nodetool tablestats to try to narrow it down. Cheers!


👉 Please support the Apache Cassandra community by hovering over the tag then click on the Watch tag button. 🙏 Thanks!

3
  • this could not be the case; because for one I can confirm that no files were added manually in the Cassandra data directory. Secondly, once the compaction failed we observed that almost all these high size idx.db files were lost and after that the size of the cassandra data directory was reduced to a value slightly more than the load. However, again we are seeing that as we use the database, the size of data directory is going more than 600 GB and the load is still around 100 GB. I am just not sure in what scenario can idx.db files and the load could have such a huge gap. Commented Jan 12, 2023 at 4:51
  • What are idx.db files? Did you mean *-Index.db files? Commented Jan 18, 2023 at 1:34
  • Yes, the files with respect to index only Commented Mar 6, 2023 at 4:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.