2

I am not sure how to put my question in a proper databse terminology. But here it is.

My understanding of data reading in relational database:

In relational databases, as all columns are present, so system knows how much bytes a row occupies on hard disk. So reading a row is like reading a chunk of memory bytes of cetain size(which is same for all rows in the table).

How does this work in a noSQL datastore like cassandra as columns are not mandatory and may be missing?

2
  • First that statement is not true for all databases. Second what do you mean by missing? Normally nil is stored when there is no valid data. Jun 10, 2023 at 21:10
  • 1
    @RohitGupta when a column is not set (not included in INSERT or UPDATE statement), that column is not stored at all on disk. In cqlsh it might look like it is set to nil but it doesn't actually exist -- it's just the way cqlsh displays unset columns to make it look "pretty" in the output. Cheers! Jun 22, 2023 at 2:53

2 Answers 2

1

When the data is written to disk, they are stored in column name/value pairs so when they are read off the disk the client knows which column the value belongs to. Let me illustrate with an example.

Here is a table that contains user details:

CREATE TABLE users (
    username text,
    name text,
    email text,
    favourite_colour text,
    PRIMARY KEY(username)
)

Let's assume the table contains the following data:

 username | name   | email                | favourite_colour
----------+--------+----------------------+------------------
 alice789 | Alice  | [email protected]       | nil
 bob23    | nil    | [email protected] | blue
 charli5  | Charli | nil                  | orange

For the first record, the data is stored on disk looking like this:

+-------------------+------------+----------------------+
| username=alice789 | name=Alice | [email protected] |
+-------------------+------------+----------------------+

Notice that the column favourite_colour is not stored at all.

For the second record, it is laid out on disk looking like this:

+----------------+----------------------------+-----------------------+
| username=bob23 | [email protected] | favourite_colour=blue |
+----------------+----------------------------+-----------------------+

Again, notice that the column name is not stored.

In contrast to RDBMS, CQL data types do not come with a fixed length so the amount of memory used to serialise/deserialise them varies depending on the contents of the columns. But since the column name is stored with the value, Cassandra knows which column the value belongs to. When the result is returned to the client, the driver also knows which columns are included in the result set because the details are included in the metadata. Cheers!

3

What you are describing is called the Read Path and this is probably under those terms that you would find documentation for any database technology.

Especially for Cassandra:

Firstly, you need to pick the proper nodes:

Cassandra is a distributed system and not all nodes contains the data you need. Each node is in charge of ranges of token. When you read the partition key is hashed, it gets you a token and Cassandra know on which nodeS the data is. (each data is likely to be on several nodes, this is something called replication factor not detailed here).

enter image description here

TLDR; Data is organized in SSTable and partitions and relatives offsets are stored.

  1. Check memory (MemTable) as data is there before disk:

  2. Checks Bloom filter (row cache is no more with recent Cassandra versions)

  3. Checks partition key cache, if enabled

  4. Goes directly to the compression offset map if a partition key is found in the partition key cache, or checks the partition summary if not If the partition summary is checked, then the partition index is accessed

  5. Locates the data on disk using the compression offset map

  6. Fetches the data from the SSTable on disk

Source, Reference and details

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.