Take the 2-minute tour ×
Database Administrators Stack Exchange is a question and answer site for database professionals who wish to improve their database skills and learn from others in the community. It's 100% free, no registration required.

In Cassandra, it's not recommended to have more than a few thousand column families, let's say 2,000 for the sake of argument. In cases where more than 2,000 types of data need to be persisted, one approach would be to shard multiple unrelated types of data into each column family.

For example, a single CF could contain Orders, Invoices, and Customers, provided their row keys were distinct (e.g. prefixed with the object type, i.e. the keys of a single CF could include both Order|1234 and Customer|1234). A second CF could contain say Addresses, LineItems, and OrderTypes. Given the basic feasibility of this approach, what are the practical limits on it? For example, what would be wrong with putting all 10,000 types of object into a single CF? As far as I can tell from the Cassandra wiki, there is no hard limitation on the size of a CF.

share|improve this question
add comment

1 Answer 1

I'm not a fan. It's about as good an idea as creating a relational table named OrdersOrCustomers with columns defined for both. The storage-engine penalty is slightly lower in Cassandra because of the sparse-cell storage under the hood, but it's still bad practice.

This bites you later when you want to map/reduce over your data; each task will have to scan over all your data, and filter out the rows that don't match what you're actually interested in (e.g., customers). And good luck making sense of the statistics that Cassandra tracks per-CF. ("Is this CF the source of 80% of my application reads because of the order data? Or because of the customer sessions it's combined with? Or the other five data types I threw in?")

If you absolutely positively need tens or hundreds of thousands of CFs? Even then I'd rather run Cassandra without arena allocation, than mutilate my data model like this.

share|improve this answer
    
Thanks for your answer, Jonathan. I'll need to read up on arena allocation to understand the last sentence. Is using arena allocation a configuration option? –  Andrew Swan Jan 11 '13 at 5:23
    
Or if arena allocation is baked into Cassandra nowadays, how hard would it be to deallocate the arenas of any column families that haven't been accessed within some configurable time period? Is it worth me logging a JIRA ticket about this? –  Andrew Swan Jan 11 '13 at 5:56
1  
Cassandra is already smart about flushing to relieve memory pressure. The problem is with worst-case scenarios where everything is active. –  jbellis Jan 12 '13 at 15:23
    
In which case is the only solution to have a second ring? –  Andrew Swan Jan 14 '13 at 21:41
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.