Hot answers tagged

120 votes
Accepted

Best database and table design for billions of rows of data

This is exactly what I do every day, except instead of using the hourly data, I use the 5 minute data. I download about 200 million records everyday, so the amount you talk about here is not a problem....
user avatar
76 votes

Best database and table design for billions of rows of data

PostgreSQL and BRIN indexes Test it for yourself. This isn't a problem on a 5 year old laptop with an ssd. EXPLAIN ANALYZE CREATE TABLE electrothingy AS SELECT x::int AS id, (x::int % ...
user avatar
  • 55.7k
48 votes
Accepted

If NoSQL stands for "Not only SQL", is SQL a subset of NoSQL?

TL;DR The main distinction between database systems is not the language used to query the database, but rather the consistency model of the system. The Venn diagram should be two intersecting sets - ...
user avatar
24 votes

If NoSQL stands for "Not only SQL", is SQL a subset of NoSQL?

NoSQL was, is, and always will be a vague buzzword, not a well-defined term. Its origin is related to a historical trend: at the beginning of this century, if you said "database", the ...
user avatar
  • 395
18 votes
Accepted

redis newbie - how to create hash within a hash?

Redis doesn't support nested data structures, and specifically it doesn't support a Hash inside a Hash :) You basically have a choice between two options: either serialize the internal Hash and store ...
user avatar
15 votes
Accepted

Is a single-table relational database similar to a simple NoSQL database?

No - a single table relational database is more akin to a spreadsheet than a NoSQL database. NoSQL is not about having no schema, it's more about having a flexible schema! Wikipedia puts it very ...
user avatar
14 votes

Best database and table design for billions of rows of data

It amazes me me that nobody here has mentioned benchmarking - that is until @EvanCarroll came along with his excellent contribution! If I were you, I would spend some time (and yes, I know it's a ...
user avatar
11 votes

What is a good way to copy data from one Cassandra ColumnFamily to another on the same Keyspace (like SQL's INSERT INTO)?

cqlsh -k mykeyspace -e 'COPY fromTable(columnNames) TO STDOUT' | head -n -1 | cqlsh -k mykeyspace -e 'COPY toTable(columnNames) FROM STDIN'
user avatar
11 votes
Accepted

Redesign the storage of large amounts of sensor data

Since (a) the information you are working with appears to be, in a of itself, a very valuable organizational resource, and (b) the volume of data will be considerable, I would decidedly (c) build a ...
user avatar
  • 8,210
10 votes

Redis takes up all the Memory and Crashes

Use the maxmemory to set a limit to how much your Redis database can grow too. Failing to do so, Redis will grow until the OS will kill it once memory is exhausted (per your current experience). The ...
user avatar
10 votes
Accepted

ALLOW FILTERING vs NOT; Cassandra Data Model Question

Right now the table only has about 320k records and I can use ALLOW FILTERING with no problem, but I realize this might not always be the case. So here's the thing: Cassandra is very good at querying ...
user avatar
  • 3,162
10 votes
Accepted

Why Cassandra is not considered a relational database?

Cassandra is not an RDBMS because it does not support the relational data model. The fundamental assumption of the relational model is that all data is represented as mathematical n-ary relations, an ...
user avatar
9 votes

Is a single-table relational database similar to a simple NoSQL database?

Yes, you can use a single-table relational database as if it were a NoSQL database. You can also use a spoon to paint your house. You have to use a tool the way it was intended else you're fighting ...
user avatar
8 votes

MongoDB vs RDBMS DBA duties

DBA is a small acronym but a large role. At various times I have seen a DBA look after storage network VMs compute nodes installation and configuration of the above backups, and testing restores DR ...
user avatar
8 votes
Accepted

What is the point of column families?

I've just uncovered some interesting information from the RocksDB FAQ. (RocksDB is a K-V store.) Here are some relevant extracts. Q: What are column families used for? A: The most common ...
user avatar
  • 403
8 votes

If NoSQL stands for "Not only SQL", is SQL a subset of NoSQL?

SQL is a language. NoSQL is an approach to choosing database systems. It makes no sense to compare them. The origin of "Not Only SQL" lies in the fact that not too long ago, when you had ...
user avatar
7 votes
Accepted

MongoDB vs RDBMS DBA duties

I have been in the NoSQL field since last 3 years. Being a MongoDB DBA you need to work closely with Development and Ops teams. Following are the things you need to do as day to day tasks as a MongoDB ...
user avatar
  • 334
7 votes

Best database and table design for billions of rows of data

If you have not already, take a look at a time series DBMS, since it is optimized for storing and querying data where the primary focus is the date/time type. Typically time series databases are used ...
user avatar
6 votes

Use SQL or NoSQL?

It's really too early to tell from what you've described - not enough detailled information. There are those who say (probably correctly) that NoSQL systems can update rapidly - even more rapidly than ...
user avatar
6 votes

Best database and table design for billions of rows of data

Clearly this is not a NoSQL problem, but I would suggest that while an RDBMS solution would work, I think an OLAP approach will fit much better and given the very limited data ranges involved, I would ...
user avatar
5 votes

Redis takes up all the Memory and Crashes

This is almost certainly memory fragmentation, as redis is well-known and loved in production and you probably haven't found a memory leak. The recommendations about setting the size of the pool won'...
user avatar
5 votes
Accepted

Data modeling with many attributes where few apply to each record

Option 4: Class Table Inheritance This is a design technique used to implement a class/subclass situation. In this situation, attributes often apply only to one subclass, and not to all rows in the ...
user avatar
  • 4,068
5 votes

Replace relational database with "poor" design in software product with non-relational DB?

Welcome to database hell! NoSQL is often championed as a solution to these types of applications. However, your problem here is clearly that the programmers have no idea what they are doing. ...
user avatar
5 votes

PostgreSQL JSON data type used as NOSQL, But VIEW as Relational Data Structure

As you have multiple rows inside the JSON column, you need a function that returns a set. This can be done using the json_to_recordset() function: select j.* from json_test, json_to_recordset(...
user avatar
5 votes

How do relations work in NOSQL?

In your question you are using "SQL" and "relational" to mean the same thing. In the same way you can think of "NoSQL" as meaning "not relational" i.e. there is no primary key / foreign key concept; ...
user avatar
5 votes

NoSQL : What is unstructured data?

Despite the mainstream of noSQL databases IMHO the decision about adopting such technology should be made according to the achievements needed according to the information stored, not only attending ...
user avatar
5 votes
Accepted

A good use case for Postgres JSONB?

This is indeed a reasonable case for storing object-like or key/value data, and representing it as JSON in jsonb fields in PostgreSQL is a reasonable way to do that. In general it's time to consider ...
user avatar
  • 51.7k
5 votes
Accepted

How do you design schema for tracking change history in NoSQL?

Add three attributes to the project and asset documents - WrittenWhen, WrittenBy and IsActive. Populate these in the "real" data. When a record changes copy it but with IsActive flag flipped. Retain ...
user avatar
5 votes

Why can't I take RDBMS Database Design practices, and use for NoSQL MongoDB Database Designs?

If you want to use a relational structure, and you're thinking about doing it like this with a NoSQL system, use an RDBMS, that's what they're designed to do. Using the right tool for the job will ...
user avatar
  • 67k
5 votes

Why Cassandra is not considered a relational database?

Why Cassandra is not Relational The other answers are close (I will explain why each incomplete), but this snippet of text from the documentation highlights what exactly makes Cassandra non-relational:...
user avatar
  • 3,432

Only top scored, non community-wiki answers of a minimum length are eligible