9

I am currently evaluating databases to use for a new project, which will require insertion and querying of large amounts of trading data. Our team is leaning towards Cassandra, but then I read this article that seems to suggest using non-ACID compliant databases can result in occasional data loss:

http://www.dbms2.com/2010/09/21/acid-compliant-transaction-integrity/

I can't find any further information on this on the web and can't understand how non-ACID compliance means that data loss can occur. Can anyone shed some light?

4
  • Neo4j is a NOSQL (graph) database that is actually ACID compliant. It comes with full transaction support and durable persistence. Neo4j also uses transaction-logs to secure write operations before applying them to the datastore. Disclaimer: I work for Neo Technology. Nov 18, 2011 at 5:55
  • 3
    According to Murphy's law (and my own experience) you can lose data with any database.
    – user1822
    Nov 4, 2013 at 19:17
  • A better phrasing might be "do NoSQL databases have a significantly greater chance of data loss or corruption than traditional RDBMS?" Still a little vague. Nov 4, 2013 at 20:01
  • Several NoSQL products offer single-row ACIDity. Few offer multi-row ACID. If your use-case is a stream single-key writes then you can succeed. With many business areas it's important to have multiple rows consistent simultaneously before effecting a change. Apr 15, 2015 at 12:24

3 Answers 3

7

Though this is years old question...

In short, you can understand ACID as guarantee of data integrity/safety in any expected circumstances. As like in generic programming, all the headaches comes from multi-threading.

The biggest issue on NoSQL is mostly ACI. D(urability) is usually a separated issue.

If your DB is single-threaded - so only one user can access at once -, that's natively ACI compliant. But I am sure virtually no server can have this luxury.

If your DB need to be multi-threaded - serve multiple users/clients simultaneously - you must need ACI-compliant transaction. Or you will get silent data corruption rather than simple data loss. Which is a lot more horrible. Simply, this is exactly same with generic multi-threaded programming. If you don't have proper mechanism such as lock, you will get undefined data. And the mechanism in DB called fully ACID compliance.

Many YesSQL/NoSQL databases advertises themselves ACID-complient, but actually, very few of them are really does.

  • No ACID compliance = You will get always undefined result under multi-user (client) environment. I don't even think what kind of DB does this.

  • Single row/key ACID compliant = You will get guaranteed result if you modify only single value at once. But undefined result (=silent data corruption) for simultaneous multi row/key update. Most of currently popular NoSQL DBs including Cassandra, MongoDB, CouchDB, … These kind of DBs are safe only for single-row transaction. So you need to guarantee your DB logic won't touch multiple rows in a transaction.

  • Multi row/key ACID compliance = You will always get guaranteed result for any operation. This is minimal requirements as a RDBMS. In NoSQL field, very few of them does this. Spanner, MarkLogic, VoltDB, FoundationDB. I am not even sure there's more solutions. These kind of DBs are really fresh and new, so mostly nothing is known about their ability or limitation.

Anyway, this is a comparison except D(urability). So don't forget to check durability attribute too. It's very hard to compare durability because range becomes too wide. I don't know this topic well…

  • No durability. You will lost data at any time.

  • Safely stored on disk. When you get COMMIT OK, then the data is guaranteed on disk. You lost data if disk break.

Also, there're difference even on ACID compliant DBs.

  • Sometimes ACID compliant / you need configuration / no automatic something.. / some components are not ACID-complient / very fast but you need to turn off something for this... / ACID-compliant if you use specific module... = we will not bundle data safety by default. That's an add-on, option or separated sold. Don't forget to download, assemble, setup and issuing proper command. Anyway, data safety may be ignored silently. Do it yourself. Check it yourself. Good luck not to make any mistake. Everyone in your team must be flawless DBA to use this kind of DB safely. MySQL.

  • Always ACID compliant = We don't trade data safety with performance or anything. Data safety is a forced bundle with this DB package. Most commercial RDBMS, PostgreSQL.

Above is typical DB's implementation. But still, any other hardware failure may corrupt the database. Such as memory error, data channel error, or any other possible errors. So you need extra redundancy, and real production-quality DB must offer fault tolerance features.

  • No redundancy. You lose all data if your data corrupted.

  • Backup. You make snapshot copy/restore. You lose data after last backup.

  • Online backup. You can do snapshot backup while the database is running.

  • Asynchronous replication. Backup for each second (or specified period). If machine down, this DB guaranteed to get the data back by just rebooting. You lose data after last second.

  • Synchronous replication. Backup immediately for each data update. You always have exact copy of original data. Use the copy if origin breaks.

Until now, I see many DB implementation lacks many of these. And I think if they lacks proper ACID and redundancy support, users will lose data eventually.

6

ACID means

  • Atomicity
  • Consistency
  • Isolation
  • Durability

What this means to you is "every write action will be done only once (no duplicate records) but will be completely stored in the database when the action is done" and that every time you read, you're getting the data you want out.

The thing about the NoSQL databases is that they're often distributed (that's what people want, flat-scalable systems that are cheap), which means that it takes time to replicate the data to all the nodes. Sometimes it's possible to be reading during a write and end up with the old data while the new data is coming out.

You're sacrificing purity for speed.

This is the short version of my answer, and I'm not sure what I need to explain further. Ask me questions!

4
  • 1
    What you're describing sounds like immediate consistency (RDBMS) vs. eventual consistency (NoSQL). However, the linked article talks about actually losing data (not simply having inconsistent data), and I don't understand what ACID compliance has to do with data loss.
    – del
    Oct 13, 2011 at 3:53
  • 1
    Durability most likely. And that is the case, that's what I'm describing (which makes it seem like data has been lost). The point to ACID is that you can't lose data. Ever. (well it could be lost to damage)
    – jcolebrand
    Oct 13, 2011 at 3:58
  • All of the NoSQL databases I've looked (HBase, Cassandra, Redis) use write-ahead logs which can be replayed in case the database crashes before changes have been persisted. Does that mean this criticism doesn't apply to any of these databases?
    – del
    Oct 13, 2011 at 4:09
  • I would imagine so. I will revisit this on the morrow, but for now, bedtime. Hopefully you get some other input besides mine before then ;-)
    – jcolebrand
    Oct 13, 2011 at 4:12
5

"It depends" is the answer - there are configuration options, mentioned here.

Small nitpick: a database can be durable but not ACID compliant, since ACID is the superset of features (A-C-I-D). I don't think any NoSQL database can claim to be fully ACID, but many of them may claim to pass individual sub-requirements, such as durability.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.