0

How to tell if a project needs a NoSQL database solution?

I think that SQL limits define it. If the project needs bigger database or bigger fields then it needs NoSQL.

Am I right?

I looked into NoSQL database limits. They are below:

For an example I'm using MongoDB assuming that DynamoDB has similar numbers.

Mongo can be installed on a number of computers/nodes. PostgreSQL does not provide built-in tool for sharding, however citus is around.

MongoDB supports databases up to 64 terabytes and document size is 16 megabytes.

MySQL has a database limit of 256 terabytes, 64TB the maximum size for a table and record limit of 4 gigabytes.

PostgreSQL has no limit on database (4 terabytes does exist somewhere for testing) and it has a limit of 1 gigabytes for the size of any one field in a table and again 64TB the maximum size for a table.

Is there anything else that is relevant for decision making process?

closed as primarily opinion-based by Daniel Vérité, Mr.Brownstone, Colin 't Hart, mustaccio, Evan Carroll Jan 2 at 2:53

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 3
    A Tank "has no limits", but do you need it to drive your kids to school? – Rick James Dec 28 '18 at 4:13
  • 2
    Yes, this is my question: How to tell if you need a Tank to drive your kids to school? – Yevgeniy Afanasyev Dec 28 '18 at 4:17
  • 5
    Rather than focusing on data volume or the largest column size, I suggest you concentrate on your business case. What sort of objects do you need to store ? How frequently are they inserted ? How long do you keep them ? What do you do with them once they are stored. BTW commercial relational databases (Oracle) have much higher capacity than the ones you mention. But that is largely irrelevant without knowing your use case. – Albert Godfrind Dec 28 '18 at 18:22
  • Yes, this is my question: What sort of objects do you need to store to be in need of a NoSQL solution? – Yevgeniy Afanasyev Jan 1 at 22:54
  • the answet to that is the empty set. a better question could be what constraints will nosql impose on data types I can store. – Jasen Jan 6 at 22:29
3

How to tell if a project needs NoSQL database solution?

I think that SQL limits define it, if project needs bigger database or bigger fields than it needs NoSQL.

Am I right?

You are not right ;)

Choosing NoSQL over a relational database has a lot more to do with business reasons like:

  • do we have people on the team already familiar with NoSQL database?
  • will it be cheaper (in terms of hardware, licensing, support, etc)?
  • will it be faster (because of tooling, or the schema-less nature of document databases, etc)?

It also has a lot to do with the types of data you're storing, and your data modeling requirements. For instance, if your application needs to store large amounts of XML or JSON data, or if every entity you save might be slightly different, you might be better off with NoSQL. But if you need to model relationships and enforce constraints, referential integrity, etc, then you would likely be better off with a relational DB.

  • 1
    MySQL and PostgreSQL has JSON data types, why this is not the same as NoSQL database? – Yevgeniy Afanasyev Dec 29 '18 at 6:34
1

Do you really plan to come anywhere near to the limits you mentioned? I think not. Look at it this way...

Let's say you will be inserting data at a rate of, say, 1000 rows per second. (This is rather fast.) Now calculate how long it would take to fill up a 64TB table. 1K rows/sec * 100 bytes/row * 30M seconds/year = 3TB/year. And that does not give you much bandwidth to read or process the data.

At these extremes, any kind of datastore will be creaking and groaning.

Can you even configure a computer in your budget range that holds 64TB?

My point is that all db limits are probably so high that you will probably never hit them. At least not before you melt down the network feeding the computer, or the CPUs in it or whatever.

Look around this Forum. Not many of the Questions ask how to deal with some "limit". And those that do are usually answered by providing some workaround (after hinting that the developer was doing something stupid).

Here are more MySQL limits.

  • 1
    "1000 rows per second". Those that bulk process data (data warehouse) laugh at this number – Michael Kutz Dec 28 '18 at 12:11
  • 1
    @MichaelKutz 95% of projects are never reached the 1k/s limit. When some project overcome the "Big Business" entry threshold, then it should be teared to shreds and refactored on the good theoretical basis. And then there is the place for all kinds of engines. But in the prenatal period any kind of engine can be used as well as it is known to the developer. – Kondybas Dec 29 '18 at 14:11
1

NOSQL solutions arise as an answer to the CAP problem - the impossibility for RDBMS to guarantee all the three Consistency, Availability and Partitional tolerance at the same time. We always have to trade between those three requirements. Most RDBMSs are consistency+availability oriented by the price of loose p.tolerance. CouchDB is A+P oriented while Redis and MongoDB are C+P ones.

The good description of the whole problem and approaches can be found here:
http://blog.grio.com/2015/11/sql-nosql-a-brief-history.html

Also the good reading (with lot of external references) is here:
https://github.com/donnemartin/system-design-primer

1

How to tell if you need a Tank to drive your kids to school?

I will answer this question, because the people of dba.stackexchange rarely help anyway. If you want to pick a database, or want input for some table schema, go to the programmers stack. This isn't because they don't want to, mostly because you are asking a programming question thinking that it's a database question.

Why I use MongoDB:

  • I want my kids safe, so I drive a tank, better safe, than sorry.
  • I want my kids to speak my native language, Java. If I find my kids speaking SQL, they get grounded.
  • My kids are never late, never miss a class. With replication, my tank has 3 engines.
  • My kids study at different schools, no problem, using shards.

My advice, you should know your kids better. Kids being your programs/apps/clients.

You only asked: If I have too many kids, what database can hold them all?


The wikipedia definition for NoSQL:

A NoSQL (originally referring to "non SQL" or "non relational") database provides a mechanism for storage and retrieval of data that is modeled in means other than the tabular relations used in relational databases.

If this is the definition of NoSQL, then this becomes the rule you should follow to answer the question bellow:

How to tell if a project needs a NoSQL database solution?

Use NoSQL if your project needs a mechanism for storage and retrieval of data that relational databases don't offer.

Your question is NOT opinion based, but any other answer makes it open-ended.


Is there anything else that is relevant for decision making process?

  1. Devs (developers)

    • Storage limits
    • Response performance
    • Drivers
    • Models/types supported
    • Data isolation (decide in what server should data be stored)
    • Transactions
    • Change streams (access real-time data changes)
  2. Ops (operations)

    • Backup/restore procedure (they can be problematic)
    • Upgrade procedure (few databases can be 100% on-line)
    • Clustering/Grid/Replication

More? Tell me what your kids need.

  • You say "you should know your kids better". You are trying to answer as it is an opinion based question, when it is not! I ask a definite technical reasons of using NoSQL solution. For example Tank is needed when people shoot at your vehicle. And this is not my opinion because it is set in tank's instruction. I know it is a narrow line, but that is my problem and I ask for help. – Yevgeniy Afanasyev Jan 1 at 23:03
  • What does Mongo have to do with Java? – ypercubeᵀᴹ Jan 2 at 1:43
  • @yper-crazyhat-cubeᵀᴹ MongoDB Java Driver: mongodb.github.io/mongo-java-driver – Marcos Zolnowski Jan 2 at 1:49
  • So you can use Java to access a MongoDB database. You can also use it to access a SQL database. Where's the difference? – ypercubeᵀᴹ Jan 2 at 1:57
  • Because there is NO SQL in it, and no stupid SQL-mapping. – Marcos Zolnowski Jan 2 at 2:02

Not the answer you're looking for? Browse other questions tagged or ask your own question.