-2

So I am making a Discord (social media) Bot in Python using discord.py-rewrite, and I need to save informations of lots of servers and users (such as how much money they have, what items thay have…). This can grow exponentially. My question was, what is the best database to use then (sqlite3, PostGreSQL/asyncpg, MySQL…)? I currently use sqlite3, but I heard it wasn't good enough for exponentially growing databases.

  • 1
    When you say "exponentially growing", do you mean the number of rows will double every day? Because you'll quickly find no database will keep up with that level of growth. Can you expound upon your requirements a bit? – Max Vernon Dec 5 '19 at 16:19
  • @MaxVernon Well, I mean that I don't know how the rows will grow but it should grow quite fast until a certain moment, where it will grow less fast – TheOneMusic Dec 5 '19 at 16:24
  • 2
    how many rows do you want to store? – Max Vernon Dec 5 '19 at 16:25
  • @MaxVernon I would say a few thousands – TheOneMusic Dec 5 '19 at 16:26
2

Based on your comment of only storing a few thousand rows, you should be fine with almost any database product. Most relational databases only start to have trouble when they're many orders of magnitude bigger, like TB's of data.

Keep in mind some free offerings, like SQL Server Express Edition do have size limits, but as you have outlined, you won't hit those. SQL Server Express Edition has a 10GB max data size, LINK: https://www.microsoft.com/en-us/sql-server/sql-server-2019-comparison#CP_StickyNav_1.

Found the limits on your chosen platform, https://www.sqlite.org/limits.html

Maximum Database Size

Every database consists of one or more "pages". Within a single database, every page is the same size, but different database can have page sizes that are powers of two between 512 and 65536, inclusive. The maximum size of a database file is 2147483646 pages. At the maximum page size of 65536 bytes, this translates into a maximum database size of approximately 1.4e+14 bytes (140 terabytes, or 128 tebibytes, or 140,000 gigabytes or 128,000 gibibytes).

This particular upper bound is untested since the developers do not have access to hardware capable of reaching this limit. However, tests do verify that SQLite behaves correctly and sanely when a database reaches the maximum file size of the underlying filesystem (which is usually much less than the maximum theoretical database size) and when a database is unable to grow due to disk space exhaustion.

You should be just fine in terms of size. The stuff about dynamic data typing is certainly interesting, and something you should be aware of.

| improve this answer | |
  • Okay, so you think I can continue using python sqlite3? – TheOneMusic Dec 5 '19 at 19:37
  • 1
    Yes, it should be fine. I have no experience with it directly, but if I were you, I would look over the FAQ, I see some interesting things there, like it ignoring the max size on strings (VARCHAR). But nothing should stop you from continuing to use it. sqlite.org/faq.html#q2 – BCM Dec 5 '19 at 19:59
  • Okay thank you ! – TheOneMusic Dec 5 '19 at 21:10
0

You've got to define what best means for you.

The best database product is Oracle or SQL Server, depending on who you ask. These have licensing costs, and you're paying for features that are pretty awesome - if you use them.

If you can't name any features of Oracle or SQL Server that you think are worth paying for, then I'd recommend going with an unlicensed product, such as Postgres.

| improve this answer | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.