I would like to observe some cryptocurrency prices and for that, I want to store a string, everey second in a database for ca. 3 days. The string will vary in length, it will be 2500 characters max. (complete orderbook), sometimes just 50 (prices). I want to analyze the prices later in Python.

Several questions arise for me: 1) Which database would you recommend? I have worked with SQLite and like it so far, but I am unsure whether it's the most efficient solution, as file sizes get large and reading might be slow. Are there more appropriate solutions? 2) If the string has a special format, e.g. JSON, should my choice from quesiton 1) change - are there special databases which fit well with my use case?

Thank you very much in advance!

closed as too broad by mustaccio, Marcello Miorelli, kevinsky, John Eisbrener, MDCCL Jul 25 at 1:13

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 3
    There are 259,200 seconds in 3 days with a maximum of 648MB of actual data. Any popular database can handle this much data efficiently enough. – Alex Jul 24 at 7:30

Hi and welcome to dba stack exchange.

While I don't think this level is suited for dba.stack exchange, since we're talking a single table with a single column. It's not very complex databasing.

However to go through the hoops a bit, if it's every second a single string being inserted, we have 3600 seconds in an hour, times 24 is 86400 per day. For 3 days straight that's 259200 records in total. Which is nothing in database terms (especially since it's 259200 single column records).

2500 Characters still fits inside a varchar (or nvarchar if unicode is a problem) in both sql server and mysql for instance. Going with SQL server for the moment, a single varchar of 2500 characters is 2502 bytes big, since the varchar datatype is 2 + n with a max of 8000 chars.

So if we assume all records are their maximum of 2500 characters, the total size of the data would be 259200 * 2502 which is 648518400 Bytes, and thus ~618.5 MB (true conversion) or ~648.5 MB (cheeky conversions).

If for some reason unicode is desired and/or needed (datatype nvarchar) the size is roughly twice as big but it still won't hit the character limit for the datatype itself. These sizes are peanuts for database systems.

Since it doesn't seem you want to connect data sets or do much else, other than store.. it doesn't really matter what you pick. Storing data as plaintext is something they can all do. You might even wonder why even store it in a database, why not just a file.

Not the answer you're looking for? Browse other questions tagged or ask your own question.