2

I am trying to estimate storage requirements for the data our application will store. After doing some research I can pull out data from the stats table. However, I don't understand how those numbers are calculated. For example, if I do a simulation as follows:

  • ~2000 document write:
    • written_bytes_total: 40,668,160
    • data_bytes: 3,804,672
    • metadata_bytes: 8,388,608

Q1. Looking at the above numbers I get roughly 20,334B = 20KB per document, which is quite a bit - the document stored has a six UUID(4) values, 3 dates, and a couple of booleans, but nothing that will require KBs (like text blobs). Is there an overhead per document that RethinkDB adds on?

Q2. How is it that written_bytes is 40MB, but data_bytes under "storage" are 3.8MB? I thought they might be cached (and not persisted), but cache size is 3,253,800B, which also does not add up.

So I guess my question, how can I estimate how much space will be required for 2K/10K/100K documents for a given table in RethinkDB? Which value(s) should I be looking at? I am not looking for byte precision, but would really like to know how those numbers are calculated. Thanks!

Result of running a query on TableA:

r.db('rethinkdb').table('stats')
  .filter(function(row){
      return row('id')(0).eq('table_server').and(
             row('table').eq('TableA'))
})


"query_engine": {
  "read_docs_per_sec": 0 ,
    "read_docs_total": 0 ,
    "written_docs_per_sec": 0 ,
    "written_docs_total": 1974
} ,
  "server":  "MacBook_Pro_local_b0b" ,
  "storage_engine": {
    "cache": {
      "in_use_bytes": 3253800
    } ,
    "disk": {
      "read_bytes_per_sec": 0 ,
      "read_bytes_total": 171520 ,
      "space_usage": {
        "data_bytes": 3804672 ,
        "garbage_bytes": 389632 ,
        "metadata_bytes": 8388608 ,
        "preallocated_bytes": 6291456
      } ,
      "written_bytes_per_sec": 0 ,
      "written_bytes_total": 40668160
    }
  }

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.