I've recently started working with DynamoDB, and most of my experience is in traditional data warehousing (e.g. sql server). One common pattern which I saw when working with SQL server was that we would sometimes separate "active" data from "historical" data, so we would have one table with last months or years data (the time interval which most of our queries were based on), and then another table with the rest of history, for occasional retrieval. We would do this improve query performance.

Now my question is does this design pattern make any sense when working with i.e. DynamoDB with a well partitioned key. So if I've partitioned my DynamoDB by customer id + time, and I'm occasionally interested in getting each customers consumption (whenever they check their dashboard in the app) during the past month or so, does it make sense to have separate tables for past month, and rest of history? As I've understood with partitioning it doesn't even consider the, or need to search for data from the partitions that contain data outside of my search range (since time was part of the partition key), but this is just my assumption. Query speed is of utmost importance, and the data I'm working with spans multiple years.

  • Just curious, how much data we talking?
    – J.D.
    Apr 13 at 11:10
  • 1
    Around 15mil json records daily, with 10 columns each
    – Anton
    Apr 13 at 11:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.