I have a dataset that is stored across 20 or so tables, most of which have a few million rows. This data will be generated fresh each month, and compared to past datasets.
I'm pretty new to dealing with datasets of this size, so I'm unsure what would be good practice in storing these past datasets. I need the data to still be accessible to queries, so dumping the database to backup files won't be enough. Is there some feasible way to move each dataset into a new database once I get a new one? Or should I just put timestamps on each row and keep everything in one giant dataset?
Sorry if this is too vague, I'm not sure how exactly to ask this question. I'd be open to changing it if there are any suggestions.