Currently I've been assigned the task to research ways to speed up an existing project. It's going to be rebuild so we can start from scratch, the current implementation has been build by another company and is lacking in performance.

The data isn't all that big except for one table, which is going to grow quite rapidly. I've been looking into sharding and partitioning tables and I can find enough info regading those topics.


Another possible solution would be lowering resolution of data gather for old data.

So for example, the table would contain answer of employees, these employees anser 1-30 questions a day, answers have a numerical part over which averages are made.

I could store only the averages for data of a couple of years old, ie average per month, for data of a couple of years old.

However I can't find any info because I don't know what to google for, does anyone recognize this pattern, or know if this solution is already flawed for whatever reason.

  • It's an Aggregation Pattern; try googling for "Database Aggregation" and you should be able to come up with something to help you on your way. – Michael Sep 12 '13 at 11:02
  • 1
    How many is many? I mean, how many rows are/will be in that fast growing table? – dezso Sep 12 '13 at 13:14
  • @dezso a worst case scenario would be around 500-600k added a day. – Xeli Sep 12 '13 at 13:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.