I need to develop a production system capable of storing a 100 kb textfile and a serial number for every piece produced. I know the approximate output over 2 years, the time the data has to be stored, and have calculated a need of around 25-30 TB of storage.
I have some experience working with databases, but handling that amount of data is new to me.
My first guess would be to simply store the files as a blob inside a table in the database. What are the problems of storing that amount of data in a single table? What are possible backup scenarios?
Another approach would be to store the data in several tables (e.g. a table per month).
A third approach would be to store references to the data in SQL Server, zip the files, and store them somewhere else.
The purpose of this storage system is to query the text file by using the serial number.
What would you do?