I am a little out of my league with this scenario and hope this community can help get me on the right direction.
I have recently been tasked with re-designing and implementing a database application where the company I work for processes a large quantity of text files (sizes ranging from a 100MB to 3GB). We currently process approximately 600 of these files per year. This data needs to be accessible after our initial processing.
The current implementation has each file being imported via Application bulk load or SSIS to a "temp" database as individual tables. I would like to keep this process in place as it gives a good entry point for when manual entry into processing is needed.
I am in the process of trying to create a new MS SQL Server installation and architecture, and am looking for advice/articles which would cover a similar scenario as to best practices (partitioning, filegroups, etc.). I have worked in databases for a few years, but not at this level.
So if you have any experience in this type of scenario or know of a good article or book you could point me to, I would be extremely grateful. I just need a nudge in the right direction. Thank you in advance.