There are so many things this could be that all I can offer are a few questions to help guide you towards an answer.
Is the DB on a server dedicated to just running SQL Server? If not, other processes might be interferring by stealing precious processor time.
Is the DB server essentially out of memory? SQL Server will attempt to allocate every single byte it can, but if it's at capacity and your queries require more data to be loaded then it has to fallback to using virtual memory, which radically increases the amount of time even simple queries might take.
Is the DB server's network bandwidth to small to handle transferring the data in a timely manner?
At the end of the day, it sounds like the machine you are hosting SQL Server on is under sized for what you are trying to do. It's entirely possible that you have finally reached those hardware limits where performance is dropping off radically. If this is the case (the above questions will help you determine that) then you'll want to move the DB to a server that is properly sized for the amount of data (and queries) you are trying to process.
This could mean using faster processors, faster drives, or just installing more RAM.