First and foremost you should assess the impact of fragmentation. Much too often fragmentation is painted as the ultimate evil cause of all the server problems w/o any consideration of its actual impact. Fragmentation impacts several aspects:
- Slow down scans due to impact on read-ahead efficiency and small IO size (page vs. extent)
- Less efficient IO due to low rows per page because of frequent splits
- Lower buffer pool cache hit ration (ie. lower page life expectancy) due to same as above
- Higher cost impact of INSERTs due to frequent splits
All above are bad, but here is the real deal: you can have 97% fragmentation w/o any of the above symptoms being present. High fragmentation can indeed cause all these problems, but only if your specific workload actually interacts with fragmentation in a manner that causes these symptoms to appear.
I would recommend using Waits and Queues methodology to locate your actual performance bottlenecks and asses them appropriately. You may very well confirm that fragmentation has a serious impact and that would require schema and possibly application changes to address the problem, but you may discover that the problem lies somewhere else and the solution is completely different.
And lastly: you may also discover that $5k spent on more RAM and a few good SSDs alleviate completely the problems you have. Before you shrug in horror at such a remedy (what, no root cause being addressed? anathema!) consider the risk and costs associated with an application change...