Is there any benefit to defragmenting SQL indexes in a SAN environment?

Question

Our SQL server lives on a SAN. It contains dozens of OLTP databases, some with several tables containing over 1m records.

We have been running Ola Hallengren's index maintenance scripts weekly, and it runs for several hours each time. Based on the fragmentation threshold, the script will either reorganize or reindex an index. We have observed that during reindexing, the log files get huge which leads to an excessive consumption of bandwidth during the log shipping.

Then comes an article from Brent Ozar in which he says to stop worrying about SQL indexes:

Your hard drives are shared with other servers that are also making drive requests at the same time, so the drives will always be jumping all over the place to get data. Defragging your indexes is just meaningless busy work.

Googling this question leads to varying opinions, most supported with arguments that seem too brief or weak. Our tentative plan is to adjust the fragmentation threshold in our maintenance script so that it reorganizes much more often than it reindexes.

What is the final verdict? Is it worthwhile to defrag SQL indexes on a SAN considering the burdens associated with running weekly maintenance jobs?

Answer 1

Defragmentation strategies help improve scan speed to/from disk.

The wide variety of opinions is because an environment's ideal defragmentation strategy should depends on many different factors. There are also multiple potential layers of fragmentation in play.

Saying that your databases are stored on a SAN isn't enough information. For example:

• Are database files stored on separate physical RAID groups or the same RAID group? What other processes are active on that same device? Are your backup files ending up there, too? You may have to ask your SAN admin for this information, because it's not always transparent.

• What are the access patterns for the databases? OLTP is generally random access, but sometimes an application is table-scan-happy and you can't change its behaviour (ISV app). Are the applications read-mostly, write-mostly, or somewhere in between?

• Are there performance SLAs in play during a recovery/failover period?

Brent's post assumes there is one giant pool of storage and everything shares it. This means the physical disks are rarely idle, and hence most access is random. If that is your situation, then the advice applies, and I agree with it for the most part. While this type of strategy is much easier to manage, it isn't necessarily (a) what you have in your environment, or (b) what is the best solution for your environment.

If index maintenance is burdensome, consider doing it less aggressively, and/or amortize the cost over the week (i.e., run light maintenance once/day, instead of heavy maintenance once/week).

You can also turn on the SortInTempdb option to potentially reduce the amount of logging that takes place in the user databases.

Answer 2

Ideally, you should be reorganizing/reindexing for ONLY those indexes that need attention, otherwise you are wasting resources and potentially causing other problems.

You need to establish a performance baseline and whenever you make changes compare the performance change against the baseline to determine if your change is worth implementing.

Answer 3

Okay, the question is about Database Indexes, which are a construct of a file or set of files. Reading the answers above would lead a person to believe that we are talking about fragmentation at the disk level and not the indexes inside of a file. These totally separate subjects.

The myopic approach here is will performance when retrieving data within and OLTP Database improve if Indexes are De-fragmented or Rebuilt. The answer is YES! However, it is important to note that disk fragmentation is also a factor.

Lowest "cost" overall? Do your Database Maintenance. Second lowest cost, detach the database, move it somewhere else, re-format your disks and follow best practices for Disk Partition Alignment http://msdn.microsoft.com/en-us/library/dd758814.aspx. Last but not least, use a 3rd party advanced defragmentor like Diskkeeper.

Keep in mind, this is ONLY recommended for NTFS type storage (e.g. Windows OS) and this is not an endorsement for any product nor am I affiliated with Condusiv Technologies or its subsidiaries.