Dell PowerEdge 2950 with two 1 Gbps NIC's going to two 1 Gbps ports on a switch which then goes to a NetApp with four 1 Gbps NIC's that present as one virtual interface. 24 drives, 7200k SATA, NetApp RAID-DP. I've mapped each host NIC to the NetApp using MPIO in the Microsoft iSCSI initiator. Testing with SQLIO my write throughput appears reasonable at about 200 MBs, but my reads are closer to 100 MBs.

Shouldn't my reads be closer to 200 MBs just like my writes? Is this a configuration problem or is there a fundamental storage issue I don't understand?

enter image description here

Update: Here are IOPS for the random workload. The reads make sense, though, I'm not sure what to make of 20000 for the writes. SAN cache is 3.2 GB. SQLIO tests are against a 25 GB file.

enter image description here

  • 3
    What's your cache on the NetApp device? Do you have a SAN admin who can pull some metrics for you? We have a NetApp and were able to identify a few issues with a combination of reports and warning logs. Ultimately, our situation was a bad fiber card, but NetApp support was quite helpful in helping us root cause.
    – swasheck
    Commented May 29, 2013 at 15:07
  • 2
    It might be worth looking into the configuration of your aggregates and volumes to make sure your disks are being used correctly (feel free to post your configuration, though I'm not sure how many of us are NetApp experts). It's normal for writes to be faster than reads, because writes can be cached on the filer before they are pushed to disk, but reads have to hit the disk unless they're already in the cache. Commented May 29, 2013 at 15:15
  • 2
    @mrdenny Where does this "99% of IO in 64k blocks" notion arise from? Bob Dorr indicates otherwise, as does Wes Brown. Even if we were to ignore those two comprehensive articles, surely common sense dictates that you're going to see 8K IO on a platform that uses an 8K page size. Commented Jun 10, 2013 at 15:32
  • 2
    @mrdenny Mine must be broken then, should I call support? I'm sat here looking at data file IO activity with process monitor and while there is the expected 64K read in abundance, there is plenty of other 8K multiple reads and of course lots of 8k write. Log activity is as expected, 512 byte multiples ranging from single 512 byte writes through to 60k. Commented Jun 10, 2013 at 16:49
  • 2
    @MarkStorey-Smith In my experience reads in 8k occur typically correlated with fragmentation. Also could indicate memory trashing, low page lifetime due to scans evicting pages (ie. most of the extent is still in memory). A well tuned system should show 64k reads. Writes of course depends on what is actually dirty. Commented Jun 10, 2013 at 17:08

2 Answers 2


Disk writes are actually going to memory (NVRAM) on the filer, to be flushed to disk later - on an idle filer, these will be incredibly fast, and iops of 20,000 are quite believable (you'll see similar speeds from most SSDs).

Reads, on the other hand, need to come from disk unless they already happen to be in the filer's read cache (which, unlike writes, are on volatile memory).

It's hard to pin storage vendors down on iops for spinning disks, but for a 7200RPM drive, 80-120 iops is quite believable. Considering that you've probably lost a couple of disks to NetApp's RAID-DP and/or spares, 2,200 iops is close to what you could expect from 22 disks performing around 100 iops each.

This may not explain your read speeds (your disks may not be doing the full 2200 iops when you're performing a sequential read), but it may at least help explain your write performance.

  • Thanks Nathan. Should I be expecting double the throughput with two NIC's and MPIO?
    – Henry Lee
    Commented May 29, 2013 at 17:20
  • 1
    Are you able to check the utilisation on your filer while you're running your sequential read tests? If it reaches 100%, your bottleneck for those is likely to be at the filer (either due to configuration or iops limitations on each disk) and MPIO / extra MPIO connections won't add anything. Your write throughput might increase further. Commented May 30, 2013 at 0:05

For posterity, after much trial and error, we figured out how to get the expected throughput.

As mentioned above, the NetApp had one virtual interface backed by four physical NICs. The host has two NICs and I had configured MPIO through the MS iSCSI Initiator so that there was a path from each NIC to the one virtual interface. The results were the throughput above - writes made sense at close to 200 MB or the speed of two NICs, but the reads were half that or the speed of one NIC.

Upon closer inspection, our SAN guy noticed that traffic was only flowing through one of the physical NICs for the reads. I'm not sure if there was a configuration mistake on our end, but there were two things we tried and both got us our throughput. One was to change from one virtual interface backed by four NICs to two virtual interfaces, each backed by two NIC's. Then map one host NIC to one virtual interface. The other thing we tried was to use "aliasing" on the SAN side to present multiple virtual interfaces. (I'm not a SAN guy, so hopefully I said that correctly.)

My take-away is that we just needed the SAN to present more than one interface so the Initiator truly saw multiple paths. Here is our throughput now:

enter image description here

  • why are the smaller writes slower now? Commented Jun 10, 2013 at 17:51
  • Not sure, we haven't been able to figure that out yet. I will post back if I figure it out.
    – Henry Lee
    Commented Jun 10, 2013 at 19:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.