Latest ESRP 1K-5K mailbox DB xfers/sec/disk results – chart-of-the-month

(SCIESRP120429-001) 2012 (c) Silverton Consulting, All Rights Reserved

The above chart is from our April newsletter on Microsoft Exchange 2010 Solution Reviewed Program (ESRP) results for the 1,000 (actually 1001) to 5,000 mailbox category.  We have taken the database transfers per second, normalized them for the number of disk spindles used in the run and plotted the top 10 in the chart above.

A couple of caveats first, we chart disk-only systems in this and similar charts  on disk spindle performance. Although, it probably doesn’t matter as much at this mid-range level, for other categories SSD or Flash Cache can be used to support much higher performance on a per spindle performance measure like the above.  As such, submissions with SSDs or flash cache are strictly eliminated from these spindle level performance analysis.

Another caveat, specific to this chart is that ESRP database transaction rates are somewhat driven by Jetstress parameters (specifically simulated IO rate) used during the run.  For this mid-level category, this parameter can range from a low of 0.10 to a high of 0.60 simulated IO operations per second with a median of ~0.19.  But what I find very interesting is that in the plot above we have both the lowest rate (0.10 in #6, Dell PowerEdge R510 1.5Kmbox) and the highest (0.60 for #9, HP P2000 G3 10GbE iSCSI MSA 3.2Kmbx).  So that doesn’t seem to matter much on this per spindle metric.

That being said, I always find it interesting that the database transactions per second per disk spindle varies so widely in ESRP results.  To me this says that storage subsystem technology, firmware and other characteristics can still make a significant difference in storage performance, at least in Exchange 2010 solutions.

Often we see spindle count and storage performance as highly correlated. This is definitely not the fact for mid-range ESRP (although that’s a different chart than the one above).

Next, we see disk speed (RPM) can have a high impact on storage performance especially for OLTP type workloads that look somewhat like Exchange.  However, in the above chart the middle 4 and last one (#4-7 & 10) used 10Krpm (#4,5) or slower disks.  It’s clear that disk speed doesn’t seem to impact Exchange database transactions per second per spindle either.

Thus, I am left with my original thesis that storage subsystem design and functionality can make a big difference in storage performance, especially for ESRP in this mid-level category.  The range in the top 10 contenders spanning from ~35 (Dell PowerEdge R510) to ~110 (Dell EqualLogic PS Server) speaks volumes on this issue or a multiple of over 3X from top to bottom performance on this measure.  In fact, the overall range (not shown in the chart above spans from ~3 to ~110 which is a factor of almost 37 times from worst to best performer.



The full ESRP 1K-5Kmailbox performance report went out in SCI’s April newsletter.  But a copy of the full report will be posted on our dispatches page sometime next month (if all goes well). However, you can get the full SPC performance analysis now and subscribe to future free newsletters by just sending us an email or using the signup form above right.

For a more extensive discussion of current SAN or block storage performance covering SPC-1 (top 30)SPC-2 (top 30) and all three levels of ESRP (top 20) results please see SCI’s SAN Storage Buying Guide available on our website.

As always, we welcome any suggestions or comments on how to improve our analysis of ESRP results or any of our other storage performance analyses.

ESRP v3 (Exchange 2010) log playback results, 1Kmbox&under – chart-of-the-month

(SCIESRP111029-003) (c) 2011 Silverton Consulting, All Rights Reserved
(SCIESRP111029-003) (c) 2011 Silverton Consulting, All Rights Reserved

The above chart is from our last Exchange [2010] Solution Review Program (ESRP) performance dispatch released in our October newsletter (sign-up upper right).  The 1K mailbox and under category for ESRP represents Exchange storage solutions for SMB data centers.

As one can see from the above the NetApp FAS2040 has done well but an almost matching result came in from the HP P2000 G3 MSA system.  What’s not obvious here is that the FAS2040 had 8 disks and the P2000 had 78 so there was quite a difference in the spindle counts. The #3&4 runs from EMC VNXe3100 also posted respectable results (within 1sec of top performer) and only had 5 and 7 disks respectively, so they were much more inline with the FAS2040 run.  The median number of drives for this category is 8 drives which probably makes sense for SMB storage solutions.

Why log playback

I have come to prefer a few metrics in the Exchange 2010 arena that seem to me to capture a larger part of the information available from an ESRP report.  The Log Playback metric is one of them that seems to me to fit the bill nicely.  Specifically:

  • It doesn’t depend on the Jetstress IO/rate parameter that impacts the database transfers per second rate.  The log playback is just the average time it takes to playback a 1MB log file against a database.
  • It is probably a clear indicator of how well a storage system (configured matching the ESRP) can support DAG log processing.

In addition, I believe Log Playback is a great stand-in for any randomized database transaction processing. Now I know that Exchange is not necessarily a pure relational database but it does have a significant component of indexes, tables, and sequentiality to it.

My problem is that there doesn’t seem to be any other real database performance benchmark out there for storage.  I know that TPC has a number of benchmarks tailored to database transaction activity but these seem to be more a measure of the database server than the storage.  SPC-2 has some database oriented queries but it’s generally focused on through put and doesn’t really represent randomized database activity and for other reasons it’s not as highly used as SPC-1 or ESRP so there is not as much data to report on.

That leaves ESRP.  For whatever reason (probably the popularity of Exchange), almost everyone submits for ESRP. Which makes it ripe for product comparisons.

Also, there are a number of other good metrics in ESRP results that I feel have general applicability outside Exchange as well.  I will be reporting on them in future posts.



Sorry, I haven’t been keeping up with our chart-of-the-month posts, but I promise to do better in the future.  I plan to be back in synch with our newsletter dispatches before month end.

The full ESRP performance report for the 1K and under mailbox category went out to our newsletter subscriber’s last October.  A copy of the full report will be up on the dispatches page of our website sometime this month (if all goes well). However, you can get performance information now and subscribe to future newsletters to receive these reports even earlier by just sending us an email or using the signup form above right.

For a more extensive discussion of block storage performance in ESRP (top 20) and SPC-1&-2 (top 30) results please consider purchasing our recently updated SAN Storage Buying Guide available on our website.

As always, we welcome any suggestions on how to improve our analysis of ESRP results or any of our other storage system performance discussions.