May 302013
 

This Storage Intelligence (StorInt™) dispatch covers Microsoft Exchange 2010 Solution Review Program (ESRP) v3.0 results.[1] Over the last quarter, in the 1,000 and under mailbox category being reviewed here, there have been three new submissions. Since we last discussed this category (July 2012) there have been a total of four new submissions and three that have been dropped (HP E5500 and E5300 results). All of our charts have new rankings with all the changes in this category.

Latest ESRP V3.0 results

We start our discussion in Figure 1 with  normalized database transfers per second per 1000 mailboxes (1Kmbx), which we calculate from ESRP primary metrics.

Figure 1 ESRP v3.0 Top 10 Normalized DB Xfers/Sec/1KmbxTop 10 bar chart, normalized database transfers per second reads and writes.

In Figure 1 there are three new submissions over this past quarter which are the HP Prolient DL380p Gen8 server at second, Infortrend EonStor DS S12E-R2140 at fourth, and EMC VNXe3150 at eight place.

The HP server was connected via SAS 2.0 to a P420i Smart array RAID controller with 8-2TB, 7200RPM disks behind it.  The Infortrend storage used iSCSI to connect to the Exchange hosts and had 8-600GB, 15KRPM drives. The EMCVNXe 3150 was connected via iSCSI with 10-900GB, 10KRPM disk drives behind it.  So the top ranked new system used the slowest drives – go figure.

The one caveat for our normalized DB/xfers/sec and just about any other ESRP metric is that Jetstress 2010 parameterizes its IO rates, which in this case ranges anywhere from 0.30 down to 0.12 IO/sec/mbx.  But this doesn’t seem to matter as much here. The fastest IO rate, 0.30 was used by the fourth placed system (Infortrend) and the top three ranked systems used 0.18, 0.24 and 0.18 IO/sec/mbx respectively.

Next, in Figure 2 we turn to transaction latencies.

Figure 2 ESRP v3 Top 10 Database Access LatenciesTop 10 bar chart database read access latency, write access latency and log write latency

In Figure 2, new submissions include the Infortrend EonStor DS S12F-R2851 at fifth place and the previously discussed, Infortrend EonStor DS S12E-R2140 at tenth place. Their DS S12F-R2851 happens to be the only FC submission in this category with most of the rest being iSCSI and two SAS connected storage systems.  It’s interesting to note that the FC submission didn’t beat all the other iSCSI submissions.  There are no SAS connected storage submissions ranked in the top ten for latency.

The FC EonStor had slower disks (8-3TB, 7200RPM) than their the iSCSI EonStor (8-600GB, 15KRPM).  We would have expected a better placement for their iSCSI storage.  The FC EonStor provided very fast write activity (database and log), ranking the best in log write latency and 2nd best (behind NetApp) in database write latency. We would almost suspect flash but none was mentioned. However, the FC EonStor did have 8GB of cache compared to the iSCSI system with 1GB cache and this might explain a lot of the speedup.

In Figure 3 we discuss Exchange log playback.

Top 10 bar chart, log playback timesFigure 3 ESRP v3 Top 10 Log Playback

In Figure 3, the new submissions came in at first (Infortrend iSCSI based EonStor), fourth (Infortrend FC based EonStor), ninth (EMC VNXe 3150) and tenth (HP Proliant SAS storage).

We have always liked log playback because it doesn’t correlate that well with Jetstress’s IO rate (discussed previously).  But I am beginning to wonder about this. It turns out log playback is measured while background database transaction activity executes against the storage. It’s unclear what IO rate is used for this “background” activity.

We suspect it may have something to do with the Jetstress specified IO rate but can’t prove it. If that is the case, a faster IO rate may be detrimental to a high log playback performance. This will take further investigation.

Finally, in Figure 4 we rank the top ten total database backup throughput systems.

Top 10 bar chart, total database backup in MB per secordFigure 4 ESRP Top 10 Total database backup (MB/sec) for 1K and under mailboxes

Figure 4 shows a metric we compute, based on the total Exchange database backup throughput, across all servers and all databases for a storage subsystem. ESRP reports database backup throughput on a “per database “or “per Exchange server” level. But we find from a pure storage perspective, it’s better to understand backup throughput across the whole storage system.

In Figure 4 there are three new submissions at second, third and ninth place. These are the Infortrend FC EonStor, HP Proliant server and Infortrend iSCSI EonStor, respectively, which have all been discussed on other charts.

Similar to the previous discussion on log playback, database backup also occurs with database transaction activity operating in the background. Again, it’s unclear what IO rate is used for this background activity and how that might effect backup throughput results.

Summary

We continue to be surprised by the number of new ESRP submissions, which continues at a healthy clip (16 in this past quarter alone across all categories). We presume that the about to be released Exchange 2013 Jetstress is forcing vendors to clean up all their current submissions to get them published before the new version comes out.

Reiterating sentiment from previous reports, there are more mysteries in ESRP reports than in any other performance results we examine. Also, ESRP/Jetstress storage system performance seems designed to be difficult to compare but in our view, merit the effort.  Note, Microsoft states that ESRP is not a benchmark and should not be used to compare storage systems. But we feel as it drives storage to a peak IO rate and reports on transaction performance, transaction latency, backup performance and log playback, it certainly feels like a benchmark. We would prefer that they choose one IO rate and stuck with it, but then it would be too easy to compare results.

Please, feel free to contact us with any constructive ideas on how to improve our ESRP analysis.  We are always open to change our views, especially if the change can lead to a better understanding and comparison of system performance.  In that regard, our contact information can be found in the footer below or on our website at SilvertonConsulting.com.

For a more complete ranking of current top storage system performance using explanation of ESRP, SPC-1 and SPC-2 results, please see our recently updated (August, 2014) SAN Storage Buying Guide or for more information on ESRP performance see our newly revised (October, 2014) SAN-NAS Buying Guide, both available for purchase on our website.

If you received this report from someone else, please signup for our free monthly newsletter using the signup form above right to receive your own copy. We analyze storage system performance results/benchmarks each month and will analyze current Microsoft ESRP results again in three months.

~~~~

[This performance dispatch was originally sent out to our newsletter subscribers in May of 2013.  If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) and we will send our current issue along with download instructions for this and other reports. Dispatches are posted to our website at least a quarter or more after they are sent to our subscribers, so if you are interested in current results you should signup for our newsletter.]  

Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community.