SCI 2013Nov27 Latest Microsoft ESRP (Exchange 2013 & 2010) performance results

In 3PAR, Data storage, ESRP, ESRP v3/Exchange 2010, ESRP v4/Exchange 2013, Hitachi Unified Storage, Hitachi Vantara, HUS 110, HUS 130, HUS VM, IBM, StoreServ 7200, XIV, XIV Gen3, XIV Gen3.2 by AdministratorLeave a Comment

This Storage Intelligence (StorInt™) dispatch covers Microsoft Exchange 2010 (E2010) and Exchange 2013 (E2013) Solution Review Program (ESRP) results.[1] Since we last reviewed this 5,000 and over mailbox category, there have been a number of new submissions. For E2013, there have been fourteen new submissions in this past quarter alone in this category, including a HP 3PAR StoreServ 7200, four from Dell Compellent, three from HDS HUS VM, three from Infortrend, and IBM Storwize V5000, V7000 and IBM XIV. We continue to analyze E2010 and E2013 results together as there doesn’t seem to be that much difference in IO performance between the two. Many of our top ten charts have changed since last time.

Latest ESRP V3.0 results

We start our discussion in Figure 1 with normalized (per 1000 mailboxes) total database transfers per second, which we calculate from ESRP primary metrics.

Stacked bar chart showing the top Exchange database transfer per second normalized over 1000 mailboxesFigure 1 ESRP v3.0 Top 10 Normalized DB Xfers/Sec/1Kmbx

In Figure 1 there are four new E2013 submissons starting out at a number one with HP 3PAR StoreServ 7200 achieving a new high of over 1300 database transfers per second per 1000 mailboxes.  Other new entries show up at number five, seven and eight all of which are HDS systems, the HUS 130, HUS VM and HUS 110 respectively.

Next, in Figure 2 we turn to total database backup throughput.

Bar chart showing the top 10 database backup performersFigure 2 ESRP v3 Top 10 total database backup throughput results

In Figure 2, there are four new E2013 submissions starting out at number three with IBM XIV Gen3.2 and the three new HUS VM submissions come in at eight, nine and ten for 120K, 43.2K and 24K mailbox submissions.

The new XIV submission used 4TB disk drives and had 6 TB of SSD cache. Unclear why the older, XIV Gen3 did better on this metric as it used 3TB drives and had no SSD caching. My understanding is that ESRP backup activity is measured as a standalone activity, so pure sequential throughput should be the way to succeed here. XIV’s top submission used 3TB drives with the smallest database size, which may have helped somewhat. But you would think that the other two XIV submissions (second place used 2TB drives) both of which had SSD caching might have done better. All the XIV submissions had the same number (180) of spindles so that shouldn’t have made any difference. And why all the XIV system should provide better backup throughput than any other system is yet another question.

In Figure 3 we show an ESRP total database transfers per spindle scatter plot

Figure 3 ESRP v3 Top 10 database transfers per spindle scatter plotScatter plot showing DB transfers per second per disk spindle count with linear trend lines for 15Krpm, 10Krpm and 7.2Krpm disk drives

In Figure 3, we show the drive RPM against the submissions total database transfers per second.  We also plot the linear regression lines for the 15Krpm, 10Krpm and 7.2Krpm sets and their regression equations. The 10Krpm drive set had the best linear regression coefficient (R**2) at 0.89, but also had the least number of entries; the 15Krpm line had the worst R**2 at 0.50 but tied with 7.2Krpm for highest quantity of entries; and the 7.2Krpm had a middling R**2 of 0.67. The 15Krpm drives provide slightly ( ~9.1%) more database transfers per second then the 10K. The 10Krpm provide much (36.7%) more transfers than the 7.2Krpm drives and the 15Krpm drives provide ~49.2% more database transfers per second than the 7.2Krpm drives, which seem to stand to reason. Why the 15Krpm drives don’t offer much better transfers per second than the 10Krpm is something of a quandary that I don’t understand.

Please note, the total database transfers have a number of factors, many of which can skew results. Probably the one factor that makes the most difference is number of mailboxes but in general more mailboxes require more disk drives so this should be a reasonable way to look at th data. Also, this plot is only for the over 5,000 mailbox submissions, so these should mostly be high end storage systems. However, I did try this plot using normalized database transfers but it yielded much worse regression coefficients. So I think we will stick with this view.

Summary

We are seeing quite a few E2013 submissions nowadays. So far, there doesn’t appear to be any significant advantage to using E2013 vs. E2010 at least from a storage performance perspective so we will continue to combine both results in our performance analysis.

The only top ten charts that have no E2013 submissions are the read latency and log playback charts. Unclear why E2010 submissions would reign supreme for these two metrics but that might be something worthwhile to investigate further if I was on the Exchange performance team.

Reiterating sentiment from previous reports, there are more mysteries in ESRP reports than in any other performance results we examine. Also, ESRP/Jetstress storage system performance seems designed to be difficult to compare but in our view, merit the effort.

Please, feel free to contact us with any constructive ideas on how to improve our ESRP analysis.  We are always open to suggestions, especially if they lead to a better understanding and comparison of system IO performance.  In that regard, our contact information can be found in the footer below or on our website at SilvertonConsulting.com.

For a more complete ranking of current top block storage system performance using our ChampionCharts for Email, OLTP and Throughput results, please see our recently updated (May 2019) SAN Storage Buying Guide or for more detailed ESRP performance results please see our recently updated (May 2019) SAN-NAS Storage Buying guide, both of which are available for purchase on our website.

~~~~

[This performance dispatch was originally sent out to our newsletter subscribers in November of 2013.  If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) and we will send our current issue along with download instructions for this and other reports. Dispatches are posted to our website at least a quarter or more after they are sent to our subscribers, so if you are interested in current results please consider signing up for our newsletter.]  

Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.