SCI latest Microsoft ESRP performance results as of Oct. 2018

In ESRP, ESRP v3/Exchange 2010, ESRP v4/Exchange 2013 by AdministratorLeave a Comment

This Storage Intelligence (StorInt™) briefing covers Microsoft Exchange 2010 and Exchange 2013/2016 Solution Review Program (ESRP) results[1]. Unfortunately, there have been no new ESRP submissions for two quarters. So instead of new submissions, below we provide some performance charts we seldom, if ever, have shown before. 

ESRP V3.0 & 4.0 (E2010&E2013/2016) performance

First up, an SCI scatter plot for database transfers per second vs. drive counts for 15K RPM, 10K RPM and 7.2K RPM disk drives. Note: these plots show disk-only and over 5000 mailbox submissions. 

Figure 1ESRP Scatter Plot: DB xfers/s vs. drive count for 15K, 10K &7.2K RPM drives for over 5K mailboxes 

First, the lines in Figure1 show a linear regression based on the set of points plotted and the regression (R2) coefficient isn’t great, except for the 10Krpm drives. In the above plot, one can see that 15K RPM drives perform better for database transfers per second than 10K RPM drives and that these drives perform better than the 7.2Krpm drives. This seems logical as 15Krpm drives should have better random-access time performance. Also, for Figure 1, please note that the submissions include all host protocols (especially FC, iSCSI and SAS). So, protocol type may be confounding these results.

In Figure 2 we show a similar plot to Figure 1 only this time we use total database backup througput (MB/sec) vs drive counts.   

Figure 2 ESRP Scatter Plot: Total DB backup vs. drive count for 15K, 10K and 7.2K RPM drives, for over 5K mailboxes

For Figure 2 the regression coefficients are poor at best, so take everything from this point on as tentative at best. But here the 10K RPMdrives performance better than the 7.2K RPM drives which performed almost 2X better (Y=17.47x vs Y=8.637) than 15K RPM drives. We suspect that both the 10K and 7.2K RPM drives are newer and as such, had better sequential throughput than the 15K RPM drives. Since the emergence of flash, 15K RPM drives have lost a lot of relative market share and as such, have not been as updated as 10K and 7.2K RPM drives. 

In Figure 3, we display yet another scatter plot, only this time we show log throughput (KB/sec) vs. drive counts. Note: ESRP reports log disk writes/sec, average log write access time and average time to play one log file (in sec., averaged over 500 [1 MB] log files) for a single storage group. We convert average time to play one log file to KB/sec by taking the reciprical of time to play one log file and multiplying by 1000.

Figure 3 ESRP Scatter Plot: Log throughput (KB/s) vs. drive counts for 15K. 10K and 7.2K RPM drives for over 5K mailboxes

In Figures 3, once again the regression coefficients are the worst yet, barely registering in positive values. This says that there’s a lot of variance in the results. That being said we can still look at the trend lines as an (very) rough indicator of disk drive performance. With log throughput, we see yet another view of Exchange IO performance. This time trend lines show the 10K RPM drives were better than 15K RPM drives which were better than 7.2K RPM drives. 

We have always preferred log playback because it’s a mixture of sequential throughput (reading the log file) as well as random access (updating the database). We believe that Figure 3 results show that the 10K RPM drives provide both great sequential throughput AND good random (transactional) throughput. Whereas the 15Krpm drives provide poor sequential throughput AND great random throughput and the 7.2Krpm drives provide good sequential throughput AND poor random throughput. Again, we are assuming that the 15K RPM drives are older than the 10K and 7.2K RPM drives. 

Summary

Exchange may be becoming less of a concern for storage vendors. We will need to see a few more quarters of no new submissions to be sure. Indeed, as this is the 2ndquarter in a row, if we see another two without new submissions, we may need to start looking for a different benchmark to report on. If any reader has a suggestion as to what we should examine instead, please do let us know. 

However, having no submissions again does provide an opportunity to reveal some comparisons or charts we don’t normally show. Hopefully, you found this report worthwhile, even without any new submissions.

If you are interested in even more information on block storage performance please examine our SAN Storage Buying Guide available for purchase on our website, which includes ChampionCharts™ for OLTP, Throughput and Email workloads across enterprise, mid-range and SMB storage systems and now includes an all flash array (AFA) ChampionsChart for OLTP. Use the QR code below left for more information on the SAN Storage Buying Guide. We also provide some additional ESRP results analysis in our recently updated, SAN-NAS Buying Guide, also available on our website.

Constructive comments on how to improve our analyses for Microsoft ESRP (Exchange) or any of our performance reports are always welcome. Moreover, if you detect errors in this or any of our other performance reports, please do let us know and we will correct them as soon as possible. Silverton Consulting, Inc., is a U.S.-based Storage, Strategy & Systems consulting firm offering products and services to the data storage community.

[This performance dispatch was originally sent out to our newsletter subscribers in October of 2018.  If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) and we will send our current issue along with download instructions for this and other reports. Dispatches are posted to our website at least a quarter or more after they are sent to our subscribers, so if you are interested in current results please consider signing up for our newsletter.]  

[Also we offer more block storage performance information plus our OLTP, Email and Throughput ChampionsCharts™ in our recently updated (May 2019) SAN Storage Buying Guide, or for more information on protocol performance results please see our recently updated (May 2019) SAN-NAS Storage Buying Guide, both of which are available for purchase on our website.]

Silverton Consulting, Inc., is a U.S.-based Storage, Strategy & Systems consulting firm offering products and services to the data storage community


[1]ESRP results from https://technet.microsoft.com/en-us/office/dn756396.aspxas of 29Oct2018

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.