This Storage Intelligence (StorInt™) dispatch covers Microsoft Exchange 2010 Solution Review Program (ESRP v3.0 results.  Unfortunately, since we last discussed this activity there have been no new submissions in any category. So similar to our previous discussion on SPECsfs2008 from a couple of monthes ago, we have elected to show our latest ESRP ChampionsChart™ that indicate our view of the best performing storage systems in each mailbox category.
Latest ESRP V3.0 results
We start our discussion with our overall ESRP ChampionsChart.
Figure 1 Overall ESRP ChampionsChart
For those of you unfamiliar with our ChampionsChart here is a cheat sheet (if you have seen this before, please skip to the next chart):
- ChampionsCharts depend on published performance data and the ESRP charts are based on the Microsoft Exchange Solution Reviewed (ESRP) program Jetstress run results that are available online.
- For ESRP on the vertical axis we measure system log playback performance. In Jetstress Exchange simulation, this is how fast the storage system can playback a given set of logs against their data base configurations. We believe this is less dependent on mailbox counts, IO rates and other variables that are ever present in ESRP Jetstress runs. (For more on why we prefer Log playback to database transfers per second and other metrics, please read our prior ESRP dispatches, also available online.)
- The horizontal is a measure of database responsiveness. At the moment we are using database read latency figuring this is a bit more important than the other latencies reported by Jetstress.
Neither axis is an absolute measure, instead we use a relative measure for the system performance within their group. On the chart above we show all three ESRP mailbox categories Over 5000, from 1001 to 5000 and 1000 and under mailbox solutions.
There are four quadrants on our ChampionsChart. They are as follows:
- Champions – these storage systems have the best log playback and database latency within their group of storage systems
- Sprinters – these storage systems have the best database latency but the relatively poor log playback times
- Marathoners – these storage systems have the best log playback times but relatively poor database latencies.
- Slowpokes – these storage systems have the worst log playback times and database latencies within their group of ESRP solutions.
Next, we identify some of the storage systems in each mailbox category.
Figure 2 ESRP ChampionsChart for over 5000 mailbox Exchange solutions
In Figure 2 we see only the over 5000 mailbox storage solutions. There are a gaggle of storage systems in the Champions quadrant. Here we identify the top 5 systems that stand out over all the rest and they are readily separated into 3 groups. Specifically;
- Group 1 is a single system and is the IBM DS8700, which had both best relative log playback and database latency in their group of storage systems. The IBM DS8700 was configured to support 20,000 mailboxes.
- Group 2 represents four storage systems and both are the HDS USP-V (the previous generation of HDS VSP enterprise class storage) and two HDS AMS 2100 storage systems (the previous generation entry-level, mid-range class storage systems from HDS) which did relatively worse on both Log playback and database latency than the IBM DS8700. The two USP-V systems were configured for 96,000 and 32,000 mailboxes respectively. The two AMS 2100 storage systems were configured to support 17,000 and 5,800 mailboxes respectively.
- Group 3 represents a single storage system and is the HP4400 EVA configured with 6000 mailboxes.
Figure 3 ESRP ChampionsChart for from 1001 to 5000 mailbox Exchange solutions
Figure 3 shows the ChampionsChart for the mid-range Exchange solutions between 1K and 5K mailboxes. Here there’s really only 4 storage systems in the Champions quadrant. We identify two of these systems below:
- The HP StorageWorks 2000sa MSA storage system (a previous generation P2000 storage system, using SAS storage interface for host access) configured to support 4,000 mailboxes generated the best relative Log playback times with respectable database latency.
- The HP StorageWorks P2000 G3 10Gbe iSCSI MSA (the latest generation P2000) provided the best relative database latency with respectable Log playback time and was configured to support 3,200 mailboxes
Figure 4 ESRP ChampionsChart for 1000 and under mailbox Exchange solutions
In Figure 4 we show the systems for the smallest ESRP category, the 1000 mailboxes and under storage solutions. There are fewer submissions here and as a result fewer Champions as well.
The EMC VNXe3100 storage system configured to support 500 mailboxes using the iSCSI storage protocol was the best overall system with superior relative Log playback and very nearly the best relative database latency in this group.
We guess that Microsoft is releasing a new version of Jetstress to support Exchange Server 2012 I/O simulation. This is probably the reason for a lack of recent submissions to ESRP over the past quarter. The same thing occurred when transitioning between Exchange 2007 and Exchange 2010. That time it took a couple of quarters before we started seeing any substantive submissions on the new level of ESRP/Jetstress. So for now our ChampionsChart will have to do.
A more complete explanation of the ESRP Champions and for that matter SPC-1 and SPC-2 ChampionsCharts are contained in our recently updated (August, 2016) SAN Storage Buying Guide available for purchase on our website. [And a more detailed analysis of ESRP performance can be found in our newly revised (July, 2016) SAN-NAS Storage Buying Guide, also available for sale on our website.]
Reiterating sentiment from previous reports, there are more mysteries in ESRP results than in any other performance results we examine. Also, ESRP/Jetstress results seem designed to be difficult to compare but in our view, merit the effort. Indeed, Microsoft would not claim that ESRP is a benchmark at all but we feel it’s as true a view of storage performance as any we examine.
Please, feel free to contact us with any constructive ideas on how to improve our ESRP analysis. We are always open to change our views if that will lead to a clearer picture of system performance. In that regard, our contact information can be found in the footer below or on our website at SilvertonConsulting.com.
If you received this report from someone else, please signup using the QR code below right to receive your own free monthly copy. We try to analyze ESRP results once/quarter when there’s new information to review. In the other months of the quarter we focus on SPC-1 and SPC-2 or SPECsfs2008 results.
[This performance dispatch was originally sent out to our newsletter subscribers in November of 2012. If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) and we will send our current issue along with download instructions for this and other reports.
Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community.
 ESRP results from http://technet.microsoft.com/en-us/exchange/ff182054.aspx, as of 25 November 2012
 Please see http://silvertonconsulting.com/cms1/product/san-storage-buying-guide/ if interested