This dispatch covers Microsoft Exchange Solution Review Program (ESRP)[1] performance results for the over 1000 to 5000 mailboxes results category. Prior reports discussed the over 5000 mailboxes and 1000 and under 1000 mailboxes result categories[2].
ESRP was never intended to compare subsystem performance but rather as a proof of concept for Microsoft and storage vendors to depict a configuration supporting a given workload. Hence, any comparisons necessarily come with some caveats and may not be real. Nonetheless, SCI feels comparisons can well serve both the vendor and end-user storage community and thus, worth noting.
Latest ESRP V2.1 results
Our first chart is new to this analysis and shows aggregate database transfers per second per physical spindle. An astute reader requested we include performance per spindle. This metric doesn’t correlate well to any other ESRP performance parameter. Note, some of these subsystems use only 4 spindles (e.g., both AX4-5i) while #1 Netapp FAS2040 used 12 and #5 Dell MD1000 used 20.
(SCIESRP100128-001) (c) 2010 Silverton Consulting, All Rights Reserved
The nice thing about transfers per spindle is the wide range of subsystems that perform well on it, e.g., 5 of these subsystems are iSCSI attached, 3 are SAS attached and the remainder are FC. In addition, the number of mailboxes supported spans almost the whole range from 1400 to 5000. We do not show the speed of the drives (15 or 10Krpm) or their interfaces. Nonetheless, if you want to attain the most from a set of spindles one would do well by going with either the FAS2040 or AX4-5i.
(SCIESRP100128-002) (c) 2010 Silverton Consulting, All Rights Reserved
Figure 2 shows the total number of database operations per second done by each subsystem. This is mostly correlated to the number of mailboxes but the HP MSA60 seems to do quite well for only having 2500 mailboxes. Even so the Dell and HP MSA70 dominate these top 10 results. HP MSA2000sa G2 and Fujitsu ETERNUS DX80 were the only new additions to this chart.
(SCIESRP100128-003) (c) 2010 Silverton Consulting, All Rights Reserved
Figure 3 is similar to Figure 2 except we have normalized subsystem database transfers/second performance by number of mailboxes (actual 1000 mailboxes). If it weren’t for the Dell MD1120, the top 5 would all be HP MSA storage. New entries to this chart are the HP MSA2000sa G2, Fujitsu ETERNUS DX80 and the EMC Celerra NS-480. As discussed in prior reports, normalized results may or may not scale up beyond their actual mailbox counts reported. For example, the subsystem results for 2KMbx may not hold up when pushed to support 5KMbx.
(SCIESRP100128-004) (c) 2010 Silverton Consulting, All Rights Reserved
Next, we show the Top 10 database backup throughput results. Both Dell’s MD1120 and MD1000 did well in this category. Once again the two new subsystems on this chart were the Fujitsu DX80 and the HP MSA2000sa G2. We like this chart because it’s a good surrogate for raw subsystem read throughput (although it’s database reads). For subsystems in this mid-range category to break 1GB/second seems very impressive.
(SCIESRP100128-005) (c) 2010 Silverton Consulting, All Rights Reserved
Figure 5 Top 10 normalized log write/second
Finally, we show the aggregate log writes per second chart. This view of subsystem performance shows the write IOPs that each subsystem can perform. The new subsystems on this chart include HP MSA2000sa G2, NetApp FAS2040, Fujitsu ETERNUS DX80, and the EMC Celerra NS-480. We have a normalized view of this activity but it looks almost the same and does not show the top end nearly as well.
Conclusions
From our perspective, ESRP results in this mid range category seem to be getting more competitive. There were 6 new ESRP results in this category over the last 9 months, and at least 2 over the last quarter. In almost every chart one can see at least 2 and in most cases 3 or more new results showing up in the top 10.
We have always liked ESRP results because they show a real worldview of subsystem performance. Additionally, there seems to be much more willingness on the part of vendors to submit results to ESRP than some of the other, standard benchmarks. Also, iSCSI, FC and SAS attached storage results are available. Given all that, it’s a great way to compare subsystem performance.
ESRP/Jetstress results are inherently difficult to compare but are worth the effort in our view. Our next ESRP/Jetstress report will return to the 1K mailbox and under tier. We added a new database transfers per spindle chart based on feedback we received and continue to welcome any feedback on how to do better. As such, feel free to contact us with any ideas, our contact information can be found below.
This performance dispatch was sent out to our newsletter subscribers in January of 2010. If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) or subscribe by email and we will send our current issue along with download instructions for this and other reports. Also, if you need an even more in-depth analysis of SAN storage system features and performance please take the time to examine our SAN Storage Briefing available for purchase from our website.
A PDF version of this can be found at
SCI 2010 January 28 Analysis on recent ESRP results (PDF 773.6 KiB)Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community.
[1] ESRP results from http://technet.microsoft.com/en-us/exchange/bb412164.aspx, as of 28 January 2010
[2] All prior SCI ESRP Dispatches can be found at http://silvertonconsulting.com/cms1/dispatches/
also did not change for this update and are dominated by IBM SVC, Fujitsu ETERNUS 8000 and IBM DS8300 (See Figure 
