SCI 2010 Apr 29 First ESRP v3 (Exchange 2010) results analysis – all mailbox counts

In AX4, ESRP, ESRP v3/Exchange 2010, EVA 8400, HPE, IBM, SVC, XIV by Administrator

This dispatch covers Microsoft Exchange Solution Review Program (ESRP)[1] V3.0 for Exchange 2010. There have been only four submissions for ESRP V3.0 and they are the HP 8400 EVA, EMC AX4-5i, IBM SVC 5.1 with DS3400s, and IBM XIV 2810 storage subsystems ranging from 500 to 40,000 mailboxes.  Given the limited submissions we have chosen to report on all results in this dispatch.  Previous reports for ESRP V2.1 and Exchange 2007 are available here[2].

Latest ESRP V3.0 results

As the mailbox counts span such a wide spectrum we have elected to normalize the database access counts to accesses per 1,000 mailboxes (1Kmbx).  Realize that the EMC AX4 supported only 500 mailboxes and may not be able to sustain this workload for the 1000 mailbox we show below.

(SCIESRP100429-001) (c) 2010 Silverton Consulting, All Rights Reserved

(SCIESRP100429-001) (c) 2010 Silverton Consulting, All Rights Reserved

In every case the read database transfer counts are significantly more than the write transfers, in some cases almost 2X the rate.  Unsure why this is and this was not evident in ESRP v2.2.  Both IBM’s XIV and HP’s EVA seem to do considerably better than the other two products in this metric.

(SCIESRP100429-002) (c) 2010 Silverton Consulting, All Rights Reserved

(SCIESRP100429-002) (c) 2010 Silverton Consulting, All Rights Reserved

Figure 2 shows the total number of (read and write) database operations per second per spindle done by each subsystem.  The IBM XIV once again dominates this category.  It’s worth noting that

  • Both the XIV and EMC’s AX4 used 7200RPM 1TB SATA drives whereas the other two systems used 15Krpm 450GB drives
  • The AX4 only had 8 disks
  • The AX4 was attached to the Exchange servers via iSCSI while all the others used FC, and
  • The XIV used only about 40% of its storage for Exchange data whereas all the other systems used 61 to 86% of their storage for data.

One would think short stroking drives like XIV did would show worse performance on a per spindle basis.  However, apparently XIV is able to spread the workload over its higher spindle count to actually attain a better per spindle performance.  This is certainly worth more investigation from our perspective.

(SCIESRP100429-003) (c) 2010 Silverton Consulting, All Rights Reserved

(SCIESRP100429-003) (c) 2010 Silverton Consulting, All Rights Reserved

Figure 3 depicts the average latency for database operations by subsystem.  Here the HP EVA shows up very well.  Unclear why XIV had such a poor showing here but, slower drives probably played some role.  It’s also interesting to see that IBM’s SVC very good write latency.  We would attribute this to advanced write caching and more cache available to absorb write data but in any case it does well with database random writes and very well with log I/O sequential writes.

(SCIESRP100429-004) (c) 2010 Silverton Consulting, All Rights Reserved

(SCIESRP100429-004) (c) 2010 Silverton Consulting, All Rights Reserved

Next, we show the Exchange backup MB/sec per database for the four submissions.  Again the three top systems used FC while the EMC AX4 used iSCSI but other than that there doesn’t seem to be much difference here.

(SCIESRP100429-005) (c) 2010 Silverton Consulting, All Rights Reserved

(SCIESRP100429-005) (c) 2010 Silverton Consulting, All Rights Reserved

Finally, Figure 5 shows the time it takes to play back a 1MB log file against the database.  This chart provides a view of performance for recovery scenarios. Here IBM’s SVC write performance helps it dominate this category.

Conclusions

Exchange 2010 has introduced Data Availability Groups (DAG) and as such, changed some of the redundancy/availability capabilities available in Exchange 2007.  This has forced some subtle modifications to the Jetstress ESRP driver.  For example, when hardware configurations can be split in two equal parts, Jetstress now simulates DAG redundant writes to half of the hardware configuration that would have corresponded to the other half’s database updates.  These redundant writes constitute the only I/O needing to be done to this half of the storage for the other half’s user activity.

As such, vendors can now submit ESRP reports using half the hardware needed to support a given workload as long as its configuration could be cut into identical halves.  This reduction in hardware requirements for ESRP reports should encourage more vendors to submit results.  However, it made for confusing reading when we first saw it.  (If you understand all this on first reading you get an A+.)

As more ESRP submissions appear we will once again break our analysis up into multiple categories but until then we will keep to normalized comparisons as much as possible.  As stated before, please realize that any normalization may imply subsystem performance beyond its present capabilities.

ESRP/Jetstress results are inherently difficult to compare but are worth the effort in our view.  We strive to improve our analysis with each report.  As such, feel free to contact us with any ideas or improvements to this analysis, our contact information can be found below.

This performance dispatch was sent out to our newsletter subscribers in April of 2009.  If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) or subscribe by email and we will send our current issue along with download instructions for this and other reports.  Also, if you need an even more in-depth analysis of SAN storage system features and performance please take the time to examine our SAN Storage Briefing available for purchase from our website.

A PDF version of this can be found at

[wpfilebase tag=’file’ id=’132′ tpl=’just-link’]

Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community.


[1] ESRP results from http://technet.microsoft.com/en-us/exchange/ff182054.aspx, as of 29 April 2010

 

[2] All prior SCI ESRP Dispatches can be found at http://silvertonconsulting.com/cms1/news-4/