Exchange 2010/ESRP 3.0 results – chart of the month

Well after last months performance reversals and revelations we now return to the more typical review of the latest Exchange Solution Review Program (ESRP 3.0) for Exchange 2010 results.  Microsoft’s new Exchange 2010 has substantially changed the efficiency and effectiveness of Exchange database I/O.  This will necessitate a new round of ESRP results for all vendors to once again show what their storage can support in Exchange 2010 mail users.  IBM was the first vendor to take this on with their XIV and SVC results.  But within the last quarter EMC and HP also submitted results.  This marks our first blog review of ESRP 3.0 results.

We show here a chart on database latency for current ESRP 3.0 results.  The three lines for each subsystem show the latency in milliseconds for a ESE database read, database write and log write.  In prior ESRP reviews, one may recall that write latency was impacted by the Exchange redundancy in use.  In this chart all four subsystems were using database availability group redundancy (DAG) so write activity should truly show subsystem overhead and not redundancy options.

Unclear why IBM’s XIV showed up so poorly here.  The HP EVA 8400 is considered a high end subsystem but all the rest are midrange.  If one considered drives being used – the HP used 15Krpm FC disk drives, the SVC used 15Krpm SAS drives and both the CLARiiON and the XIV used 7.2Krpm SATA drives.  Still doesn’t explain the poor showing yet.

Of course the XIV had the heaviest user mail workload at 40,000 user mailboxes being simulated and it did perform relatively better from a normalized database transactions perspective (not shown).  Given all this perhaps this XIV submission was intended to show the top end of what the XIV could do from a mailbox count level rather than latency.

Which points up one failing in our analysis. In past ESRP reviews we have always split results into one of three categories <1Kmbx, 1001..5Kmbx, and >5Kmbx.  As ESRP 3.0 is so new there are only 4 results to date and as such, we have focused only on “normalized” quantities in our full newsletter analysis and here.  We believe database latency should not “normally” be impacted by the count of mail users being simulated and must say we are surprised by XIVs showing because of this.  But in all fairness, it sustained 8 80 times the workload that the CLARiiON did.

Interpreting ESRP 3.0 results

As discussed above all 4 tested subsystems were operating with database availability group (DAG) redundancy and as such, 1/2 of the simulated mail user workload was actually being executed on a subsystem while the other 1/2 was being executed as if it were a DAG copy being updated on the subsystem under test.  For example, the #1 HP EVA configuration requires 2-8400s to sustain a real 9K mailbox configuration with DAG in operation.  Such a configuration would support 2 mailbox databases (with 4500 mailboxes each), one active mailbox database residing on each 8400 and the inactive copy of this database residing on it’s brethern.  (Naturally, the HP ESRP submission also supported VSS shadow copies for the DAGs which added yet another wrinkle to our comparisons.)

A couple of concerns simulating DAGs in this manner:

• Comparing DAG and non-DAG ESRP results will be difficult at best.  It’s unclear to me whether all future ESRP 3.0 submissions will be required to use DAGs or not.  But if not, comparing DAG to non-DAG results will be almost meaningless.
• Vendors could potentially perform ESRP 3.0 tests with less server and storage hardware. By using DAGs, the storage under test need only endure 1/2 the real mail server I/O workload and 1/2 a DAG copy workload.  The other half of this workload simulation may not actually be present as it’s exactly equivalent to the first workload.
• Hard to determine if all the hardware was present or only half.  It’s unclear from a casual  skimming of the ESRP report whether all the hardware was tested or not.
• 1/2 the real mail server I/O is not the same as 1/2 the DAG copy workload. As such, it’s unclear whether 1/2 the proposed configuration could actually sustain a non-DAG version of an equivalent user mailbox count.

All this makes for exciting times in interpreting current and future ESRP 3.0 results.  Look for more discussion on future ESRP results in about a quarter from now.

As always if you wish to obtain a free copy of our monthly Storage Intelligence newsletter please drop us a line. The full report on ESRP 3.0 results will be up on the dispatches section of our website later this month.