SCI 2011 Apr 28 Latest Microsoft ESRP v3.0 (Exchange 2010) results

In Compellent, Data storage, Dell, DS8000, DS8700, ESRP, ESRP v3/Exchange 2010, EVA 8400, Hitachi AMS, Hitachi AMS 2100, Hitachi AMS 2500, Hitachi Vantara, HPE, IBM, PowerVault MD1200, SAN Volume Controller (SVC), Storwize V7000 by AdministratorLeave a Comment

This Storage Intelligence (StorInt™) dispatch covers recent Microsoft Exchange Solution Review Program (ESRP)[1] v3.0 results for Exchange 2010. Since the last time we discussed this over-5K mailbox category nine months ago, there have been more than a dozen new submissions. Future dispatches will report on the under-1K and 1K-to-5K mailbox categories, but all prior ESRP V2 and ESRP V3 performance dispatches are available on SCI’s website[2].

Latest ESRP V3.0 results

We start our ESRP analysis with Exchange database and log access latency results.  Recall that this chart is sorted by database read latencies.

SCIESRP110428-001 (c) 2011 Silverton Consulting, Inc., All Rights Reserved

SCIESRP110428-001 (c) 2011 Silverton Consulting, Inc., All Rights Reserved

Figure 1 Top ESRP database-read latencies

For Figure 1, the new IBM DS8700 takes top honors with HDS AMS systems taking then next 4 slots followed by HP.  The IBM DS8700 is the latest hardware from IBM for open and mainframe enterprise class storage and used 128-450GB, 15Krpm FC attached drives with 114.6GB of cache.  In contrast, the (#6) HP and (#10) Compellent both used SAS 450GB 15Krpm drives with 24 and 66 drives respectively.

There has been quite a lot of commentary on my blog[3] about ESRP’s latency numbers.  What’s clear is that Jetstress has a cut off at 20 msec. and as long as an ESRP result fits within that latency it’s considered acceptable by Microsoft.  I argued that even though that’s the case superior latency is still something to be strived for regardless of the maximal cutoff.  Other parties disagreed and said something to the effect that anything less than 20msec is a passing grade.  Nevertheless, that the DS8700 had almost ½ the read latency of the Compellent system is significant and worth noting.

We now turn to absolute ESRP database transfers per second.

SCIESRP110428-002 (c) 2011 Silverton Consulting, Inc., All Rights Reserved

SCIESRP110428-002 (c) 2011 Silverton Consulting, Inc., All Rights Reserved

Figure 2 Top 10 Database transfers per second

There are three new submissions on this chart namely, the (#1) IBM DS8700, their Storwize v7000 subsystem (#7) with 60K mailboxes and the HDS AMS 2500 with 20.9K mailboxes (#10).  We should note that the Storwize system had (24) SSDs in their configuration while no one else in this top 10 used FlashCache or SSDs.

Normally, at this mailbox level we don’t show absolute transferst because the spread is so wide.  For example, we have from 20K (#1 IBM DS8700) to 100.8K mailboxes (#3 HDS AMS 2500) on display here.  But there is continued interest in absolute numbers and as such, report them occasionally.

On the other hand, we always report on the normalized ESRP database transfer counts per 1K mailboxes (see Figure 3 below).

SCIESRP110428-003 (c) 2011 Silverton Consulting, Inc., All Rights Reserved

SCIESRP110428-003 (c) 2011 Silverton Consulting, Inc., All Rights Reserved

Figure 3 Top normalized DB (per 1K mailboxes) transfers per second

IBM’s DS8700 system showed up well here but used a 0.80 IOs/mailbox Jetstress driver parameter. In contrast, the #2, HDS AMS 2100 system used 0.18 IOs/mailbox that just happened to be the median for these top 10 systems.

Unclear what advantage having more or less IOs/mailbox would be but if one’s subsystem can handle more workload, probably raising the IO rate or reducing the mailbox count would help this ranking.  Naturally, having both helps even more.

Next we examine log playback performance.

SCIESRP110428-004 (c) 2011 Silverton Consulting, Inc., All Rights Reserved

SCIESRP110428-004 (c) 2011 Silverton Consulting, Inc., All Rights Reserved

Figure 4 Top log playback times

Over time I have come to better appreciate a select set of ESRP metrics and the Log playback times is one of them. Log playback is the average time it takes a storage system to process a number of 1MB log files applied to a single database.

We assume that all 1MB log files contain the same number of transactions.  Possibly the number of mailboxes per database might impact this processing (for these top 10, the numbers range from 250 to over 1400 mailboxes per database).  Certainly, how a single database is spread across drives (and/or SSDs) and system cachinge size/effectiveness should impact this value.

In any event, it simulates a pretty complex, database workload, driven as fast as the storage can handle it.  Whereas, many of the other ESRP workloads are all driven by the IO/mbx/sec, number of mailboxes, and/or other Jetstress parameters.

Such results as shown above (see Figure 4) probably says that the IBM DS8700 is a pretty good subsystems to process Exchange-like, database transactions.

Another of my favorite ESRP reported metrics is DB backup activity. There are two measurements reported in ESRP for database backup activity 1) Database data backed up in MB/sec/database and 2) Database data backed up in MB/sec/server.  For reasons of brevity, we shall only discuss the later and save the former for another time.

SCIESRP110428-005 (c) 2011 Silverton Consulting, Inc., All Rights Reserved

SCIESRP110428-005 (c) 2011 Silverton Consulting, Inc., All Rights Reserved

Figure 5 Top 10 database backup activity in MB/second/server

Figure 5, shows somebody other than IBM DS8700 in top place, the Dell PowerVault MD1200 with 24-2TB SAS connected, 7200RPM drives.  One surprising result here is that Dell (now that it owns Compellent) has five of the top seven results on this metric.  The other surprise is that seven of the top ten on this chart used SAS connected disks.  It’s apparent that the drive count does not impact this particular metric.

As stated in prior Dispatches using Exchange 2010’s DAGs and lagged databases has reduced the need for backups. But, we firmly believe there are many valid reasons for database backups and as such, an ongoing need for mailbox backup performance will remain. Thus, we continue to report on this valuable metric.

SCIESRP110428-006 (c) 2011 Silverton Consulting, Inc., All Rights Reserved

SCIESRP110428-006 (c) 2011 Silverton Consulting, Inc., All Rights Reserved

Figure 6 Top 10 DB xfers/sec/spindle

Finally, our last chart shows the top ten per drive performance in total database transfers/sec/spindle.  We exclude any subsystem with FlashCache or SSDs from this analysis to get a pure per drive view of performance.  We see that Dell PowerVault MD1200 and IBM’s DS8700 has done well here again.


Well this is our second ESRP analysis on the over-5K mailbox category and we have quite a few new entries for this report.  Nonetheless, the obvious ESRP winner in this mailbox level was the IBM DS8700. This system is the latest generation of enterprise storage hardware that IBM has been delivering since the ESS (AKA, the “shark”) was retired and seems to be well represented.

Finally, as discussed in prior dispatches, ESRP/Jetstress results seem destined to be difficult to compare but in our view, merit the effort.  Thus, we strive to refine our analysis with each report.  As always, feel free to contact us with any ideas on how to improve.  In that regard, our contact information can be found below or on our website at

[This performance dispatch was originally sent out to our newsletter subscribers in April of 2011.  If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) or subscribe by email and we will send our current issue along with download instructions for this and other reports.  Also, if you need an even more in-depth analysis of SAN storage system features and performance please take the time to examine our SAN Storage Briefing available for purchase from our website.]


Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community



[1] ESRP results from, as of 28 April 2011

[2] All prior SCI ESRP Dispatches can be found at


Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.