SCI’s latest Microsoft ESRP performance report as of October’16

In Dell, ESRP, ESRP v3/Exchange 2010, ESRP v4/Exchange 2013, EVA4400, FlashArray//m20, MD3860f, Oracle FS1-2, Pure, Storwize V5000, Tegile by Administrator0 Comments

This Storage Intelligence (StorInt™) dispatch covers Microsoft Exchange 2010 and Exchange 2013/2016 Solution Review Program (ESRP) results[1].  Since we last reviewed this 5000 and over mailbox category nine months ago, there have been at least 17 new submissions from Nutanix (1), Pure Systems (1), Tegile (1), Oracle (1), InforTrend (2), Lenovo (2) and Dell (9). A few of these are AFA (Pure & Tegile) and some are hybrid (Oracle, Nutanix & Lenovo). Just about every one of our top 10 charts have changed. We select 3 of these to show below.

Latest ESRP V3.0 & 4.0 (E2010&E2013) performance

In Figure 1 we show our Exchange database access latency chart.

SCIESRP161026-001Figure 1 SCI’s ESRP reported access latencies (sorted by read latency)

In Figure 1 Oracle, Pure and Tegile are new. Here we can see the influence of AFA and hybrid storage both of which provide great database access latencies. The Oracle FS1-2 had a read latency of 1.0msec while Pure and Tegile tied for 2nd with a 1.1msec. read access latency.

Database write latency shows a whole other side to these systems. Pure performed best with a write latency of 1.6 msec, while Oracle provided 1.8msec. and Tegile supplied a 3.1 msec write response time. AFA and Hybrid storage perform worse for small block writes versus read, but for Pure and Oracle. they still respond much faster than disk-only solutions.

On the other hand, Log write latency is a measure of large block sequential write activity.  AFAs generally can perform better on larger than small writes, but surprisingly, Pure’s #1 log write response time of 0.41msec was only slightly better than the all disk, IBM Storwize V5000 system.  Large sequential writes still perform very well on disk.

Next, we report on ESRP Log playback time in Figure 2

Figure 2 ESRP Top 10 Log Playback resultsSCIESRP161026-002

In Figure 2, Pure (#2), Oracle (#4), Tegile (#5), and the two Dell MD3860f (#8 & #10) storage systems are all new. We like the Log Playback metric because it is a mixture of large block, read sequential and small block, database updates (read/writes). Recall that ESRP’s Log Playback is the average time it takes to play back 1 log file (1MB) against a single database, averaged across 10K runs. Hybrid and AFAs are not as dominant in log playback as in access latency above. The current #1 in ESRP Log Playback (HP[E]4400 EVA) was an all disk, mid-range system. The new Dell systems are a midrange, all disk, FC direct submissions (FC attached with no switch) running with 15K and 20K mailboxes.

Finally, we present in Figure 3, our normalized DB transfers/second/1K mailboxes.

SCIESRP161026-003Figure 3  SCI Top 10 ESRP DB transfers/second normalized for 1K mailboxes

In Figure 3, the only new submission is the Pure FlashArray//m20 which came in 2nd with 751 read and 373 write database transfers/second per 1000 mailboxes.  The #1, HP(E) 3PAR storage used 120 2TB disk drives and the #2 Pure FlashArray had 20 SSDs, showing that there’s a significant (~6:1) performance advantage on a one-to-one, per device basis. It’s surprising there are no other hybrid or AFA submissions in Figure 3. But this may be proof of Pure’s architecture sophistication as compared to the other new AFA/hybrid solutions.

Summary

We’re always happy to see new submissions. We firmly believe that ESRP is the most comprehensive, real world measure of storage system performance that exists today, even if Microsoft will not call it a “benchmark”. More AFA’s and hybrid storage are good to see as they both seem to have taken over the market for enterprise storage.

On the other hand, how AFA compression/deduplication impact ESRP activity is unknown. ESRP needs to report realized compression and deduplication ratios, at least for AFAs, as it may provide a significant performance advantage. Also AFA submissions need to fully detail compression/deduplication storage settings for database and log devices.

 

Constructive comments on how to improve our analyses for Exchange or any of our performance reports are always welcome. Moreover, if you detect errors in this or any of our other performance reports, please do let us know and we will correct them as soon as possible.

[Also we offer more block storage performance information plus our OLTP, Email and Throughput ChampionsCharts™ in our recently updated (August 2017) SAN Storage Buying Guide, or for more information on some select ESRP performance results please see our recently updated (July 2017) SAN-NAS Storage Buying Guide, both of which are available for purchase on our website.]

[This performance dispatch was originally sent out to our newsletter subscribers in October of 2016.  If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) and we will send our current issue along with download instructions for this and other reports. Dispatches are posted to our website at least a quarter or more after they are sent to our subscribers, so if you are interested in current results please consider signing up for our newsletter.]  

 Silverton Consulting, Inc., is a U.S.-based Storage, Strategy & Systems consulting firm offering products and services to the data storage community.

[1] ESRP results from https://technet.microsoft.com/en-us/office/dn756396.aspx as of 25Oct2016