1K-to-5Kmbx – Silverton Consulting

Latest ESRP 1K-5K mailbox DB xfers/sec/disk results – chart-of-the-month

Posted on June 19, 2012October 8, 2012 by Ray in Block Storage, DAS, Disk storage, ESRP, ESRPv3/Exchange 2010, Ethernet, iSCSI, SAS, Storage performance, System effectiveness

(SCIESRP120429-001) 2012 (c) Silverton Consulting, All Rights Reserved

The above chart is from our April newsletter on Microsoft Exchange 2010 Solution Reviewed Program (ESRP) results for the 1,000 (actually 1001) to 5,000 mailbox category. We have taken the database transfers per second, normalized them for the number of disk spindles used in the run and plotted the top 10 in the chart above.

A couple of caveats first, we chart disk-only systems in this and similar charts on disk spindle performance. Although, it probably doesn’t matter as much at this mid-range level, for other categories SSD or Flash Cache can be used to support much higher performance on a per spindle performance measure like the above. As such, submissions with SSDs or flash cache are strictly eliminated from these spindle level performance analysis.

Another caveat, specific to this chart is that ESRP database transaction rates are somewhat driven by Jetstress parameters (specifically simulated IO rate) used during the run. For this mid-level category, this parameter can range from a low of 0.10 to a high of 0.60 simulated IO operations per second with a median of ~0.19. But what I find very interesting is that in the plot above we have both the lowest rate (0.10 in #6, Dell PowerEdge R510 1.5Kmbox) and the highest (0.60 for #9, HP P2000 G3 10GbE iSCSI MSA 3.2Kmbx). So that doesn’t seem to matter much on this per spindle metric.

That being said, I always find it interesting that the database transactions per second per disk spindle varies so widely in ESRP results. To me this says that storage subsystem technology, firmware and other characteristics can still make a significant difference in storage performance, at least in Exchange 2010 solutions.

Often we see spindle count and storage performance as highly correlated. This is definitely not the fact for mid-range ESRP (although that’s a different chart than the one above).

Next, we see disk speed (RPM) can have a high impact on storage performance especially for OLTP type workloads that look somewhat like Exchange. However, in the above chart the middle 4 and last one (#4-7 & 10) used 10Krpm (#4,5) or slower disks. It’s clear that disk speed doesn’t seem to impact Exchange database transactions per second per spindle either.

Thus, I am left with my original thesis that storage subsystem design and functionality can make a big difference in storage performance, especially for ESRP in this mid-level category. The range in the top 10 contenders spanning from ~35 (Dell PowerEdge R510) to ~110 (Dell EqualLogic PS Server) speaks volumes on this issue or a multiple of over 3X from top to bottom performance on this measure. In fact, the overall range (not shown in the chart above spans from ~3 to ~110 which is a factor of almost 37 times from worst to best performer.

Comments?

~~~~

The full ESRP 1K-5Kmailbox performance report went out in SCI’s April newsletter. But a copy of the full report will be posted on our dispatches page sometime next month (if all goes well). However, you can get the full SPC performance analysis now and subscribe to future free newsletters by just sending us an email or using the signup form above right.

For a more extensive discussion of current SAN or block storage performance covering SPC-1 (top 30), SPC-2 (top 30) and all three levels of ESRP (top 20) results please see SCI’s SAN Storage Buying Guide available on our website.

As always, we welcome any suggestions or comments on how to improve our analysis of ESRP results or any of our other storage performance analyses.

Latest Microsoft ESRP v3 (Exchange 2010) 1K to 5K mailbox performance results – chart of the month

Posted on August 25, 2011October 8, 2012 by Ray in Block Storage, Disk storage, ESRP, ESRPv3/Exchange 2010, Ethernet, FC, Storage Backup, Storage performance, System effectiveness

SCIESRP110726-004 (c) 2011 Silverton Consulting, All Rights Reserved

Microsoft specifies two different metrics on sequential read rates for database backup activity in their Exchange Solution Reviewed Program (ESRP) reports

MB read/sec per database
MB read/sec total per server

Our problem with these metrics is that they don’t say much about the storage systems performance. Some ESRP submissions could have a single database while others can have 100s of databases. And the same thing applies to servers, although 20 servers seems to be about the max we have seen. So as one can see the MB/s/DB or MB/s/server can vary all over the place depending on the Exchange configuration that one uses, even for the same exact storage system.

In the above chart, we have attempted to move beyond some of these problems and use the information supplied in the ESRP reports to aggregate DB backups across all databases. As such, we have derived a new metric called “total database backup”. (Pretty simple actually just multiply the MB/s/DB times the number of databases in the Exchange configuration).

A couple of problems with our approach.

Current ESRP reports typically utilize a shadow storage system and shadow Exchange servers which host 50% of the databases and email activity. So what I am showing for those ESRP reports is what two storage systems can accomplish not one.
Another potential way to get the same result would be to use the number of servers times the MB/sec/server metric. (But try as I might these two approaches didn’t work to get the same answer so I am using the computation above – must be the way I am recording the number of [shadow] servers).
Although ESRP reports the average MB/sec/database to backup a single database it’s not clear that these measurements were taken while backing up all active databases at the same time, especially for those submissions with 100s of databases.

Probably the last is the most problematic critique to our new measure but may not be that harmful for smaller configurations. Nonetheless, we produced the above chart and published it in our last months review of ESRP results for the 1001 to 5000 mailbox category.

One item we discussed in our report was that numbers of disk drives didn’t seem to correlate well with high positions on this chart. The number ten position (Fujitsu ETERNUS JX40) used over 140 disks, the number two position (Dell PowerEdge R510) had only 12 disk drives, and the number one solution (HP E5700) consisted of 56 drives, close to the average for this category.

One striking finding using this measure is that performance varies considerably from the top providing over 1600 MB/sec of database backup to the lowest of the group providing only ~800 MB/sec of backup performance. What with Exchange 2010 and lagged DAGs, some people feel that backup activity is no longer needed but we would disagree. We continue to believe that taking backups of Exchange data still makes a whole lot of sense and shouldn’t go away, ever.

It’s our hope that this or some similar follow-on metric will remove some of the Exchange configuration parameters from confounding ESRP reported storage system performance results. We realize that this quixotic quest may never be entirely successful nevertheless we perform this duty in the hope that it will benefit today and future storage performance analysts everywhere.

Comments?

—–

The full ESRP report went out to our newsletter subscribers last month. A copy of the full report will be up on the dispatches page of our website later next month. However, you can get this information now and subscribe to future newsletters to receive these reports even earlier by just emailing us at SubscribeNews@SilvertonConsulting.com?Subject=Subscribe_to_NewsletterR or using the signup form above and to the right.

As always, we welcome any suggestions on how to improve our analysis of ESRP or any of our other storage system performance discussions.

Latest ESRPv3 (Exchange 2010) results analysis for 1K-to-5Kmailboxes – chart of the month

Posted on November 23, 2010October 8, 2012 by Ray in Block Storage, Disk storage, ESRP, ESRPv3/Exchange 2010, Storage, Storage performance

(c) 2010 Silverton Consulting, Inc., All Rights Reserved

The chart is from SCI’s October newsletter/performance dispatch on Exchange 2010 Solution Reviewed Program (ESRP v3.0) and shows the mailbox database access latencies for read, write and log write. For this report we are covering solutions supporting from 1001 up to 5000 mailboxes (1K-to-5Kmbx), larger and (a few) smaller configurations have been covered in previous performance dispatches. On latency charts like this – lower is better.

We like this chart because in our view this represents a reasonable measure of email user experience. As users read and create new emails they are actually reading Exchange databases and writing database and logs. Database and log latencies should show up as longer or shorter delays in these activities. (Ok, not exactly true, email client and Exchange server IO aren’t the same thing. But ultimately every email sent has to be written to an Exchange database and log sometime and every new email read-in has to come from an Exchange database as well).

A couple of caveats are in order for this chart.

Xiotech’s top run (#1) did not use database redundancy or DAGs (Database Availability Groups) in their ESRPv3 run. Their feeling is that this technology is fairly new and it will take some time before it’s widely adopted.
There is quite the mix of SAS (#2,3,6,7,9&10), FC (#1,5&8) and iSCSI (#4) connected storage in this mailbox range. Some would say that SAS connected storage should have an advantage here but that’s not obvious from the rankings.
Vendors get to select the workload intensity for any ESRPv3/Jetstress run, e.g. the solutions shown here used between 0.15 IO/sec/mailbox (#9&10) and 0.36 IO/sec/mailbox (#1). IO intensity is just one of the myriad of Jetstress tweakable parameters that make analyzing ESRP so challenging. Normally this would only matter with database and log access counts but heavier workloads can also impact latencies as well.

Wide variance between read and write latencies

The other thing of interest in this chart is the interesting span between read latencies and write (database and log) latencies for the same solution. Take the #10 Dell PowerEdge system for example. It showed a database read latency of ~18msec. but a database write latency of ~0.4msec. Why?

It turns out this Dell system had only 6 disk drives (2TB/7200 RPM). So few disk drives don’t seem adequate to support the read workload and as a result, show up poorly in database read latencies. However, write activity can mostly be masked with cache until it fills up, forcing write delays. With only 1100 mailboxes and 0.15 IOs/sec/mailbox, the write workload apparent fits in cache well enough to be destaged over time, without delaying ongoing write activity. Similar results appear for the other Dell PowerEdge (#6) and the HP Smart Array (#7) which had 12-2TB/7200 RPM and 24-932GB/7200 RPM drives respectively.

On the other hand, Xiotech’s #1 position had 20-360GB/15Krpm drives and EMC’s Celerra #4 run had 15-400GB/10Krpm drives, both of which were able to sustain a more balanced performance across reads and writes (database and logs). For Xiotech’s #5 run they used 40-500GB/10Krpm drives.

It seems there is a direct correlation between drive speed and read database latencies. Most of the systems in the bottom half of this chart have 7200 RPM drives (except for #8, HP StorageWorks MSA) and the top 3 all had 15Krpm drives. However, write latencies don’t seem to be as affected by drive speed and have more to do with the balance between workload, cache size and effective destaging.

The other thing that’s apparent from this chart is that SAS connected storage continues to be an effective solution for this range of Exchange configurations, following a trend first shown in ESRP v2 (Exchange 2007) results. We reported on this in our January ESRPv2 analysis dispatch for this year .

The full dispatch will be up on our website in a couple of weeks but if you are interested in seeing it sooner just sign up for our free newsletter (see upper right) or subscribe by email and we will send you the current issue with download instructions for this and other reports.

As mentioned previously ESRP/Jetstress results are difficult to compare/analyze and we continue to welcome any constructive suggestions on how to improve.