HDS 2010 Sep 27 announcement of their new Hitachi VSP enterprise storage subsystem

Sep 272010

HDS announces Hitachi VSP

HDS recently announced their latest version of enterprise storage, the Hitachi Virtual Storage Platform (VSP). The Hitachi VSP is a scale up and out update of their popular USP-V storage subsystem, which supports storage virtualization, and 3D scaling. The VSP is a significant technological advance over prior generation Hitachi storage.

Hitachi VSP

Hitachi VSP provides 3D scaling, i.e., scale out, up and deep in enterprise class storage. Scale out is accomplished by dynamically combining multiple units into a single logical system with shared resources, scale up is by dynamically adding processors, connectivity and capacity in a single unit and scale deep is by virtualizing external storage.

VSP hardware

The VSP incorporates a controller blade cabinet design and now uses19” racks. A single controller rack contains one controller chassis and includes up to two drive chassis. One controller chassis can support from 2 to 4 Virtual Storage Director (VSD) blades, from 2 to 8 Cache Adaptors (CA), from 2 to 12 Front-end Director (FED) blades, and from 2 to 4 Back-end Director (BED) blades. Two controller chassis can be connected together to form one storage subsystem and would support up to 8 VSD blades, up to 24 FED blades with 192-8Gbs FC/FICON ports, up to 16 CAs with 1TB of total cache, and up to 8 BED blades with 64-6Gbs SAS storage links.

A minimum internal storage configuration would include 2-VSDs, 2-FEDs, 2-CAs and 0-BEDs supporting a diskless storage subsystem with from 64GB to 1TB of cache storage and no BEDs. Although, a diskless system makes sense as a storage virtualization subsystem used with external storage it could also be used for migration.

Up to 4 additional storage only racks can be interconnected to the storage complex for a total of 16 drive chassis across 6 racks. The VSP also includes a 5^th generation Hitachi cross bar switch for internal connections and uses PCI-E as an interconnect between the two control chassis. Together the total system acts as one storage subsystem complex with any VSD processor core accessing any individual FE, BE, or cache resource to support I/O and other storage processing activity.

A single VSD contains a quad-core Intel processor, which speeds up I/O processing and storage administration. Also the shared or control memory is now distributed in L2 cache across all VSDs rather than centralized as in the USP-V. IO and other processing activity are parceled out across all available VSD processor cores in the cluster to balance subsystem performance.

VSP cache is used to hold all read and write data, is global across the cluster. Write data is always mirrored in cache and backed up to onboard flash drive(s). Each control chassis supports up to 512GB of cache.

BED and FED blades have onboard data accelerator ASICs to assist in data path processing, IO routing, parity calculations and encryption activities. Any IO coming into a Front-end port can be directly routed to whichever core handles that particular LDEV.

Drive storage is attached to control chassis via redundant, 6 Gbs SAS links. Disk storage comes in 2.5” and 3.5” drive configurations, with each 14U storage bay containing up to 128-2.5” or 80-3.5” drives per chassis. As expected, any drive can be replaced without powering down the subsystem. Also, SAS or SATA drives can be intermixed. In addition, the VSP also supports 2X the flash drives or SSDs in USP-V, using highly reliable SLC NAND technology. SSDs support up to 70X random read performance or up to 14X random write performance more than disk drives.

Finally, cooling airflow for the VSP is from front to back and power is now single phase vs. three-phase for USP-V. The VSP uses about 48% less power than the USP-V.

VSP functionality

Hitachi’s new 3D management is to manage UP, OUT and DEEP. Manage UP refers to their new automated data placement/dynamic tiering, dynamic provisioning, the ability to manage up to 5M logical objects, 255PB of virtualized capacity and the ability to scale up virtual server deployments.

Dynamic Tiering extends to external storage and applies on a 42MB page (sub-LDEV) basis to a storage pool. Three storage tiers can be supported. Dynamic Tiering can be done at the page level or at the object level (LDEV or file). Data is written first to highest performing tier and then later migrated to other tiers based on reference activity.
Dynamic (thin) provisioning can now be extended to external storage. Fixed provisioned volumes can be converted to thin volumes by migrating them into a Dynamically Provisioned storage pool. Also, Dynamic Provisioning automatically stripes data across all drives in the pool; increasing IO performance considerably over fixed provisioned LDEVs.

Manage OUT refers to their new unified management scheme under the Hitachi Device Manager (HDvM) for block, file and content storage across all Hitachi storage systems. This includes end-to-end visibility of applications, virtual machines/servers and logical storage devices for traditional and VMware environments.

Manage DEEP refers to the tight integration between Hitachi Device Manager and Hitachi Command Suite to supply consistent implementation of management activities, service level management for capacity and performance, and common GUI, data repository and workflows across the two software suites.

In addition, the VSP shares advanced storage functionality of the USP-V subsystem with the addition of dynamic tiering at the page level and dynamic provisioning for external storage. For example, Hitachi VSP supports multi-tenancy for guaranteed QOS, storage virtualization for externally attached storage, automated data mobility of objects or blocks to different storage tiers, VMware VAAI support, storage encryption, Hitachi Universal Replicator, High Availability Manager, and Virtualized storage replication to name just a few.

At initial release there are some restrictions with storage replication, e.g. no support for Extended consistency groups and no delta resynch support. However the VSP can replicate to a USP-V.

VSP performance

Preliminary performance numbers were provided that shows the substantial performance advantage of the VSP. Specifically, random read is 1.9X and sequential read throughput is 1.4X better than the current USP-V. Also, random writes are 2.7X better and sequential write throughput is 1.2X better than the USP-V subsystem. These results represent full VSP and USP-V subsystems, the VSP per front-end port performance numbers are even better. Also these performance numbers come from hard disk drive subsystems, which don’t include Flash or SSDs in the VSP or USP-V.

Announcement significance

Obviously, Hitachi has taken another major step to upgrade their enterprise class storage subsystems with their VSP offering. The VSP offers a number of advantages over their current USP-V system including better performance, lower power consumption, better storage density, to name just a few.

VSP compares well to EMC’s VMAX Symmetrix subsystem. While VMAX is also scaleable, EMC uses separate nodes, which contain processors as well as cache, front-end, and back-end control and uses a switching fabric for inter-node communications. VMAX was a radical and total revision of their previous Symmetrix architecture. At announcement, VMAX was described as a cloud enabler because of its scaleability.

However, Hitachi’s move to a blade system is not as radical a departure from their USP-V storage architecture. Yes, adding more and faster cores, L2 control cache, 2.5” disks, SSDs, SSD backed up cache, and using PCI-E for control chassis inter-connects all advance Hitachi storage technology to the next level. But one can see in all this a continuation and expansion of the USP-V architecture rather than a complete restructuring like EMC’s VMAX. It will be interesting to see how VSP is received in the market place

Perhaps just as important is the continuing advance of storage functionality including Dynamic Provisioning for external storage and Dynamic Tiering at sub-LDEV level for both internal and external storage. Adding this new functionality to Hitachi’s extensive storage virtualization capabilities, one can have all the latest functionality applied to older storage subsystems.

A PDF version of this can be found at

HDS 2010 Sep 27 announcement of their new enterprise storage, the VSP (PDF 365.2 KiB)

Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community.

SCI 2010 Aug 27 latest SPC benchmark results analysis with storage energy use

Aug 272010

We return now to that classic block storage benchmark, Storage Performance Council (SPC) results*. There have been only two new SPC submissions since our last report, namely the Oracle Sun Storage 6780 and HP StorageWorks 6400 EVA (with SSD) for SPC-1 and SPC-1/E respectively. You may recall that SPC-1/E is an energy monitored version of the SPC-1 test so we have been combing the two test results into one series

SPC-1*results

The chart that has changed most since our last report is LRT results.

(SCISPC10827-001) (c) 2010 Silverton Consulting, All Rights Reserved

The new HP 6400 EVA with SSDs showed up at #8 on this chart. Not as good as the TMS SSD runs but not bad for a mid-range subsystem. The EVA supported RAID 5 and had 8-73GB SSDs which put it on the small side. Nonetheless, the EVA SSD subsystem was priced around ~$121K US, which puts it at ~$350/GB pricy for storage today but this is SSD.

(SCISPC10827-002) (c) 2010 Silverton Consulting, All Rights Reserved

This chart also changed with the addition of the Oracle Sun 6780 with 146GB/15Krpm FC drives but I would defy anyone from showing us what has changed. For the discerning among you, the Oracle subsystem shows up at around 62K IOPS with an LRT of ~1.8msec and costing ~$43K US.

(SCISPC10827-003) (c) 2010 Silverton Consulting, All Rights Reserved

The SPC-1/E benchmark submissions report on IOPs/Watt measurements taken at Nominal, Medium and High daily workloads. As one can see the HP EVA with SSDs did fairly well at the Medium and High workloads but as it’s SSD based storage we would expect it to. But the Nominal workload IOPs/W for the SSD subsystem is worthy of discussion.

SPC defines moderate and heavy workloads at 50 and 80% of maximum reported performance respectively and then defines

Nominal daily workload as 16 hours of idle time and 8 hours of moderate workload,
Medium daily workload as 6 hours of idle time, 14 hours of moderate workload and 4 hours of heavy workload, and
High daily workload as 0 hours of idle time, 6 hours of moderate workload, and 18 hours of heavy workload.

As one can see the subsystem will determine the levels of IOPs used to determine nominal, medium, and heavy. Curent results show two things about the HP EVA SSDs

The idle power consumption of HP EVA SSDs is pretty high compared to the Xiotech’s systems.
The relative lack of power difference between nominal (465.0w) and high (471.9w) workloads for SSDs punishes them in the nominal usage scenario.

(SCISPC10827-004) (c) 2010 Silverton Consulting, All Rights Reserved

For the three current SPC-1/E benchmarks (Xiotech Emprise 5000 with 146GB drives, Xiotech Emprise 5000 with 600GB drives and the new HP StorageWorks EVA with SSDs) we show a new bubble chart depicting power and IOPS for medium and high workloads. The smallest bubble pair shown above is the Xiotech with 146GB, the middle size bubble pair is the Xiotech with 600GB and the largest bubble pair is the HP EVA with SSDs.

So why don’t we see a significant increase in power consumption with the SSDs. We have discussed this before but SSD power consumption doesn’t increase much as you drive it harder. So the 50% IOPS rate consumed ~469 watts and the 80% IOPS rate consumed about ~473 watts. Unclear why the SSD subsystem actually consumed more power than the other two disk drive subsystems but realize the EVA was a fully configured storage subsystem with FC attached SSDs and 8GB of cache (not to say the Emprise subsystems were not).

Significance

The other charts for SPC-1 and SPC-2 were unchanged and have all been covered in prior reports. SPC benchmark submissions have been slowing down of late. Unsure but this might be indicative of new products coming out, delaying submissions.

SSD storage energy profiles are certainly different than hard drives. I would have thought that any SSD subsystem would consume less power than a comparable hard drive subsystem. But that certainly wasn’t the case for the Xiotech vs. HP EVA. Nonetheless, the EVA SSD subsystem did put up some interesting response times and on a iops/Watt basis it did very well in the medium and high daily workloads.

We can only hope for more SPC-1/E test submissions. It’s exactly the same as the SPC-1 testing except for the power monitoring and it would supply power consumption metrics as well as performance with the same submission. Makes sense to me, but I don’t have to pay for the monitoring equipment.

As always if you have any suggestions on how we can improve our SPC or any performance analysis, please don’t hesitate to drop us a line. Our contact information can be found in the footer of this page.

This performance dispatch was sent out to our newsletter subscribers in August of 2010. If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) or subscribe by email and we will send our current issue along with download instructions for this and other reports. Also, if you need an even more in-depth analysis of SAN storage system features and performance please take the time to examine our SAN Storage Briefing available for purchase from our website.

A PDF version of this can be found at

SCI 2010 Aug 27 Latest Storage Performance Council (SPC) results analyzed (PDF 837.3 KiB)

Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community.

* All results from www.storageperformance.org as of 27 Aug 2010

SCI 2010 May 27 latest SPC benchmark results analysis

May 272010

We return now to that classic block storage benchmark, Storage Performance Council (SPC) results*. We mostly discuss SPC-1 latest results below with a short discourse on an interesting SPC-1C result.

SPC-1*results

There have been five new SPC-1 results this past quarter – two Huwaei Symantec Oceanspace subsystems the S2600 and the S5600, two from Fujitsu the ETERNUS DX400 and DX8400 and the latest IBM 8700 (R5.1) with SSDs, SATA drives and Easy Tier automation. It’s unclear whether IBM’s Easy Tier had sufficient runtime to effect performance optimization for any SPC-1 runs (see discussion below). Nevertherless, none of these subsystems made it into the top 10 in IOPS™. However, three of them did make it into the best LRT™ results discussed below.

(SCISPC10527-001) (c) 2010 Silverton Consulting, All Rights Reserved

The new IBM DS8700 and the two Fujitsu systems showed up well in as #6, 8 & 9 in top 10 LRT results. Recall that LRT measures the average response time during the 10% load factor run and as such, should correspond to the best response time from a relatively idle subsystem. We have discussed the other Top 10 LRT subsystems in prior dispatches and do not cover them here#.

(SCISPC10527-002) (c) 2010 Silverton Consulting, All Rights Reserved

Both Huawei Symantec subsystems showed up well in most of the $ based comparison and here one can see the $/IOPS metric where they came in at #6 & 8. One caution here is that Huawei subsystem pricing was given in CNY which we converted into USD for comparison purposes at CNY ~6.8 to the $. Nonetheless, they compare well in price performance as SPC now calls it.

In addition, it has been brought to my attention that the SPC-1/E benchmark runs are equivalent to the SPC-1 runs. As such, we have added Xiotech’s recent SPC-1/E run for their Emprise 5000 with 146GB Huricane drives which now comes in at #7.

(SCISPC10527-003) (c) 2010 Silverton Consulting, All Rights Reserved

In previous discussions we showed SPC-1 scatter plots for IOPS vs. Capacity and IOPS vs. $/GB. In this report we now return to IOPS vs. LRT. As one enhancement, we examined statistical trendlines (not shown) for this data but there does not appear to be any with a high correlation, so have left them out.

The two new Fujitsu systems, the DX440 and DX8400 show up well at the ~1.5msec line with ~100K and ~170K IOPS respectively and the medium cost subsystem at ~1msec LRT with ~33K IOPS is the new IBM DS8700 with Easy Tier, SSD and SATA disks.

(SCISPC10527-004) (c) 2010 Silverton Consulting, All Rights Reserved

We have a new winner for this chart as the low-end Huawei Symantec S2600 subsystem managed to crack 350 IOPS™ per drive. Their higher-end subsystem was a respectable 258 IOPS™ per drive but didn’t break into this top ten. Unclear why the low-end Oceanspace did so well, it was just using 48-146GB 15Krpm SAS drives in a RAID 1 configuration.

However we have also updated this chart to include SPC-1/E data and now show the Xiotech’s 146GB and 600GB drive SPC-1/E runs come in at #2 and #5. Not bad for a 600GB disk drive.

You will recall that we have excluded pure SSD subsystems from this analysis as they tend to be off the chart, literally. Not sure whether Easy Tier should or should not be ok here but as it came in at the middle of the pack, we felt including was justifiable.

(SCISPC10527-005) (c) 2010 IBM, All Rights Reserved

As discussed above IBM was using SSD and SATA disks along with new automated storage tiering called Easy Tier. The effect of Easy Tier& is to move “hot” extent data from SATA disk to SSD. Hot is determined by subsystem activity monitoring over some time period.

One can see in this SPC-1 generated graph that I/O activity began ~15K IO/s and peaked out at ~50K IO/s before the SPC-1 driver dropped down to the requested workload (~33K IO/s). The requested workload amount is chosen at the discretion of the vendor running the test but from our perspective it looks like it could have sustained 50K IO/s for the rest of the 24hr run. The other thing of interest is the absolute lack of variability in the IO/s for the remainder of the run which probably says something about SPC-1’s working set size.

Other SPC results

(SCISPC10527-006) (c) 2010 Silverton Consulting, All Rights Reserved

Another new SPC submission was for its SPC-1C, which as you may recall is a component level benchmark. Oracle submitted a Sun F5100 Flash drive storage system which blew out all the other components by ~70X. Please note that this chart shows 100% IOPS load on a logarithmic scale and without this one could barely see the other results. The F5100 was SAS connected to a SPARC server running the benchmark. In fact with over 300K IOPS™ the Oracle component storage would easily have qualified in the top 3 IOPS results for the normal SPC-1 if only it supported FC attachment. Seems to be a “screamer” SAS-DAS storage.

Significance

There has been only one new SPC-2 submission this past quarter for the Fujitsu ETERNUS DX80 but it did not place in the top 10 in MBPS and so, we do not show any results for SPC-2. As for the other SPC benchmarks, there have been no results this past quarter.

It seems like SPC might need to come up with a “pure SAS-SSD” benchmark. I would think that there are other SAS-Flash storage vendors who might want to take on the Oracle F5100 juggernaut in a separate competition.

Also as automated storage tiering (like Easy Tier) goes mainstream it’s unclear how benchmarks should change to take advantage of these capabilities and how to better report on such capabilities. The vast majority of current SPC-1 submissions only use one drive type. Having multiple drive types and tiering automation certainly confounds any accurate performance comparisons.

This performance dispatch was sent out to our newsletter subscribers in May of 2010. If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) or subscribe by email and we will send our current issue along with download instructions for this and other reports. Also, if you need an even more in-depth analysis of SAN storage system features and performance please take the time to examine our SAN Storage Briefing available for purchase from our website.

A PDF version of this can be found at

2010 May 27 SCI's latest analysis of SPC results (PDF 1.2 MiB)

Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community.

* All results from www.storageperformance.org as of 27 May 2010

# See http://silvertonconsulting.com/cms1/dispatches/

& From http://www.storageperformance.org/benchmark_results_files/SPC-1/IBM/A00092_IBM_DS8700_EasyTier-SSDs/a00092_IBM_DS8700_EasyTier-SSDs_SPC1_full-disclosure.pdf

IBM 2010 April 19 announcement of new DS8700 version 5.1

Apr 192010

Announcing IBM DS8700 5.1

IBM recently announced a new revision of the tier-1 storage platform the DS8700 version 5.1 that includes a new automated storage tiering solution they called Easy Tier.

Easy Tier

With the popularity of SSDs, storage subsystems need some mechanism or automated solution that moves performance constrained data from hard disk to SSDs. With Easy Tier, IBM has now offered just such a solution. Easy Tier is available free of charge and depends upon a new concept called the Easy Tier-managed extent pool or hybrid extent pool.

In the DS8700 5.1 a hybrid extent pool of storage is monitored for performance and supports automated migration of data between storage tiers. Once volume or LUN data storage is migrated into a hybrid extent area, automatic performance monitoring is initiated. About 24 hours after monitoring starts, performance constrained data will begin to be migrated to SSDs and cold data can be moved off SSDs. Easy Tier data migration is a background activity and is limited such that it will not impact subsystem performance similar to the DS8700’s background FlashCopy services.

The performance monitoring is only done for disk or SSD access and thereby ignores cache hits. Hybrid extent storage is broken up into 1GB extents like DS8700’s thin provisioning and volume space allocation. However, with Easy Tier managed storage these extents are automatically moved up or down the storage hierarchy to optimize pool performance. With 5.1, only two tiers of storage are supported by Easy Tier migration, i.e., enterprise or SATA disk drives and SSDs but future versions will include a third tier of storage as well.

Included with DS8700 is a manual migration mode that can be used to non-disruptively to migrate data LUNs or volumes in or out of a hybrid extent pool. In this fashion, current data volumes could be incorporated into a hybrid extent pool to be automatically managed by Easy Tier. In addition, two hybrid extent pools can be merged into one or a volume can be migrated from one pool to another with manual migration mode.

Also included is a new Windows based, Storage Tier Advisor tool which provides guidance as to what volumes or extents are hot in your DS8700 and thus, could benefit from Easy Tier management. Also the Advisor tool can provide estimates of performance changes available from adding SSD storage to current DS8700 configurations.

Other changes with DS8700 5.1

Included in 5.1 are some additional changes such as

New enterprise 600GB disk drives and 2TB SATA disk drive support,
8 SSD drive mini-pack availability
Thin provisioning update
Faster concurrent, non-disruptive code loading
Full disk encryption enhancements to support payment card industry (PCI) data security standards
More granular management of multiple Global Mirror sessions
Remote pair flash copy updates
High performance FICON support at extended distances (system/z performance enhancement option)

Announcement significance

With Easy Tier, now SSDs can be effectively integrated into a fully automated, multi-tier storage hierarchy at the sub-volume level. Easy Tier implementation comes free, is easy to implement and within a day of monitoring, automatically moves data around to optimize the performance of the DS8700 subsystem storage.

Most of the other enhancements were available on previous versions of the DS8000 that were held back until fully verified. Such capabilities have now been tested and are enabled with DS8700 5.1. In contrast, the new 2TB and 600GB drive offerings and the new 8-SSD mini-pack that reduces acquisition cost to add SSDs are new with the DS8700 5.1.

With the recent addition of XIV storage to IBM’s line up there has been some confusion as to which subsystem will continue to see development by IBM. To put this to rest IBM stated that the DS8700 remains their premier storage platform and continued investment in R&D for enhanced capabilities lends credence to this view. The DS8700 is the culmination of IBM’s long history with disk storage dating back to the 1950’s. Given all this I don’t foresee an end to IBM’s continued investment in this technology.

A PDF version of this can be found at

IBM 2010 April 19 Announcement of DS8700 5.1 with new Easy Tier (PDF 280.6 KiB)

Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community.

SCI 2010 Feb 23 latest SPC benchmark results analysis

Feb 232010

We once again return to the classic block storage benchmark, the latest Storage Performance Council (SPC) results*. Not much new in SPC-1C/E or -2C results so only SPC-1 and SPC-2 activity is discussed below.

SPC-1*results

(SCISPC100223-001) (c) 2010 Silverton Consulting, All Rights Reserved

There have been three new SPC-1 results this past quarter – two based on IBM SVC5.1 with DS8700 backends and one from Infortrend. Both new SVC5.1s managed to crack into the top 10 in IOPS™ performance as #1 and 2 with similar backend DS8700 hardware using 1024-146GB 15Krpm drives in two DS8700’s, 384GB of cache, and 16-4GFC connections each. Each SVC node had 24 GB of memory/cache running with 4-8GFC connections. The only difference between the two new SVC results was the number of nodes (6 for the top result and 4 for #2).

We have discussed other subsystems on this top 10 chart in previous dispatch editions and do not reiterate that here#. However, the rest of these subsystems seem to now pale in comparison to the 6-node SVC run.

(SCISPC100223-002) (c) 2010 Silverton Consulting, All Rights Reserved

The only other changes for our other reported top 10 SPC-1 charts was on the IOPS/$/GB. This is sort of a weird metric as it combines IOPS rate, subsystem cost and capacity into one performance number.

Here one can see the Infortrend storage subsystem hitting #1 on the top 10 and the SVC subsystems coming in as #3 and #7 respectively. Infortrend was able to do so well because it combined high performance (180K IOPS), relatively low price (under $1M USD) with reasonable capacity (~49TB). What strikes one when looking at this chart is that the 6-node SVC is so much better (almost 2X) than the 4-node subsystem. Realize that the added performance for the 6-node subsystem at only small additional cost (~$31K USD difference) with equivalent capacity drove this IOPS/$/GB. So if you’re interested in IOPS performance and running IBM SVC5.1, add nodes.

(SCISPC100223-003) (c) 2010 Silverton Consulting, All Rights Reserved

In previous discussions we showed SPC-1 scatter plots for IOPS vs. LRT, IOPS vs. $/GB. In this report we now add IOPS vs. Subsystem Capacity.

What’s surprising here is the high correlation (R**2 of ~0.8) between capacity and performance. Any subsystem above the line on this chart gets better performance out of its subsystem capacity than the norm. This makes sense for the two IBM SVC runs (one above and the other below with equal capacity). However the one significant outlier is TMS RAMSAN at around 250K IOPS using SSD, which has relatively small capacity and as such, shows up extremely well here. The other, less significant outlier at around 275K IOPS with ~60TB was an IBM SVC4.3 benchmark run.

SPC-2 results

(SCISPC100223-004) (c) 2010 Silverton Consulting, All Rights Reserved

There have been only one new SPC-2 submission for the IBM DS8700 and it did break into the top 10 MPBS™ at #3 (see Figure 4. above). The DS8700 used RAID5 and 300GB disk drives for its top 10 performance. It’s somewhat surprising that there were no new SVC5.1 SPC-2 benchmark submissions but then perhaps they exhausted their budget doing the SPC-1 runs.

(SCISPC100223-005) (c) 2010 Silverton Consulting, All Rights Reserved

We have added the IBM DS8700 to our Top 10 MPBS spider chart showing the actual performance for all three classes of workloads in the SPC-2 benchmark. Once again, HDS and IBM seem to have found some magic formula for their caching that allows their LDQ (large database query) to be significantly better than the other two workloads (VOD-video on demand and LFP-large file processing). The IBM DS8700 LDQ performance is almost 30% better than their other results, fairly significant from my perspective.

Significance

It seems some of the lesser SPC benchmarks are not gaining as much traction as subsystem level benchmarks. This could be a matter of time or maybe just popularity. I see SPC has yet another benchmark coming, called the SPC-3BR benchmark for backup restore. We can only applaud SPC for introducing more benchmarks but just wish more vendors would submit results for what’s currently available.

This performance dispatch was sent out to our newsletter subscribers in February of 2010. If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) or subscribe by email and we will send our current issue along with download instructions for this and other reports. Also, if you need an even more in-depth analysis of SAN storage system features and performance please take the time to examine our SAN Storage Briefing available for purchase from our website.

A PDF version of this can be found at

SCI 2010 February 23 Review of Latest SPC-1 and SPC-2 Performance results (PDF 773.3 KiB)

Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community.

* All results from www.storageperformance.org as of 27 August 2009

# See http://silvertonconsulting.com/cms1/dispatches/

SCI 2009 Nov 19 latest SPC benchmark results analysis

Nov 192009

We once again return to the classic block storage benchmark, the latest Storage Performance Council (SPC) results*. Also we report on the new SPC-1/E energy usage benchmark for the first time.

SPC-1*results

There have been four new SPC-1 results this past quarter, IBM Power 595 with SSDs, TMS RamSan-620 with SSDs, Sun Storage 6180 and Fujitsu ETERNUS DX80 (rebadged 8000) storage subsystems. Both SSD subsystems made it into the top 10 on a number of charts. Let’s start with IOPS™.

(SCISPC091119-001) (c) 2009 Silverton Consulting, All Rights Reserved

The top IOPS storage subsystem now stands as an IBM Power 595 server using SSDs and comes in at ~301K IOPS. TMS’s RamSan-620 at number 5, hit almost 255K IOPS. All the remaining, non-SSD, top-10 IOPS results save one (IBM SVC 3.1), had over 1000 drives. In contrast, the TMS RamSan-620 used only 20 SSDs. Not sure what the IBM Power 595 is doing in a storage subsystem benchmark but for SAS attached SSD storage, it’s a screamer.

(SCISPC091119-002) (c) 2009 Silverton Consulting, All Rights Reserved

Both the RamSan-620 and the IBM Power 595 tied for 3^rd at 0.5 msec LRT. All the rotating disk results range from 0.9 to 1.7 msec LRT. It’s almost inconceivable that the TMS RamSan 400 hit a 0.1 msec LRT and its counterpart, the TMS RamSan-320 hit only 0.2 msec but both have been reported before. What’s somewhat surprising is that the FC attached SSDs (TMS) and the SAS attached SSDs (IBM) perform equally well in LRT results, probably indicating that LRT performance does not always depend on drive interface.

(SCISPC091119-003) (c) 2009 Silverton Consulting, All Rights Reserved

Recall that in our last report we now restrict the IOPS/drive chart to only those subsystems using 140GB drives or larger. To that we now must add that results for SSDs are also excluded along with other memory subsystems. We would need a log scale to include the latest SSD results here, as the TMS RamSan-620 hit over 12.7K IOPS/Drive and the IBM Power 595 hit over 3.5K IOPS/Drive.

In contrast, both the Sun 6180 and Fujitsu’s DX80 made it into the top 10 for IOPS/drive at 326 and 300 IOPS/drive respectively. Also the Sun 6180 and IBM’s DS5020 Express perform exactly alike and seem to represent almost the same storage subsystem (OEMed probably from LSI, see also SPC-2 results below).

(SCISPC091119-004) (c) 2009 Silverton Consulting, All Rights Reserved

Readers will recall that this chart used to show a minimum LRT of 1.5 msec. With the addition of the TMS RamSan-620 we have had to rescale the chart below 1.5 msec. The other two additions to this chart were the Sun 6180 and the Fujitsu DX80, although they are difficult to discern in the crowd around 25K IOPs and 2.0 msec LRT. This chart always seems to tell us that subsystem price is not the lone factor in determining SPC-1 performance.

SPC-1/E

There have been no new SPC-1C or SPC-1C/E benchmarks this last quarter but a new benchmark has been released for subsystem wide energy use, the SPC-1/E. Xiotech has released results for their Emprise 5000 system with both 146GB/15Krpm and 600GB/10Krpm drives.

(SCISPC091119-005) (c) 2009 Silverton Consulting, All Rights Reserved

Xiotech used the same number of drives in each case (20), probably why the 600GB drive subsystem cost so much more (note bubble size). But more significant is that even with over 4 times the storage capacity the subsystem running the newer drives operates at ~26% less power. In all honesty the new 600GB drives operate slower, at 10Krpm than the 15Krpm 146GB drives. However, peak performance dropped only 14% from 6962 to 6057 IOPS and as such, seems a viable tradeoff.

SPC-2 results

There were eight new SPC-2 results submitted this last quarter, Sun 6180 and IBM DS5020 Express at RAID5 and RAID6 and Sun 6780 and IBM DS5300 with 8GFC at RAID5 and RAID6. Similar to the discussions above (see IOPS/drive), these two sets of subsystems perform exactly alike, i.e., the Sun 6180 equals the IBM DS5020 Express and the Sun 6780 equals the IBM DS5300 in performance, and so seem to be two of the same subsystems OEMed from the same vendor (probably LSI).

(SCISPC091119-006) (c) 2009 Silverton Consulting, All Rights Reserved

Actually the Top 11 are shown as the old (4GFC) Sun 6780/IBM 5300 results are tied for last place here. One can see the new (8GFC) Sun 6780/IBM DS5300 showing up at positions 5 through 7. We would have thought the 8GFC might make more of a difference with the SPC-2 throughput oriented testing but it only seemed to boost MBPS by ~17% (for RAID5). From our perspective, the sad part about this chart is that there really are only four subsystems represented here the HDS and it’s OEM, the two (LSI) OEMs, and the IBM SVC.

Significance

Power use continues to gain more interest. We again applaud SPC for providing yet another new energy benchmark. The other items of note from these results are that SSDs perform well whether FC or SAS attached and that a subsystem with only 20 SSDs (TMS RamSan-620) can easily break into the top 10 IOPS chart.

This performance dispatch was sent out to our newsletter subscribers in November of 2009. If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) or subscribe by email and we will send our current issue along with download instructions for this and other reports. Also, if you need an even more in-depth analysis of SAN storage system features and performance please take the time to examine our SAN Storage Briefing available for purchase from our website.

A PDF version of this is available at

SCI 2009 Nov 19 Update to SPC benchmark results (PDF 938.6 KiB)

As always we welcome any feedback on how to do this better. So, if you have any comments please don’t hesitate to contact us.

Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community.

* All results from www.storageperformance.org as of 27 August 2009

SCI 2009 May 27 latest SPC benchmark results analysis

May 272009

We once again return to discuss the latest Storage Performance Council (SPC) results*.

SPC-1*results

There have been only five new SPC-1 results these past quarter, one for IBM DS5300, one for 3PAR InServ F400, and three for HDS AMS2500, AMS2300, and AMS2100. None of these made the top 10 in IOPS™ or $/IOPS™ so these charts can be found in prior dispatches#. However, two of these products did make the top 10 for IOPS/$/GB (see Figure 1 below).

(SCISPC090527-001) (c) 2009 Silverton Consulting, All Rights Reserved

The two new subsystems here are the 3PAR InServ F400 and the HDS AMS2500, both of which would be considered mid-class storage subsystems. We calculate this metric as = IOPS / ($/GB) and created this metric as another way to factor in performance against cost and capacity. Although none of these metrics can tell the complete story.

(SCISPC090527-002) (c) 2009 Silverton Consulting, All Rights Reserved

As for response time, the latest IBM DS5300 managed to break into the top 10 at an average least response time (LRT) of 1.77 msec. I have been told this metric is not as important anymore but given all the interest in SSDs these days I find that hard to reconcile.

(SCISPC090527-003) (c) 2009 Silverton Consulting, All Rights Reserved

We have redone the IO operations/second/Drive (IOPS/Drv) chart so that it now only include drives over 140GB as the smaller drives held too great an advantage here. All of the latest benchmark results show up in this Top 10 IOPS/Drv with HDS holding the number 4, 7, and 10 spots, IBM DS5300 at number 8, and 3PAR F400 at number 9. Also, as mentioned in a prior report, both the J4400 and J4200 results had no RAID protection so may not be suitable comparisons for normal customer environments.

Previously this chart had an error for the ETERNUS2000 that caused us to report double its actual IOPS/Drv rate. We have fixed that error and it now shows the correct IOPS/Drive for the ETERNUS2000.

(SCISPC090527-004) (c) 2009 Silverton Consulting, All Rights Reserved

Figure 4 SPC-1 Bubble chart of IOPS against LRT, bubble size proportional to subsystem price

Always looking for an interesting cut on SPC-1 data we came up with this new bubble chart. It shows a scatter plot view of subsystem performance with the x-axis as LRT and the y-axis as IOPS. The one thing always missing from a pure performance analysis is subsystem cost, added here as bubble size. To keep this interesting we capped the subsystems shown on the chart to a $100/GB maximum to eliminate the higher priced subsystems.

From an end-user perspective it’s interesting to note that one can obtain a reasonably performing subsystem (~100K IOPS with LRT <2msec) for about 1/4^th or less the price of truly high performance subsystem. Also, similarly priced or even more expensive subsystems can have much worse performance on an IOPS and/or a LRT basis indicating that pricing isn’t always the best factor in subsystem selection.

SPC-2, SPC-1C and SPC-2C results

There were no new SPC-2, SPC-1C and SPC-2C benchmarks released for this update and as such we stand with our last report SPC StorInt Dispatch&.

Significance

As we show above midrange subsystems can be high performers when configured properly. Although higher end subsystems dominate the Top 10 IOPS chart, the midrange subsystems discussed here have all managed to do well on most of the other performance metrics.

Over the years we have tried to come up with ways to compare performance to price and have used both $/IOPS and IOPS/$/GB (and others on occasion) as attempts to further this analysis. In the end we find that both of these metrics, although interesting on their own, leaves something out. The new bubble chart (see Figure 4 above) is our latest attempt to incorporate pricing information with subsystem performance. Hopefully, the reader will view this as a worthy addition to our SPC analysis. As always we welcome your feedback on how to do this better.

This performance dispatch was sent out to our newsletter subscribers in May of 2009. If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) or subscribe by email and we will send our current issue along with download instructions for this and other reports. Also, if you need an even more in-depth analysis of SAN storage system features and performance please take the time to examine our SAN Storage Briefing available for purchase from our website.

A PDF version of this can be found at

SCI 2009 May 27 Update to SPC benchmark results (PDF 616.1 KiB)

Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community

* All results from www.storageperformance.org as of 23 February 2009

# Prior SPC performance dispatches can be found at http://www.silvertonconsulting.com/page2/page2d/storage_int_dispatch.html

& Available at http://www.silvertonconsulting.com/page2/page2d/storage_int_dispatch.html

SCI 2009 Feb 24 latest SPC benchmark results analysis

Feb 242009

We once again return to discuss the latest Storage Performance Council (SPC) results*. Also, SPC has recently introduced new versions of its benchmarks to measure storage component performance. We report on these new results below.

SPC-1*results

There have been only two new SPC-1 results these past three months one for Pillar Data Axiom 600 and the other for Sun Storage 6780. Neither of these two made the top 10 in IOPS™, LRT™, or $/IOPS™ so these charts can be found in prior dispatches#. However, both of these products did make the top 10 for IOPS/$/GB (see Figure 1 below)

(SCISPC090224-001) (c) 2009 Silverton Consulting, All Rights Reserved

The big two players here from last report are 3PAR and SVC 4.3. Again, we calculate this metric as = IOPS / ($/GB) and created this metric as another way to factor in performance against cost and capacity.

subsystems, such as Xiotech’s Emprise may be capable of putting up high IOPS rates by aggregating a number of smaller subsystems but in their current instantiation, their relatively modest $/GB doesn’t compensate for the resultant IOPS and hence they cannot compete on this chart. In such a configuration, even when taking additional switch port costs into account, may still be significantly less costly than the systems shown here.

Again it would seem the FAS3170, Sun Storage 6780 and Pillar Axiom 600 look out of place here with these multi-million $ subsystems (3PAR, IBM SVC 4.3&4.2, Sun T9990V, HDS USP-V, and HP XP2400) but seems to provide relatively good performance for its price and capacity. Also, once again the Sun J4400 has no RAID protection whatsoever and probably does not belong here.

SPC-2 results

There were also two new SPC-2 benchmarks recorded for this update, both from Sun, one the SUN 6780 with RAID5 and the other the same system with RAID6. Both these systems are new entries in the top 10 MBPS™

(SCISPC090224-002) (c) 2009 Silverton Consulting, All Rights Reserved

There was a slight degradation in performance for the RAID6 over the RAID5 version of the Sun Storage 6780 of roughly 3%. All the other systems were reported on in prior SPC performance dispatches.

(SCISPC090224-003) (c) 2009 Silverton Consulting, All Rights Reserved

With this report SCI is introducing a new way to show subsystem performance for each of the three workloads that constitute the composite MPBS™ score. Here one can see some variability in the scale of workloads each vendor’s product can attain. There seems to be some interesting difference between the LDQ (large database query) workloads and the other two LFP (large file processing) and VOD (video on demand).

It’s unclear why the HP XP24000, HDS USP-V and the SVC 4.2 do so well on the LDQ workload as compared to the other two workloads. SCI believes the secret lies somewhere in their respective caching algorithms being optimized for LDQ and not as well optimized for the other two workloads. This may say that the other products such as the Fujitsu ETERNUS8000, Sun Storage 6780 and IBM DS5300 could improve their LDQ performance if they tweaked their caching algorithms somewhat.

New SPC-1C and -2C Results

SPC has been busy and have created new versions of their SPC-1 and SPC-2 benchmarks specifically to measure storage component performance. Currently there are not many released results. Also, SCI is having some difficulty trying to compare a 24 drive RAID subsystem against a single drive SATA system but other than that we believe the results are worthy of elaboration.

SPC-1C Results

SPC-1 results have always reported details for 10%, 50%, 80%, 90%, 95%, and 100% load levels with IOPS counts and response times in msec. 100% is based on exceeding a pre-determined response time cap, which SCI believes to be greater than 30msec.

(SCISPC090224-004) (c) 2009 Silverton Consulting, All Rights Reserved

For the single drive results instead of reporting IOPS™ which is at the 100% level SCI decided to use the 50% load level and plot error bars up to 80% load and down to 10%. At the 100% load point many of these single drives are producing response times just shy of the 30msec cut off. Unclear whether anyone would run these drives at that slow a response time. Thus, SCI decided to show the 50% load level with error bars. Somewhere around 80% load, the drive busy-ness takes over and how it manages its seek queue drives performance rather than seek speed.

All of these devices are running 7200RPM and have 1TB of storage. The first drive listed uses a SAS interface, all the others use SATA. LSI SAS3041X-R HBAs were used for all these runs. A couple of caveats here

Seagate sponsored all of these results. But the results as reported were from benchmark runs executed at the SPC lab with sponsored supplied hardware.
There are three other SPC-1C results, for 12, 24, and 15 drive RAID systems that don’t seem comparable to these results so they have been left out of this analysis.

(SCISPC090224-005) (c) 2009 Silverton Consulting, All Rights Reserved

Many end users are more interested in what a storage subsystem could do at a constant 10msec response time. As such, we introduce another new chart for the SPC-1C that depicts the expected IOPS at an average response time to 10msec. For most drives, the IOPS shown in the chart is an interpolated value. However for the Samsung Spinpoint F1 HE103UJ the value shown represents what they actually achieved at the 100% workload value.

For some reason both the Samsung devices seem to do well at 10msec response time while the others fall off. This could possibly be due to just different seek optimization profiles targeted at varying workload levels. The 50% workload level is typically closer to the 10msec mark for all the other drives. For the other Samsung drive, the 10msec response time is closer to its 95% load.

SPC-2C results

(SCISPC090224-006) (c) 2009 Silverton Consulting, All Rights Reserved

Finally, for the new SPC-2C results we show the various workloads in a radar chart format. Here we can obviously see the slowness of the VOD workload vis-à-vis the other workloads. Also there appears to be something amiss with the HDS Ultrastar drive during the LFP workload and the Samsung non-RAID class drive during the VOD workload. Possibly another artifact of varying seek optimization profiles.

Aside from the fact that these are 7200RPM drives, SCI would have assumed this chart would have looked closer to the subsystem SPC-2 chart shown earlier. Obviously, subsystem caching and data striping has an impact and can radically alter subsystem performance in comparison to drive performance on the same workloads. However, the difference between all these drives for the LDQ vs. VOD workloads are significantly different than what was shown on the earlier chart. For example, a maximum of 30% difference in performance between LDQ and VOD workloads on the subsystem chart vs. a minimum of almost 50% difference in performance for the current chart seems disquieting and worthy of more research.

The same caveats apply to SPC-2C that applied to SPC-1C results (see above).

Significance

SCI is always interested in understanding subsystem performance. The new SPC-1C and -2C benchmark results open up a new way to judge storage components rather than storage subsystems. Seagate has taken a shot at 7200RPM drives. It would be even more interesting to show 10K and 15KRPM drives, as well as HBA’s.

As much as we like the new SPC-1C and -2C benchmarks, some method needs to be implemented to insure comparable benchmarks are run. It’s unclear to SCI how to compare a 12, 15, and 24 drive storage subsystem against a single drive subsystem other than on a per drive basis. Of course this problem also occurs with the SPC-1 and SPC-2 benchmarks as well but at least there we are talking about complete storage subsystems rather than storage components. Perhaps as more multi-drive SPC-1C and -2C results are released we can use the per drive basis as a comparison.

Finally, SPC has once again allowed a sponsor to supply results for other vendor products (Seagate for SPC-1C and -2C results). SCI believes that while this may be expedient, it may hurt the long run value of SPC benchmarks. Because some non-sponsoring vendors feel slighted when not getting an opportunity to review, submit their own benchmark results, and/or reject the results entirely. Unclear how to solve this dilemma, given SPC’s stated policy but perhaps allowing the non-SPC member vendors to review and dispute benchmark results ahead of publication may be a necessary first step.

This performance dispatch was sent out to our newsletter subscribers in Febuary of 2009. If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) or subscribe by email and we will send our current issue along with download instructions for this and other reports. Also, if you need an even more in-depth analysis of SAN storage system features and performance please take the time to examine our SAN Storage Briefing available for purchase from our website.

A PDF version of this can be found at

SCI 2009 February 24 Update to SPC performance results (PDF 823.1 KiB)

Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community

* All results from www.storageperformance.org as of 23 February 2009

# Prior SPC performance dispatches can be found at http://www.silvertonconsulting.com/page2/page2d/storage_int_dispatch.html

SCI 2008 Nov 18 Latest SPC-1 and SPC-2 results analysis

Nov 182008

We once again return to our quarterly SPC results and as such we report on the latest benchmark submissions below.

SPC-1*results

(SCISPC081118-001) (c) 2008 Silverton Consulting, All Rights Reserved

There have been only four new SPC-1 results these past three months. IBM released a new SVC 4.3 and a new DS5300 benchmark, 3PAR released an Inserv T800 and Fujitsu has released a new ETERNUS Model 200 benchmark. None of these results changed the LRT Top 10 chart which still stands from last May.

However, the new benchmarks did impact some of the other charts SCI maintains for SPC-1 data. For example when we look at my favorite metric, IOPS™ and these new results have changed the top 10.

The new IBM SVC 4.3 and 3PAR results have moved into the top 10 IOPS™. The SVC 4.3 has an 8 node configuration with 8 separate DS4700’s behind it with a total of 1536 disks. 3PAR’s Inserv T800 also had an 8 controller node configuration with only 280 disks but also costs less than the IBM SVC 4.3 system. Of course the top result is still provided by TMS and their DRAM SSD subsystem.

(SCISPC081118-002) (c) 2008 Silverton Consulting, All Rights Reserved

As for other metrics, the SPC-1 $/IOPS™ Top 10 results have also changed but this time the culprit was the lowend Fujitsu ETERNUS2000 Model 200. Considering the fact that it is mirrored 146GB drive storage it’s not clear why the other systems couldn’t compete at this level other than expense. Again both the Sun J4400 and J4200 have no RAID protection and probably should not be compared to the rest of the subsystems.

(SCISPC081118-003) (c) 2008 Silverton Consulting, All Rights Reserved

Finally, the big jump up here is entirely in 3PARs favor but IBM’s SVC 4.3 has also made a significant uptick. Again, we calculate this metric as = IOPS / ($/GB) and created this metric as another way to factor in performance against cost and capacity. Both these products have significantly increased this metrics performance.

On the other hand, as has been pointed out to SCI, this metric may unfairly advantage big, monolithic subsystems at the expense of smaller subsystems. The monolithic subsystems generate such high IOPS counts that their relatively expensive $/GB doesn’t impact their ranking on this chart. In contrast, smaller subsystems, such as Xiotech’s Emprise may capable of putting up high IOPS rates by aggregating a number of smaller subsystems but in their current instantiation, their relatively modest $/GB doesn’t compensate for the resultant IOPS and hence they cannot compete on this chart. In such a configuration, even when taking additional switch port costs into account may still be significantly less costly than the systems shown on this chart.

Again the FAS3170 looks out of place here with these multi-million $ subsystems (3PAR, IBM SVC 4.3&4.2, Sun T9990V, HDS USP-V, and HP XP2400) but seems to provide relatively good performance for its price and capacity. Also, once again the Sun J4400 has no RAID protection whatsoever and probably should not be listed here.

SPC-2 results

(SCISPC081118-004) (c) 2008 Silverton Consulting, All Rights Reserved

There were also four new SPC-2 benchmarks recorded for this update, two for the IBM DS5300 (RAID 5&6) and one from HDS USP-V and the last from HP the XP24000 (an OEM version of the HDS USP-V). All four of these new SPC-2 results have cracked into the top 10 MPBS™.

As shown above the two HDS and HP benchmarks now reach over 8700 MBPS™ and the two DS5300s hit over 4600 MBPS™. Once again it is somewhat surprising that a relatively low-cost subsystem such as the IBM DS5300 can compete with all these million dollar subsystems (HP XP24000, HDS USP-V, and IBM SVC 4.2, 4.1, & 3.1)

Another place where the latest SPC-2 submissions have broken into the top 10 is in $/MBPS™. Perhaps it’s not surprising that the IBM’s DS5300 RAID 5 would beat out the RAID 6 version but it’s still somewhat confounding that the two HDS products with RAID 1 protection would crack into the top 10 here. Again, both the Sun J4200 and J4400 have no data protection and should probably not be compared to these other products.

This performance dispatch was sent out to our newsletter subscribers in November of 2008. If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) or subscribe by email and we will send our current issue along with download instructions for this and other reports. Also, if you need an even more in-depth analysis of SAN storage system features and performance please take the time to examine our SAN Storage Briefing available for purchase from our website.

A PDF version of this can be found at

SCI 2008 November 18 SPC performance update (PDF 1.0 MiB)

Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community

* All results from www.storageperformance.org as of 18 November 2008

SCI 2008 Oct 21 latest ESRP v2 (Exchange 2007) results analysis – under 1K mailboxes

Oct 212008

This is SCI’s third report on Microsoft Exchange Solution Review Program (ESRP)[1] performance results. This report focuses on 0 to 1000 mailboxes category and our prior reports discussed over 5000 and 1000 to 5000 mailboxes result categories. See our prior ESRP/Jetstress dispatches for a better description of ESRP benchmarks and our reported metrics.

Latest ESRP V2.0 results

(SCIESRP081021-001) (c) 2008 Silverton Consulting, All Rights Reserved

In order to better compare Jetstress results we report on both normalized and un-normalized results. For normalized results in this bottom-tier category we use operations per mailbox (mbx). As an example, Compellent reports results for 600 mailboxes at ~331, but normalized to per mbx their results are 0.5 database transfers/sec/mbx (see Figure 1).

We have added a new chart for this dispatch ranking the log file playback.. This metric depicts the average time a 1MB log file takes to be played against the database and is not always specified for ESRP results but all of these ESRP results in this bottom-tier include this metric.

The Top 3 normalized Jetstress results for the 0 to 1000 mailbox category were IBM DS3300 in Clustered Continuous Replication (CCR), Dell PowerEdge 2900 CCR and Sun Fire X4150 (see Figure 1). A few considerations on normalized results

Most of these results were for iSCSI subsystems, Dell and Sun results were SAS attached storage and the Compellent result was for FC attached storage.
Normalized results don’t always scale well. Although 5 of these results were for 1000 mailboxes (IBM, Dell, SUN, EMC NS20, Compellent, and one of the EMC CX3-20) the rest had an average of ~730 mailboxes. For example, although the AMI StorTrends 3200i may do well at 600 mbx it may or may not scale much beyond that.
Another surprise is that the normalized results span such a large range. The top 10 normalized results range from over 1.8 down to around .5 database operations per second per mbx over a factor of 3X, for relatively similar numbers of mailboxes. The IBM DS3300 is the clear winner here.

(SCIESRP081021-002) (c) 2008 Silverton Consulting, All Rights Reserved

The top 3 subsystems in this category are IBM DS3300 CCR, Dell and the Sun subsystems. In contrast to the normalized results above all of the unnormalized top 10 reported results are for 1000 mailboxes. There was one less iSCSI result here in comparison to the normalized results as the other Dell PowerEdge 3000 result was also SAS.

(SCIESRP081021-003) (c) 2008 Silverton Consulting, All Rights Reserved

There should be some correlation between the other results and the database backup results but apparently there is none. The top 3 results were all SAS subsystems and the next two were iSCSI and FC respectively. This implies that SAS has an advantage when it comes to raw data reading over iSCSI. Not sure why the FC product didn’t do better but that’s subject for another discussion. Again the storage group results bear little relation to the aggregate database backup results but the number of storage groups is one of those vendor-optimized variables that confound ESRP/Jetstress result comparisons.

(SCIESRP081021-004) (c) 2008 Silverton Consulting, All Rights Reserved

Our first report on Log playout results does not appear that interesting. However, the Log results don’t correlate that well with any of the previous results. As the ESRP defines this metric, this sort of activity would typically occur during a crash recovery. Given these results all the subsystems are pretty comparable. But if crash recovery time is important perhaps the Dell PowerEdge 2900 or EMC CX3-20 may be worth a look.

Conclusions

From our view on ESRP results in this bottom-tier competition is abating. The only new benchmark during the past quarter for this tier was the AMI result at 600 mailboxes. In fact, if one closely examined this and our prior ESRP result dispatches they would notice that some results showed up twice. All results for 1000 mailboxes were reported on in both this tier and the next higher tier (1K to 5K mailboxes). We did this because there are so few results <1000 mailboxes.

iSCSI storage represents the majority of reported results in this category although the few SAS subsystems do almost as well. SAS does seem to have an advantage in backup speeds but other than that iSCSI handles the other workloads better. FC is just not well represented in this tier probably due to cost.

This is our third ESRP report and we have nov analyzed top 10 results for every category. We continue to welcome any feedback on how to do better. Jetstress results are inherently un-comparable. Nonetheless we believe Exchange results deserve some comparison and result rankings so that the public can be properly informed and as such make better storage purchases. Our next ESRP/Jetstress report will return to the >5K mailbox tier.

This performance dispatch was sent out to our newsletter subscribers in October of 2008. If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) or subscribe by email and we will send our current issue along with download instructions for this and other reports. Also, if you need an even more in-depth analysis of SAN storage system features and performance please take the time to examine our SAN Storage Briefing available for purchase from our website.

A PDF version of this can be found at

SCI 2008 October 21 Update to ESRP/Jetstress benchmark performance results (PDF 1003.7 KiB)

Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community

[1] ESRP results from http://technet.microsoft.com/en-us/exchange/bb412164.aspx, as of 21 October 2008

Older Entries

Hitachi VSP

VSP hardware

VSP functionality

VSP performance

Announcement significance

SPC-1*results

Significance

SPC-1*results

Significance

Easy Tier

Other changes with DS8700 5.1

Announcement significance

SPC-1*results

Significance

SPC-1*results

SPC-1/E

SPC-2 results

Significance

SPC-1*results

SPC-2, SPC-1C and SPC-2C results

SPC-1*results

SPC-2 results

New SPC-1C and -2C Results

SPC-1C Results

SPC-2C results

Significance

SPC-1*results

SPC-2 results

Latest ESRP V2.0 results

NEWS and EVENTS

TOP DOWNLOADS

TWITTER FEED...

POPULAR POSTS

NEWSLETTER

TRANSLATE THIS PAGE

Dispatch Tag Cloud

Dispatch categories

Dispatch archives