SCI 2009 Aug 27 latest SPC-1 benchmark results analysis

In IOPS, LRT, SPC, SPC-1 by Administrator

We once again return to the classic block storage benchmark, the latest Storage Performance Council (SPC) results*.  Also we report on the new SPC-1C/E energy usage benchmark for the first time.

SPC-1*results

There have been only two new SPC-1 results this past quarter, one for IBM DS5300 with encryption and the other for IBM DS5020 Express. Neither of these made the top 10 in IOPS™, $/IOPS™ or IOPS™/$/GB so these charts can be found in prior dispatches#.  However, one of these products did make the top 10 for LRT and both products made the top 10 for IOPS/drive (see below).

(SCISPC090827-001) (c) 2009 Silverton Consulting, All Rights Reserved

(SCISPC090827-001) (c) 2009 Silverton Consulting, All Rights Reserved

Actually this shows the top 12 as there is currently a three-way tie for 10th place.  Both the IBM DS5300 with and without full drive encryption (FDE) had the same LRT at 1.8msec.  From my perspective, the good news is, full drive encryption seems to have a negligible (un-measurable) affect on response time.  Considering also that FDE has little to no impact on IOPS performance or cost, it should easily become widely adopted in this class of subsystems.

(SCISPC090827-002) (c) 2009 Silverton Consulting, All Rights Reserved

(SCISPC090827-002) (c) 2009 Silverton Consulting, All Rights Reserved

Figure 2 Top 10 IOPS/drive results

Actually this shows the top 11 as there is a tie for 9th place.  The latest IBM DS5020 Express (channel ready product) managed to be the top subsystem here with an average IOPS per drive of 326.  Also the DS5300 with encryption made it into the top 10 matching the performance of the DS5300 without encryption.  As can be seen above, most products that do well here are midrange and not the high-end subsystems.

Recall that in our last report we now restrict this chart to only those subsystems using 140GB drives or larger.  Also, this metric is for only those subsystems using rotating media.

(SCISPC090827-003) (c) 2009 Silverton Consulting, All Rights Reserved

(SCISPC090827-003) (c) 2009 Silverton Consulting, All Rights Reserved

We return to the IOPs versus LRT bubble chart we have shown previously.  The one thing always missing from a pure performance analysis is subsystem cost, added here as bubble size.  To keep this interesting we capped the subsystems shown to a $100/GB maximum to eliminate the higher priced subsystems.

From an end-user perspective it’s interesting to note that one can obtain a reasonably performing subsystem (~100K IOPS with LRT <2msec) for about 1/4th or less the price of truly high performance subsystem.  Also, similarly priced or even more expensive subsystems can have much worse performance on an IOPS and/or a LRT basis indicating that price isn’t always the best factor to predict subsystem performance.

SPC-1C/E

SPC has a new benchmark category that displays energy usage for storage subsystems. So far there have only been two results submitted, IBM System Storage EXP12S with SSDs and Seagate Savvio 10K.3 ST93000603S and show some interesting results. Both subsystems use a SAS interface with no RAID protection.

(SCISPC090827-004) (c) 2009 Silverton Consulting, All Rights Reserved

(SCISPC090827-004) (c) 2009 Silverton Consulting, All Rights Reserved

As can be inferred from the above, the big bubbles are an IBM SSD system and the small bubbles are the Seagate Savvio 10Krpm drive system.  The Blue bubbles show the “nominal” energy use and IOPS that are obtained from the two systems.  Yellow bubbles show similar data at “peak” IOPS.

Nominal energy use is an aggregate workload constructed by SPC to show typical energy use and IOP/second obtainable from subsystems. This data is summarized and reported directly off the SPC-1C/E report.  Peak IOPs & energy use took some hunting, but represents the max IOPS and the energy consumption at that IO rate.

I suppose it’s no surprise that the rotating media would consume more power for higher IO operations per second.  However, the difference in power consumption for the IBM’s SSD subsystem at nominal versus peak IO rates is almost negligible (less than 2%).  We are not sure which is the more fair comparison so we have elected to show both.

We do not show idle subsystem comparison but both have similar energy consumption. As such, rotating media has a much more slanted energy profile than SSD storage.

SPC-2, SPC-1C and SPC-2C results

There were no new SPC-2, SPC-1C and SPC-2C benchmarks released for this update and as such, can be found in prior reports.

Significance

Power use is starting to take on more importance.  We should all applaud SPC for instituting a new energy use benchmark.  As more results show up, it should become much more interesting but having SSDs vs. a 10Krpm subsystem as starters works well.

The fact that drive encryption seems to have no impact on performance or subsystem cost seems counterintuitive.  By all accounts, there should have been at least some cost differential and certainly some performance degradation.  Once more my assumptions are overturned by reality.  Drive encryption seems to be here and deserves higher adoption if these results can hold up.

As always we welcome any feedback on how to do this better.  So, if you have any comments please don’t hesitate to contact us.

This performance dispatch was sent out to our newsletter subscribers in August of 2009.  If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) or subscribe by email and we will send our current issue along with download instructions for this and other reports.  Also, if you need an even more in-depth analysis of SAN storage system features and performance please take the time to examine our SAN Storage Briefing available for purchase from our website.

A PDF version of this is available at

SCI 2009 August 27 Update to SPC performance results

Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community


* All results from www.storageperformance.org as of 27 August 2009

 

# Prior SPC performance dispatches can be found at http://www.silvertonconsulting.com/page2/page2d/storage_int_dispatch.html