SCI 2010 Feb 23 latest SPC-1 & SPC-2 benchmark results analysis

In DS8700, IBM, IOPS, LDQ, LFP, LRT, MBPS, SPC, SPC-1, SPC-2, SVC, VOD by Administrator

We once again return to the classic block storage benchmark, the latest Storage Performance Council (SPC) results*.  Not much new in SPC-1C/E or -2C results so only SPC-1 and SPC-2 activity is discussed below.

SPC-1*results

(SCISPC100223-001) (c) 2010 Silverton Consulting, All Rights Reserved

(SCISPC100223-001) (c) 2010 Silverton Consulting, All Rights Reserved

There have been three new SPC-1 results this past quarter – two based on IBM SVC5.1 with DS8700 backends and one from Infortrend. Both new SVC5.1s managed to crack into the top 10 in IOPS™ performance as #1 and 2 with similar backend DS8700 hardware using 1024-146GB 15Krpm drives in two DS8700’s, 384GB of cache, and 16-4GFC connections each.  Each SVC node had 24 GB of memory/cache running with 4-8GFC connections.  The only difference between the two new SVC results was the number of nodes (6 for the top result and 4 for #2).

We have discussed other subsystems on this top 10 chart in previous dispatch editions and do not reiterate that here#. However, the rest of these subsystems seem to now pale in comparison to the 6-node SVC run.

(SCISPC100223-002) (c) 2010 Silverton Consulting, All Rights Reserved

(SCISPC100223-002) (c) 2010 Silverton Consulting, All Rights Reserved

The only other changes for our other reported top 10 SPC-1 charts was on the IOPS/$/GB.  This is sort of a weird metric as it combines IOPS rate, subsystem cost and capacity into one performance number.

Here one can see the Infortrend storage subsystem hitting #1 on the top 10 and the SVC subsystems coming in as #3 and #7 respectively.  Infortrend was able to do so well because it combined high performance (180K IOPS), relatively low price (under $1M USD) with reasonable capacity (~49TB).  What strikes one when looking at this chart is that the 6-node SVC is so much better (almost 2X) than the 4-node subsystem.  Realize that the added performance for the 6-node subsystem at only small additional cost (~$31K USD difference) with equivalent capacity drove this IOPS/$/GB.  So if you’re interested in IOPS performance and running IBM SVC5.1, add nodes.

(SCISPC100223-003) (c) 2010 Silverton Consulting, All Rights Reserved

(SCISPC100223-003) (c) 2010 Silverton Consulting, All Rights Reserved

In previous discussions we showed SPC-1 scatter plots for IOPS vs. LRT, IOPS vs. $/GB.  In this report we now add IOPS vs. Subsystem Capacity.

What’s surprising here is the high correlation (R**2 of ~0.8) between capacity and performance.  Any subsystem above the line on this chart gets better performance out of its subsystem capacity than the norm.  This makes sense for the two IBM SVC runs (one above and the other below with equal capacity).  However the one significant outlier is TMS RAMSAN at around 250K IOPS using SSD, which has relatively small capacity and as such, shows up extremely well here.  The other, less significant outlier at around 275K IOPS with ~60TB was an IBM SVC4.3 benchmark run.

SPC-2 results

(SCISPC100223-004) (c) 2010 Silverton Consulting, All Rights Reserved

(SCISPC100223-004) (c) 2010 Silverton Consulting, All Rights Reserved

There have been only one new SPC-2 submission for the IBM DS8700 and it did break into the top 10 MPBS™ at #3 (see Figure 4. above).  The DS8700 used RAID5 and 300GB disk drives for its top 10 performance.  It’s somewhat surprising that there were no new SVC5.1 SPC-2 benchmark submissions but then perhaps they exhausted their budget doing the SPC-1 runs.

(SCISPC100223-005) (c) 2010 Silverton Consulting, All Rights Reserved

(SCISPC100223-005) (c) 2010 Silverton Consulting, All Rights Reserved

We have added the IBM DS8700 to our Top 10 MPBS spider chart showing the actual performance for all three classes of workloads in the SPC-2 benchmark.  Once again, HDS and IBM seem to have found some magic formula for their caching that allows their LDQ (large database query) to be significantly better than the other two workloads (VOD-video on demand and LFP-large file processing).  The IBM DS8700 LDQ performance is almost 30% better than their other results, fairly significant from my perspective.

Significance

It seems some of the lesser SPC benchmarks are not gaining as much traction as subsystem level benchmarks.  This could be a matter of time or maybe just popularity.  I see SPC has yet another benchmark coming, called the SPC-3BR benchmark for backup restore.  We can only applaud SPC for introducing more benchmarks but just wish more vendors would submit results for what’s currently available.

This performance dispatch was sent out to our newsletter subscribers in February of 2010.  If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) or subscribe by email and we will send our current issue along with download instructions for this and other reports.  Also, if you need an even more in-depth analysis of SAN storage system features and performance please take the time to examine our recently revised (November 2017) SAN Storage Buying Guide available for purchase from our website.

A PDF version of this can be found at

SCI 2010 February 23 Review of Latest SPC-1 and SPC-2 Performance results

Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community.


* All results from www.storageperformance.org as of 27 August 2009

 

# See http://silvertonconsulting.com/cms1/news-4/