SCI 2010 Nov 18 Latest SPC-1 results

In Emprise 5000, FAS/V3270, Huawei Symantec, IOPS, ISE, LRT, NetApp, S8100, SPC, SPC-1, Storwize V7000, X-IO Storage (nee XIOTech) by Administrator2 Comments

We return now to that classic block storage benchmark, Storage Performance Council (SPC) results*.  There have been five new SPC-1 and -1/E submissions since our last report, namely two from Xiotech (Emprise 5000 with 300GB and 600GB drives), IBM Storwize V7000, Huawei Symantec S8100 4-node, and NetApp FAS3270A (SPC-1/E).  You may recall that SPC-1/E is an energy monitored version of the SPC-1 test so we have been combining these two test results into our SPC-1 analysis

SPC-1*results

The chart that has changed most since our last report is the IOPs/drive results.

SCISPC101118-001 (c) 2010 Silverton Consulting, Inc., All Rights Reserved

SCISPC101118-001 (c) 2010 Silverton Consulting, Inc., All Rights Reserved

Figure 1 Top 10 SPC-1* IOPS/Drive spindle

Higher is better on the IOPS/drv chart.  Recall that our IOPS/drive chart excludes drives under 146GB and eliminates systems that use SSDs or Flash Cache.  The #1 and 6, (new Xiotech 300GB and 600GB, respectively) results on this chart are new for this quarter.  On a purely IOPs per drive perspective, Xiotech seems to be doing quite well with 5 out of the top 10.  Obtaining ~400 SPC-1 IOPS/drive out of a 15Krpm/300GB disk drive seems pretty good to me, considering the raw drive probably can do about ~286 average random read seeks per second (e.g. Cheetah 15K.5/300GB drive with an average 3.5msec read seek) and this doesn’t account for rotational latency or data transfer time.

Silverton Consulting, Inc. StorInt™ Dispatch
SCISPC101118-002 (c) 2010 Silverton Consulting, Inc., All Rights Reserved

SCISPC101118-002 (c) 2010 Silverton Consulting, Inc., All Rights Reserved

Figure 2 Top LRT™ results

Lower is better on the LRT. The top Least Response Time (LRT) chart also has new entries coming in at #5 and #10 for the new NetApp FAS3270 and Xiotech Emprise 5000/300GB drive subsystems.  As the NetApp system has 1TB of Flash Cache, it represents an interesting mid-level performance between the all SSD systems to the left and the mostly [except HP EVA 6400 and IBM DS8700 (R5.1) w/SSDs] all disk systems to the right of them.  It stands to reason that SSD and Flash Cache systems would perform well on LRT but the three all disk drive systems showing up here seem to have some secrets to tell as well on how to perform faster.

SCISPC101118-003 (c) 2010 Silverton Consulting, Inc., All Rights Reserved

SCISPC101118-003 (c) 2010 Silverton Consulting, Inc., All Rights Reserved

Figure 3 Top IOPS/$/GB

Higher is better on IOPS/$/GB which is an attempt to normalize performance.  The other new winner was the recent Huawei submission in the IOPS/$/GB category.  Huawei’s system pricing is listed in CNY and we use a conversion rate based on the day of our analysis report (0.1507$:1CNY as of 11/10/2010).  Although conversion rates may shift over time, given Huawei’s low cost (CNY 7,847,296) and its high IOPS rate (~160K) it’s liable to be in the top 10 IOPS/$/GB for quite some time.


SCISPC101118-004 (c) 2010 Silverton Consulting, Inc., All Rights Reserved

SCISPC101118-004 (c) 2010 Silverton Consulting, Inc., All Rights Reserved

Figure 4 SPC-1/E IOPS per Watt under varying workloads

Higher is better on IOPS/Watt chart.  The SPC-1/E benchmark submissions all report on IOPs/Watt measurements taken at distinct IO workloads.  At this time these four represent the only available submissions for SPC-1/E.

As discussed previously[1] the three SPC-1/E workloads (nominal, medium and high) all measure different IO intensity and idle time.  Also realize that the HP EVA 6400 only has SSDs.

One can see the impact of NetApp’s Flash Cache and HP’s SSDs at both the medium and high workloads as compared to the Xiotech systems (which only have disks).  Moreover, we mentioned last time that the significant IOPS/Watt differences for the nominal workload seem entirely due to the SSDs & Flash Cache drawing similar power across all workloads but delivering correspondingly less IOPS at nominal with its high proportion of idle time.  In contrast, the Xiotech all disk systems seem to draw less power when little used or idle.

Significance

Other performance charts for SPC-1 and SPC-2 were unchanged except for some of the SPC-1 scatter plots and have been discussed in previous reports.  There have been no new SPC-2 benchmark submissions over the past two quarters, which seem to have been slowing down of late – unsure why.

There was one additional SPC-1C submission from Oracle but I am having some difficulty comparing a component level performance with Oracle’s 3-Flash Accelerator (PCIe) cards against single and multiple drive submissions.  Perhaps I will figure this out for next time.

NetApp seemed to have refreshed their entire mid-range and enterprise product lines this past week and the FAS3270 is the top of their mid-range line.  I was disappointed not to see anything on the HDS VSP subystem, also new as of last month, but maybe they’re too busy shipping systems to benchmark one.  We will certainly let you know as these products show up in future SPC analyses.

As always, any suggestions on how to improve our SPC analysis are welcomed.  Additionally, we now provide a fuller version of all these charts included in SCI’s SAN Storage Briefing which can be purchased separately on our website[2].

 

This performance dispatch was sent out to our newsletter subscribers in November of 2010.  If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) or subscribe by email and we will send our current issue along with download instructions for this and other reports.  Also, if you need an even more in-depth analysis of SAN storage system features and performance please take the time to examine our recently revised (November 2018) SAN Storage Buying Guide available for purchase from our website.

A PDF version of this can be found at

[wpfilebase tag=’file’ id=’155′ tpl=’just-link’]

Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community.


* All results from www.storageperformance.org as of 10 Nov 2010

 

[1] Available at http://silvertonconsulting.com/cms1/news-4/

[2] Available at http://silvertonconsulting.com/cms1/shop/

Comments

  1. Hi Ray,

    This is great stuff — thanks especially for highlighting the IOPS/spindle aspect. As you know, depending on the choice of array vendor, the same workload might require twice as many spindles using an inefficient array.

    Couple of notes however;

    You said the IOPS/drive chart “excludes systems that use SSDs or Flash Cache”. I note that the NetApp FAS3270A uses 2x Flash Cache ‘PAM’ modules (~$100K worth) and should therefore be excluded here. NetApps best previous result was only 224 IOPS per (15K) spindle, making NetApp one of the least efficient architectures for this kind of workload. This appears to be the result of their RAID-6 implementation and the “write penalty”.

    2) The Xiotech ISEs with 600GB disks (showing at #2 on your chart) tested at 12,603 IOPS from 40 spindles (315 IOPS/spindle), so it should be much further down the list.

    Finally, a couple that are not on your chart yet are;

    (1) the Xiotech model tested with 20x300GBx15K drives – impressive performance per spindle, but at $28,000 per TByte (list price for 2.3TB usable capacity), it costs as much as Flash SSD!?!?!

    (2) the new IBM D3524 (LSI 2600) results look fantastic. 308 IOPS per spindle (using 10K disks and 94% of disk stroke), at $7,000/TByte and $3.26/IOPS — both are list price (no discounting tricks). I don’t think I’ve ever seen an SPC-1 result that clobbered both cost/IOPS and cost/Capacity metrics on the same test.

    Looking forward to your next update…thanks again.

    KD

    1. Author

      KD,

      Thanks for your comment and good catches.
      – Yes, I missed the use of the Flash Cache on the FAS3270A, and have removed it from my next SPC update on this chart.
      – The #3 Xiotech ISE was labeled wrong, it was actually the Xiotech with 20-300GB drives.
      – The New IBM DS3524s also look pretty good in my latest SPC performance update (due out late this month to my newsletter subscribers).

      Sorry, I didn’t see this earlier as I hadn’t set the dispatch posts to use intense debate. I will fix that shortly and should be more responsive in the future.
      Ray

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.