SCI’s latest SPC performance report as of November’16

In DS8880, DS8888, IBM, IOPS, LRT, SPC, SPC-1, Spectrum Virtualize, Storwize V5000 by Administrator0 Comments

This Storage Intelligence (StorInt™) dispatch covers Storage Performance Council (SPC) results[1]. SPC has recently released a new version of their SPC-1 benchmark, V3. There have been no new SPC-1 V1 submissions since our last report but there has been 1 new V3 submission, the IBM DS8888 (AFA). For now, we will include all V3 and V1 SPC-1 submissions together in our analysis. The major IO difference between the two is that V3 uses a larger fixed block size than V1, (8KB vs. 4KB)[2] and while this might take an extra microsecond or two to transfer at 16Gbps, it seems unlikely to impact IOPS and LRT results that much. There have been no new SPC-2 submissions

SPC-1 results

We begin our discussion with top ten SPC-1 LRT™ (Least Response Time) results shown in Figure 1.

Figure 1 Top 10 SPC-1 LRT results

SPC-1 V3 no longer reports on LRT but there is a similar metric (RAMPD_10 Average Response IO time) in the detailed metrics provided in the submission. From now on we will just refer to this as LRT but note LRT is not formerly reported anymore. There are no changes in Figure 1 from last quarter’s review but the IBM DS8888 just missed the top ten with an LRT*[3] of 0.254 msec.

Next, we turn to SPC-1 top ten IOPS™ in Figure 2.

Figure 2 SPC-1 Top 10 IOPS™

Here the IBM DS8888 came in 5th place with 1.5M IOPS. The DS8888 had 240 400GB Flash modules with 2048GB of DRAM memory and used 16Gbps FC as the frontend interface. As this was SPC-1 V3 the DS8888 was also doing larger fixed block IOs than the (V1) competition but again the difference should be negligible. There is only one hybrid (disk-SSD) system that remains on our SPC-1 IOPS Top Ten and that is the Huawei OceanStor 18800 (#8).

We turn now to IOPS/Drive for disk-only systems in Figure 3.

 Figure 3 Top 10 SPC-1 IOPS/(disk-)drive

In Figure 3, the IBM Storwize V5030 system came in at #9 with 354 IOPS/disk drive. It had 240 300GB 15KRPM disk drives with 64GB of DRAM memory and the disks were SAS attached to the two controllers. As you may recall, we mentioned the IBM Storwize V5030 as the first V3 submission in our last report but at the time had not incorporated it into our (V1) performance analysis, we do so now. Again, the SPC-1 V3 benchmark does larger fixed block IOs (8KB vs 4KB) which might be a slight disadvantage but probably isn’t measurable at this scale.

It’s unusual to see a new disk-only SPC-1 submission, especially from a major vendor (IBM). It’s almost as if they no longer want to even talk of disk-only storage anymore. And yet we believe, there’s still room for performance improvement even in disk-only storage, let alone hybrid systems. 

Significance

For the moment, after reviewing both the V3 and V1 specification for SPC-1 we have concluded to just incorporate them both into the same SPC-1 analysis. It’s unclear whether SPC has stopped accepting new submissions with SPC-1 V1. We reserve the right to split V3 submissions out if we determine that the V3 and V1 workloads are different enough to warrant their separation.

AFA systems continue their march to dominating LRT and now IOPS top ten positions. There remains only one hybrid storage system left in the IOPS top ten but it’s being slowly pushed off by all the new AFA storage system performance.

As always, suggestions on how to improve any of our performance analyses are welcomed.

[Also we offer more block storage performance information plus our OLTP, Email and Throughput ChampionsCharts™ in our recently updated (August 2017) SAN Storage Buying Guide, or for more information on protocol performance results please see our recently updated (July 2017) SAN-NAS Storage Buying Guide, both of which are available for purchase on our website.]

[This performance dispatch was originally sent out to our newsletter subscribers in November of 2016.  If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) and we will send our current issue along with download instructions for this and other reports. Dispatches are posted to our website at least a quarter or more after they are sent to our subscribers, so if you are interested in current results please consider signing up for our newsletter.]  

Silverton Consulting, Inc., is a U.S.-based Storage, Strategy & Systems consulting firm offering products and services to the data storage community

[1] All SPC results available from http://www.storageperformance.org/home/ as of 24Nov2016

[2] SPC-1 V3 specification can be found at http://www.storageperformance.org/specs/SPC1_v320_final.pdf

[3] The RAMPD_10 Avg. Response IO time metric can be found in SPC1_Metrics_0_Quick_Look.xlsx file in Supporting Files for SPC-1 V3 benchmark submissions

[4] Available at http://silvertonconsulting.com/cms1/product/san-storage-buying-guide/