SCI’s latest SPC performance report as of May’17

In $/MBPS, MBPS, MPBS/drive, Oracle ZFS ZS5-2, SPC-2 by Administrator0 Comments

This Storage Intelligence (StorInt™) dispatch covers Storage Performance Council (SPC) results[1]. There have been one new SPC1 v3(MacroSAN MS3000G2) submission and two new SPC-2 v1.6 (Oracle ZFS Storage ZS5-2 AFA and Oracle ZFS Storage ZS5-2 disk-only) submissions since our last report. Although the MacroSAN was flash only storage, it didn’t manage to break into any of our top ten SPC-1 charts. As such for this report we concentrate our analysis on SPC-2, where the two Oracle submissions present an interesting story in contrast.

SPC-2 results

We begin our discussion with top ten SPC-1 MPBS by workload type, where we use a new chart type, shown in Figure 1.

Figure 1 Top 10 SPC-2 MPBS results by workload type

In Figure 1, we sort the systems by SPC-2 MBPS™ (not shown on the chart) and we can see the new Oracle ZFS Storage ZS5-2 AFA and disk-only systems came in at #8 and #9 in MBPS with an aggregate MPBS of 24.4 GB/sec and 19.6 GB/sec, respectively. One thing to note about these two Oracle ZFS submissions is that the AFA storage had 40 800GB SSDs and the disk-only version had 288 600GB disk drives. The disk-only version had smaller and more ~7.2X drives than the AFA SSDs but still only managed ~80% of the AFA’s throughput.

We normally show this analysis in a spider chart but we now believe the clustered column chart shows a better view. Nonetheless, the disk-only system did worse across all workloads and it might be hard to see on Figure 1 but it performed relatively better on the LDQ workload (~85% of AFA, 24.8GB/sec vs. 28.8GB/sec) and slightly worse on the LFP workload (~72% of AFA, 14.4GB/sec vs. 20.0GB/sec). For the VoD workload, the disk-only version was ~80% of the AFA, at 19.7GB/sec vs. 24.4GB/sec.

It is worth mentioning-that the Oracle ZFS Storage ZS5-2 disk-only system also had more total ASU capacity (47.6TB vs. 8.4TB), potentially allowing them to spread the IO workload across all their drives. Just to complete the story, the Oracle ZFS ZS5-2 disk-only version also cost more ($253.6K vs. $217.0K) than the AFA system.

In Figure 2 we examine SPC-2 Top Price Performance™ or $/MBPS metric reported by SPC.

Figure 2 SPC-2 Top 10 $/MBPS performance

In Figure 2 the same two Oracle ZFS Storage ZS5-2 systems ranked #3 and #5 in price performance, respectively. For the Oracle ZS5-2 AFA the price performance was $8.89/MBPS and for the disk-only version the price performance was $12.93/MBPS.

Next, for disk-only systems we have historically reported on IO performance per drive metric shown in Figure 3.

Figure 3 SPC-2 Top 10 MBPS/disk drive

In Figure 3 the new Oracle ZFS Storage ZS5-2 came in at #3 with 68.1MBPS/disk drive, right in the middle of 2nd tier systems which all had between 65 and 70 MBPS/disk drive. The Oracle ZFS ZS5-2 used 10Krpm drives, which most of this top ten also used, the exceptions being #1 and #4-5, which we believe had 15Krpm drives.

Next, we turn to a never reported metric, the SPC-2 top composite response times, in Figure 4.

Figure 4 SPC-2 Top 10 composite min. avg. response times

We must start by stating that response times aren’t normally reported for bandwidth intensive workloads. They seem less important here than OLTP IO but we would disagree. Waiting for a video to start to play, a large file transfer to begin or a large database query to initiate is still a valid a concern, even though the end of these activities take sheer throughput to complete.

The data for average response times have always been available in the SPC-2 FDR. We use a composite number, as each workload has different characteristics and this is not an average (like SPC-2 MPBS), it is based on a proprietary formula which we also use to rank systems in our SCI SAN Storage Buying Guide[2] to rate storage systems.

That being said, the new Oracle ZFS Storage ZS5-2 AFA shows up as #8 in this top 10 with a composite response time of 0.8 msec., respectable but not that great for an AFA system. Unlike SPC-1, top response times in SPC-2 are not all AFA, in fact only 6 of these are AFA systems.

Significance

The new SPC-2 results were for v1.6 of the benchmark. SPC reports these results along with v1, and herein we follow their lead. Bandwidth benchmark submissions seem less popular than SPC-1 and as such, it’s unusual to see one new SPC-2 submission, seeing two is a treat.

Furthermore, having the same storage system in a AFA and a disk-only version for SPC-2 presents an interesting case study on the merits of SSD vs. disk. However, the days of disk-only systems doing best on bandwidth workloads seem to be numbered. The lone advantage left for disk-only is $/GB but even that advantage needs scrutiny, as often you need more disk drives (and thus capacity) just to optimize performance on a disk-only storage system.

As always, suggestions on how to improve any of our performance analyses are welcomed.

[Also we offer more block storage performance information plus our OLTP, Email and Throughput ChampionsCharts™ in our recently updated (August 2017) SAN Storage Buying Guide, or for more information on protocol performance results please see our recently updated (October 2017) SAN-NAS Storage Buying Guide, both of which are available for purchase on our website.]

[This performance dispatch was originally sent out to our newsletter subscribers in May of 2017.  If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) and we will send our current issue along with download instructions for this and other reports. Dispatches are posted to our website at least a quarter or more after they are sent to our subscribers, so if you are interested in current results please consider signing up for our newsletter.]  

Silverton Consulting, Inc., is a U.S.-based Storage, Strategy & Systems consulting firm offering products and services to the data storage community

[1] All SPC results available from http://www.storageperformance.org/home/ as of 24Nov2016

[2] Please see http://silvertonconsulting.com/cms1/product/san-storage-buying-guide/

[3] Ibid