SCI’s latest SPC-1 & SPC-2 performance report as of February’16

In $/IOPS, DataCore, Huawei OceanStor 6800 v3, LDQ, LFP, LRT, MBPS, SANSymphony V10, SPC-1, SPC-2, VOD by Administrator0 Comments

This Storage Intelligence (StorInt™) dispatch covers Storage Performance Council (SPC) results[1]. There have been three new (and one updated) SPC-1/1E and one new SPC-2 submissions since our last report last November. The new SPC-1/1E results are for the DataCore SANSymphony V10, the MacroSAN MS5520 and the NEC Storage M510 and the new SPC2 submission was Huawei OceanStor 6800 V3. We have a mix of all flash, hybrid and all disk storage systems this time.

SPC-1 results

We begin our discussion with top ten SPC-1 LRT™ (Least Response Time) results shown in Figure 1.

SCISPC160224-001Figure 1 Top 10 SPC-1 LRT results

The new DataCore SANSymphony V10 is our new #1 with an LRT of 80msec. SANSymphony in this submission acted as a hyper converged, software defined storage system, only with 16 SATA attached, SFF Samsung 480GB SSDs and 8 15Krpm 300GB disk drives, with about 419GB of DRAM cache. Even at 100% of IO workload their response time was only 0.32msec.

There are some advantages to using hyper converged solution for SPC: 1) there’s no storage network you have to traverse, they were using a internal loop back Virtual iSCSI logical interface to the storage and 2) they get to share the memory with the SPC benchmark driver, so once the data was in cache it was a memory to memory transfer to read or write it from/to the storage system.

We talked with DataCore on this SPC-1 submission and they said the other thing which helped them is enhanced parallelism. It seems like they have re-discovered multi-thread/multi-core code design. DataCore seemed to have come of age in high performance computing environment and when they went to storage virtualization they had to limit their multi-processing/parallelism. But with all the cores in today’s, off the shelf, servers, they were able to eliminate most, if not all these limitations and the results are what you see above.

I am just not sure this is entirely fair to the other vendors, many major [and minor] vendors offer software defined storage options of their systems and any of them could have submitted a SPC-1 benchmark. But I will give DataCore credit for being the first here and see what the industry response looks like over time.

Next, we turn to one of SCI’s flash based metrics, top ten SPC-1 IOPS/GB-NAND in Figure 2.

SCISPC160224-002Figure 2 SPC-1 Top 10 IOPS™/GB-NAND

First let me clarify, when I claim a SPC-1 submission as “all-SSD” or “all-flash” I am considering any system with more than 50% of ASU capacity in SSDs as an all-flash system. Yes many of these have disk drives (e.g. DataCore) but the vast majority of their capacity is in flash and I believe it’s only fair to place them in the all flash bucket.

In Figure 2, again we can see the results from DataCore’s submission, which came in at #5 on the IOPS/GB-NAND metric. Historically we have used a IOPS/drive metric to gauge the efficiency of disk-only storage systems but we have yet to settle on one measure for flash or hybrid based systems but have been migrating to IOPS/GB-NAND, as NAND is commodity priced raw-storage and seems to be a reasonable counterpart to drive normalization. We have used SSDs in the past as well. But for some reason this chart seems to display a more diverse product list from high-end enterprise solutions down to SMB storage are all on this top 10.

The other thing that argues for IOPS/GB-NAND vs. IOPS/SSD is that some vendors create their own flash modules and as such have an unfair advantage when it comes to SSDs. But the debate is still on and last quarter we showed IOPS/SSD so this quarter we went with IOPS/GB-NAND.

The last SPC-1 chart is the top ten $/IOPS or SPC-1 PricePerformance in Figure 3.

SCISPC160224-003Figure 3 SPC-1 Top $/IOPS

In Figure 3, we display SPC-1’s PricePerformance top ten and DataCore is our new #1 here as well. With ~460K IOPS and a price under $40K they were able to reach $0.08 $/IOPS. Yet again there’s a slight advantage to the pricing here for the software defined storage system but they did include the Lenovo hardware, storage, and various software licenses, so it’s hard to fault them. But they had no need for HBA/NICs or FC line cards that other systems  would need to do standard storage benchmarks. But even at $8K for a pair of them, it wouldn’t make much of a dent in their placement here, they might reach all the way up to $0.10/IOPS. 

SPC-2 results

In Figure 4 we show our SPC-2 MBPS spider or radar chart, which depicts the achievements for the three workloads which make up the SPC-2 aggregate MBPS.

SCISPC160224-004Figure 4 Top 10 SPC-2 MBPS results, spider chart

The new Huawei OceanStore 6800 V3 came in 4th place with an impressive MBPS 42.8GB/sec. They would have done even better with a little improvement in their VoD workload performance. It’s also impressive that they did this with 800 disk drives rather than flash that the #1&#2 used. There appears to be a continuing market for all-disk systems at least for bandwidth intensive workloads..

Significance

It’s always good to see new SPC-1 and -2 submissions. We had a good mix of flash and disk only systems this quarter and one of the disk-only systems made it into the top ten (for throughput).  DataCore’s software defined storage represents a new class of systems at least for SPC-1 submissions and it came in with impressive economics as well as response time, any response time sub 0.1msec seems almost magical in a storage system.

We are still playing with how to measure flash effectiveness. This quarter we used IOPS/GB-NAND last quarter IOPS/SSD. It’s unclear which is the superior metric but hopefully someday we will reach clarity on this.

As always, suggestions on how to improve any of our performance analyses are welcomed.

[Also we offer more block storage performance information plus our OLTP, Email and Throughput ChampionsCharts™ in our recently updated (May 2017) SAN Storage Buying Guide, or for more information on some select performance results please see our recently updated (April 2017) SAN-NAS Storage Buying Guide, both of which are available for purchase on our website.]

[This performance dispatch was originally sent out to our newsletter subscribers in February of 2016.  If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) and we will send our current issue along with download instructions for this and other reports. Dispatches are posted to our website at least a quarter or more after they are sent to our subscribers, so if you are interested in current results please consider signing up for our newsletter.]  

Silverton Consulting, Inc., is a U.S.-based Storage, Strategy & Systems consulting firm offering products and services to the data storage community

[1] All SPC results available from http://www.storageperformance.org/home/ as of 26Feb2016

[2] Available at SAN Storage Buying Guide