SCI 2013Oct29 Latest SPC-1 & SPC-2 performance results

In 3PAR, Data storage, DS8000, DS8800, DS8870, E-Series, HPE, IBM, IOPS, Kamanario K2, Kaminario, LDQ, LFP, M700, MBPS, MPBS/drive, NEC, Oracle, Oracle ZFS ZS3-4, P10000 3PAR, SPC, SPC-1, SPC-2, Storwize V7000, SVC, VOD by AdministratorLeave a Comment

We return now to Storage Performance Council (SPC) results*.  There have been two new SPC-2 submissions, the Oracle ZFS ZS3-4 and NEC Storage M700, and one new SPC-1submission, the Kaminario K2 storage system. We will start this report with SPC-2 results as we haven’t done much here in a while and will talk later about the new SPC-1 result.

SPC-2*results

We begin our discussion with the Top 10 mega-bytes per second (MBPS™) chart for SPC-2.

SPC-2 Top 10 MB/second throughput  storage systems, bar chartFigure 1 SPC-2* Top 10 MPBS

Higher is better on this Top 10 chart. In contrast to SPC-1 IOPS results all of these systems are disk-only storage systems. In figure 1, the new Oracle ZS3-4 solution came in with the top aggregate score of 17.2 MBPS and the new NEC M700 system came in fifth with an aggregate score of 14.4 MBPS. Both the new Oracle and NEC systems have a unit tested price under $400K with a storage capacity of ~50TB or under.

 

MPBS is a composite score computed as an average of large file processing (LFP), large database query (LDQ) processing and video on demand (VOD) throughput. It’s interesting to note the top composite MPBS storage system (see Figure 1) but I find it more fascinating to understand their respective performance on the three, independent

workloads. As such, in Figure 2 we show a spider chart of the workload components for the top 10 MBPS storage systems.

Figure 2 SPC-2 Top 10 MPBS by workloadSPC-2 spider chart of LDQ, LFP and VOD MB/second throughput workloads

One thing of interest in Figure 2 is the variability in the way each system performs under the three workloads. In the case of Oracle ZS3-4, one can see excellent (~second best) LDQ performance and the best VOD performance but relatively poor (~fifth) LFP performance. For NEC M700 storage, it did fifth in LDQ performance, second in VOD performance and fifth in LFP. Why ZS3 did so well (23% better than thenext best) is a mystery worthy of investigation. All the workloads here are throughput intensive and many (6 of the 10) of the storage systems used the same physical disk drives (15KRPM/300GB SAS). Most had more drives than Oracle or NEC’s storage systems. Without more information I can only attribute both Oracle’s ZS3-4 and NEC’s M700 superior VOD performance to better sequential detection and/or better sequential data layout across their disk drives.

Speaking of drives we next turn to Top 10 MBPS/drive in Figure 3.

SCISPC131029-003 © 2013 Silverton Consulting, All Rights ReservedFigure 3 Top 10 SPC-2 MBPS/drive

Figure 3 shows a view of drive effectiveness or efficiency using MBPS/drive spindle. The only new system here was the Oracle ZS3-4 coming in fifth place. Except for the ZS3-4 system, all the rest of the storage systems on this chart had 128 or less disk drives. The Oracle ZS3-4 system used 384 drives. Again, without additional information, I would have to attribute their good showing here to their overall top MBPS performance. The top 3 systems here are all OEM versions of NetApp’s E-systems storage that seem to do better than most in drive effectiveness.

SPC-1*results

Next, lets turn to SPC-1 performance and in Figure 4 we see the top 10 IOPS™ storage subsystems.

Figure 4 Top 10 SPC-1 IOPSTop 10 SPC-1 IO operations per second (IOPS)

The new Kaminario K2 system beat their prior entry results with ~1.6% more IO/second. It’s notable that this system used DRAM plus SSDs whereas the prior system used DRAM only –storage technology.  All the systems here except for the IBM SVC and IBM DS8870 are hybrid or all SSD systems, which continue to dominate SPC-1 workload performance.

Unclear why Kaminario does so well in IOPS, it’s not that they use lots of DRAM (only 40GB in latest entry) and don’t have the fastest SSDs (800GB SSDs, assumed to be MLC or eMLC). One paradox is that their latest system didn’t do that well in LRT™ (not shown here but the K2 system came in 14th). However, a clue may be that they have more controllers, one per SSD, which may help to explain their superior IOPS performance. Another consideration may be their lack of data services. The top 3 systems in IOPS probably don’t support the data services of HDS VSP, IBM SVC, IBM DS8870 or HP P10000 3PAR. Nonetheless, over a million IOPS is nothing to make light of.

Significance

It’s great to see new SPC-2 results that impact our top 10 rankings. I still don’t understand why more SSD or hybrid disk-SSD subsystems aren’t submitting SPC-2 performance. The only one so far has been the TMS (now IBM) RamSan-630, which did well for its time but now ranks thirteenth in MPBS. All this seems continuing proof that there is an ongoing need for disk-only storage in IT, at least for throughput intensive applications.

Flash-only, DRAM-only and flash-disk hybrids continue to dominate the non-drive oriented SPC-1 rankings. We saw this again with the top 10 IOPS chart above. It’s noteworthy that there are still a couple of disk-only storage systems ranking well in some of the non-drive, top 10 SPC-1 charts. But the trend seems clear and it’s only a matter of time before we see the last of the disk-only storage in our non-drive oriented, SPC-1 top 10.

Given all the above it seems clear that the use of disk-only storage is undergoing a transition that will relegate it to just high-throughput applications in the future while SSD and hybrid SSD-disk storage will displace disk-only storage for OLTP-like applications. We have seen something similar with tape when it transitioned from backup storage to more long-term, archive storage while deduplicated disk took over its backup role.

as always, suggestions on how to improve any of our performance analyses are welcomed.  Additionally, if you are interested in more block performance details, we now provide a fuller version (Top 30 results) of some of these charts and a set of new OLTP and Email ChampionsCharts™ for enterprise, mid-range and SMB SAN storage in our recently updated (February 2019), SAN Storage Buying Guide available from our website. Also, we show more of SPC and ESRP top 30 charts in our recently updated (December 2019) SAN-NAS Buying Guide if you are interested in more performance details.

~~~~

[This performance dispatch was originally sent out to our newsletter subscribers in October of 2013.  If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) and we will send our current issue along with download instructions for this and other reports. Dispatches are posted to our website at least a quarter or more after they are sent to our subscribers, so if you are interested in current results please consider signing up for our newsletter.]  

Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community.


* All results from www.storageperformance.org as of 28Oct2013

* All results from www.storageperformance.org as of 28Oct2013

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.