SCI 28May2014 SPC-1 performance results analysis

In 3PAR, DS8000, DS8800, DS8870, IOPS, LRT, SPC, SPC-1, SVC, V800 by AdministratorLeave a Comment

This Storage Intelligence (StorInt™) dispatch covers Storage Performance Council (SPC) results[1]. There have been two new SPC-1 and no new SPC-2 submissions since our last report this February. The new SPC-1 results are both from IBM, one was the IBM Flex System (Storwize) V7000 and the other was an IBM Storwize V3700 (Turbo) storage system. Alas, neither of these submissions resulted in any changes to our standard top ten charts. So we have chosen to update some of our non-top ten charts instead.

SPC-1 results

We begin our discussion with a scatter plot of IOPS™ vs. disk drives shown in Figure 1.

Scatter plot: Max IOPS vs. number of disk drives, with linear regression formula y=217.14x, R**2=0.89675Figure 1 Scatter plot: Maximum IOPS vs. disk drive count

First, although it’s hard to see, the two new IBM submissions had 120- and 240-15KRPM 300GB disk drives and resulted in 32K and 72K maximum IOPS, for the V3700 and the V7000 respectively. They can be seen, above the regression line, in the lowest section about ¼ and ½ of the way from 0 to 500 drives.

As can be seen in Figure 1 the trend is pretty solid now, with a regression coefficient of ~0.9. According to the linear regression an average storage subsystem generates ~217 maximum IOPS per disk drive.

There is much more variation as you move to higher drive counts, which can be seen at around 1500 and 2000 drives. The interesting item about these 6 points (2-1500 drives and 4-2000 drives) is that with one exception they are all IBM submissions, one IBM DS8870 (~1500 drives, ~450K IOPS), and four IBM SAN Volume Controller (SVC) runs with one being SVC version 4.3 (~1500 drives, ~275K IOPS), two runs using SVC version 5.1 (~2000 drives with ~315K IOPS, 4 nodes & ~380K IOPS, 6 nodes) and one using SVC version 6.2 (~2000 drives, ~520K IOPS, 8 nodes). The lone exception being the HP 3PAR system with ~2000 drives at ~450K IOPS. What this says to me is that except for HP 3PAR, IBM SVC owns the 1500 drives and up category in SPC-1 performance.

Next we turn to a bubble chart of LRT vs. IOPS with size as a function of system cost ($/GB) shown in Figure 2.

Bubble chart: IOPS vs. LRT with bubble size $/GB, for SPC-1 systems under $100/GBFigure 2 Bubble chart: Maximum IOPS vs. LRT with size as a function of $/GB

We have two versions of this chart one with bubble size as a function of overall system price and this one using $/GB. To us they show similar stories. That is, you can purchase vastly different performing system at the same latency level with significantly varying maximum IOPS or at the same maximum IOPS level with significantly different latencies. Take for instance ~450K IOPS level. There are only two systems that were able to perform to this level but their minimum latencies (LRT) are ~1.2msec or ~2.2msec. But now if you consider their relative costs in $/GB, maybe there is a good reason for the difference in latencies. We have already discussed these two systems under Figure 1 above. But just to be clear, these are the IBM DS8870 and the HP 3PAR, respectively. It turns out the HP 3PAR is vastly cheaper (~8X) on a $/GB basis.

There’s at least one caveat to consider when comparing $/GB. Larger capacity systems (in SPC-1 parlance, Total ASU Capacity) generally cost less, sometimes much less than smaller systems in $/GB. In the case above, the HP 3PAR system had a configured ASU capacity of 230TB whereas the IBM DS8870 had a configured ASU capacity of 56TB. The other interesting thing about these two is that the DS8870 used ~1500-146GB drives and the 3PAR used ~2000-300GB drives and achieved roughly the same IOPS.

So that’s why we mostly supply the bubble chart with size as a function of overall system cost as shown in Figure 3.

Bubble chart: IOPS vs. LRT, this time with bubble size system price ($K), for SPC-1 systems under $100/GBFigure 3 Bubble chart: IOPS vs. LRT with size as subsystem price

Figure 3 indicates that the discrepancy in price is much lower. Here one can see that the HP 3PAR system cost ~3/5ths as much as the IBM DS8870 system or ~$3M vs. ~$5M, respectively. Something in the above two figures probably also speaks volumes about the two disparate architectures, but I cannot make sense of it just yet.

Significance

It’s great to see new SPC-1 results even if they don’t impact our top 10 rankings. It’s also good to see new disk-only SPC-1 submissions. But alas, without more flash somewhere in their subsystem it seems they will be relegated to non-top ten status. We are starting to consider a disk only version of our top ten charts but feel that perhaps the time for this is past.

As always, suggestions on how to improve any of our performance analyses are welcomed. Additionally, if you are interested in more block performance details, we now provide a fuller version (Top 30 results) of some of these charts and a set of new OLTP and Email ChampionsCharts™ for enterprise, mid-range and SMB SAN storage, in our recently updated (November 2018), SAN Storage Buying Guide available from our website.

[This performance dispatch was originally sent out to our newsletter subscribers in May of 2014.  If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) and we will send our current issue along with download instructions for this and other reports. Dispatches are posted to our website at least a quarter or more after they are sent to our subscribers, so if you are interested in current results please consider signing up for our newsletter.]  

Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community.

[1] All SPC results available from http://www.storageperformance.org/home/ as of 28May2014

 

Leave a Reply