We now turn to analysis of the new SPECsfs®2008* benchmark results. Unfortunately there were not a lot of new SPECsfs2008 results, most notably NetApp released results with their Performance Acceleration Module (PAM) for both FC and SATA disks and one new CIFS result from Fujitsu Siemens Computers.
Latest SPECsfs2008 results
NetApp’s use of their PAM to speed up SATA disk displayed some interesting results, namely that SATA disks can perform similar to FCAL disks when used with PAM. Two items of note about the PAM results:
- The FCAL result had half the number of spindles as the non-PAM result (112 disks with PAM vs. 224 without PAM) but attained similar SPEFsfs2008 throughput.
- The SATA results also had the same number of disks as the FCAL-PAM benchmark or 112 disks. The only ignificant difference between the two PAM results was in the ORT results (see below).
Recall that NetApp’s PAM is an onboard cache extension using DRAM devices and comes in 16GB cards with special FlexScale software to control PAM use. These FAS3140 results incorporated 2-PAM DRAM modules adding 32GB on top of the 9GB of cache used in the base FAS3140 benchmark, for a total of 41GB of cache.
Also, as mentioned last time, the SGI product result used Infiniband, both ExaStore benchmarks used10GbE and the rest used gigabit Ethernet. In all fairness the networking connection may not be a limiting factor in SPECsfs2008 results.
Turning to ORT results, here one can clearly see the advantage of PAM for FC disks. The PAM-FCAL benchmark reported a 1.7 msec. ORT. Without PAM, the FAS3140 system with twice the disk drives, only delivered a 2.6 msec. ORT. It is also interesting that the PAM-SATA benchmark came in at an ORT of 2.8 msec., just an ~8% degradation from the FAS3140 with FCAL disks. Unclear what the cost difference would be between a FAS3140 with 224 FCAL disks vs. a FAS3140 with 32GB PAM and 112 SATA disks but they look about the same from a SPECsfs2008 performance perspective.
One rule of thumb often used in the mainframe space is that doubling cache size alone should reduce response time by 10%. NetApp’s benchmarked systems increased cache size by 4X (9GB to 41GB) leading us to expect a decrease in response time by ~20%. NetApp managed to exceed this with a reported ~35% decrease in ORT while at the same time halving the number of spindles.
Next we turn to CIFS results. So far only four results have been released but what reports exist are shown below. Recall the SGI is using Infiniband while the others all use GigE hardware interfaces.
Figure 3 SPECsfs2008* CIFS results
Our last SPECsfs2008 analysis showed some preliminary correlations showing that CIFS has about 2X the throughput of NFSv3 for similar systems. Some interesting discussions resulted from this claim but we still stand by what we said there. More combined CIFS-NFSv3 results for the same systems would certainly help clarify this claim.
Presently, SPECsfs2008 remains more of a curiosity from an end-user perspective given the limited number of vendor submissions. However, it clearly has some uses considering the recent NetApp results. Hopefully, more vendors will see the benefit in releasing results that should transform this curiosity into viable tool for end-users to better understand NAS performance.
This performance dispatch was originally sent out to our newsletter subscribers in March of 2009. If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) or subscribe by email and we will send our current issue along with download instructions for this and other reports. Also, if you need an even more in-depth analysis of NAS system features and performance please take the time to examine our NAS Briefing available for purchase from our website.
A PDF version of this can be found atSCI 2009 March 30 Update to the SPECsfs® 2008 performance results
Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community
*SPECsfs2008 results from http://www.spec.org/sfs2008/results/
 Available at http://www.silvertonconsulting.com/page2/page2d/page2d2/StorInt_Dispatch_2008.html