Tag: CIFS throughput per #disks

Latest SPECsfs2008 benchmarks analysis, CIFS vs NFS corrected – chart of the month

Posted on by Ray in CIFS/SMB throughput, Ethernet, File Storage, NFS throughput, SPECsfs, SPECsfs2008, Storage performance, System effectiveness
(SCISFS110929-001) (c) 2011 Silverton Consulting, All Rights Reserved
(SCISFS110929-001) (c) 2011 Silverton Consulting, All Rights Reserved

We made a mistake in our last post discussing CIFS vs. NFS results using SPECsfs2008 benchmarks by including some storage systems that had SSDs in this analysis. All of our other per spindle/disk drive analyses exclude SSDs and NAND cache because they skew per drive results so much.  We have corrected this in the above chart which includes all the SPECsfs2008 results, up to the end of last month.

However, even with the corrections the results stand pretty much the way they were. CIFS is doing more throughput per disk drive spindle than NFS for all benchmark results not using SSDs or Flash Cache.

Dropping SSD results changed the linear regression equation. Specificall,  the R**2 for CIFS and NFS dropped from 0.99 to 0.98 and from 0.92 to 0.82 and the B coefficient dropped from 463 to 405 and from 296 to 258 respectively.

I would be remiss if I didn’t discuss a few caveats with this analysis.

  • Now there are even less results in both CIFS and NFS groups, down to 15 for CIFS and 38 for NFS.   For any sort of correlation comparison, more results would have better statistical significance.
  • In the NFS data, we include some NAS systems which have lots of DRAM cache (almost ~0.5TB).  We should probably exclude these as well, which might drop the NFS line down some more (at least lower the B value).
  • There are not a lot of enterprise level CIFS systems in current SPECsfs resuslts, with or without SSD or NAND caching.  Most CIFS benchmarks are from midrange or lower filers.  Unclear why these would do much better on a per spindle basis than a wider sample of NFS systems, but they obviously do.

All that aside, it seems crystal clear here, that CIFS provides more throughput per spindle.

In contrast, we have shown in the past posts how for the limited number of systems that submitted benchmarks with both CIFS and NFS typically show roughly equivalent throughput results for CIFS and NFS. (See my other previous post on this aspect of the CIFS vs. NFS discussion).

Also, in our last post we discussed some of the criticism leveled against this analysis and provided our view to refute these issues. Mostly their concerns are due to the major differences between CIFS state-full protocol and NFS stateless protocol.

But from my perspective it’s all about the data.  How quickly can I read a file, how fast can I create a file.  Given similar storage systems, with similar SW, cache and hard disk drives, it’s now clear to me that CIFS provides faster access to data than NFS does, at least on a per spindle basis.

Nevertheless, more data may invalidate these results, so stay tuned.

—-

Why this is should probably be subject for another post but it may have a lot to do with the fact that it is stateless….

Comments?

CIFS vs. NFS the saga continues, recent SPECsfs2008 results- chart of the month

Posted on by Ray in CIFS/SMB throughput, File Storage, NFS throughput, SPECsfs, SPECsfs2008, Storage, Storage performance

SCISFS110628-004A (c) 2011 Silverton Consulting, Inc., All Rights Reserved
SCISFS110628-004A (c) 2011 Silverton Consulting, Inc., All Rights Reserved

When last we discussed this topic, the tides had turned and the then current SPECsfs 2008 results had shown that any advantage that CIFS had over NFS was an illusion.

Well there has been more activity for both CIFS and NFS protocols since our last discussion and it showed, once again that CIFS was faster than NFS but rather than going down that same path again, I decided to try something different.

As a result, we published the above chart which places all NFS and CIFS disk only submissions in stark contrast.

This chart was originally an attempt to refute many analysts contention that storage benchmarks are more of a contest as to who has thrown more disks at the problem rather than some objective truth about the performance of one product or another.

But a curious thought occurred to me as I was looking at these charts for CIFS and NFS last month. What if I plotted both results on the same chart?  Wouldn’t such a chart provide some additional rationale to our discussion on CIFS vs. NFS.

Sure enough, it did.

From my perspective this chart proves that CIFS is faster than NFS.  But, maybe a couple of points might clarify my analysis:

  1. I have tried to eliminate any use of SSDs or NAND caching from this chart as they just confound the issue.  Also, all disk-based, NFS and CIFS benchmarks are represented on the above charts, not just those that have submitted both CIFS and NFS results on the same hardware.
  2. There is an industry wide view that CIFS and NFS are impossible to compare because one is state-full (CIFS) and the other state-less (NFS).  I happen to think this is wrong.  Most users just want to know which is faster and/or better.  It would be easier to do analyze this if SPECsfs2008 reported data transfer rates rather than operations/second rates but they don’t.
  3. As such, one potential problem with comparing the two on the above chart is that the percentage of “real” data transfers represented by “operations per second” may be different.  Ok, this would need to be normalized if they were a large difference between CIFS and NFS.  But when examining the SPECsfs2008 user’s guide spec., one sees that NFS read and write data ops is 28.0% of all operations and CIFS read and write data ops is 29.1% of all operations.  As they aren’t that different, the above chart should correlate well to the number of data operations done by each separate protocol. If anything, normalization would show an even larger advantage for CIFS, not less.
  4. Another potential concern one needs to consider is the difference in the average data transfer size between the protocols.  The user guide doesn’t discriminate between access transfer rates for NFS or CIFS, so we assume it’s the same for the two protocols. Given that assumption, then the above chart provides a reasonable correlation to the protocols relative data transfer rates.
  5. The one real concern on this chart is the limited amount of CIFS disk benchmarks.  At this time there are about 20 CIFS disk benchmarks vs. 40 NFS disk benchmarks. So the data is pretty slim for CIFS, nonetheless, 20 is almost enough to make this statistically significant.  So with more data the advantage may change slightly but I don’t think it will ever shift back to NFS.

Ok, now that I have all the provisos dealt with, what’s the chart really telling me.

One has to look at the linear regression equations to understand this but, CIFS does ~463.0 operations/second per disk drive and NFS does ~296.5 operations/second per disk drive.  What this says is, for all things being equal, i.e., the same hardware and disk drive count, CIFS does about 1.6X (463.0/296.5) more operations per second than NFS and correspondingly, CIFS provides ~1.6X more data per second than NFS does.

Comments?

—–

The full SPECsfs 2008 report went out to our newsletter subscribers last month. The above chart has been modified somewhat from a plot in our published report, but the data is the same (forced the linear equations to have an intercept of 0 to eliminate the constant, displayed the R**2 for CIFS, and fixed the vertical axis title).

A copy of the full SPECsfs report will be up on the dispatches page of our website later next month. However, you can get this information now and subscribe to future newsletters to receive these reports even earlier by just emailing us at SubscribeNews@SilvertonConsulting.com?Subject=Subscribe_to_Newsletter or using the signup form above and to the right.

As always, we welcome any suggestions on how to improve our analysis of SPECsfs or any of our other storage system performance discussions.