CIFS vs. NFS the saga continues, recent SPECsfs2008 results- chart of the month

When last we discussed this topic, the tides had turned and the then current SPECsfs 2008 results had shown that any advantage that CIFS had over NFS was an illusion.

Well there has been more activity for both CIFS and NFS protocols since our last discussion and it showed, once again that CIFS was faster than NFS but rather than going down that same path again, I decided to try something different.

As a result, we published the above chart which places all NFS and CIFS disk only submissions in stark contrast.

This chart was originally an attempt to refute many analysts contention that storage benchmarks are more of a contest as to who has thrown more disks at the problem rather than some objective truth about the performance of one product or another.

But a curious thought occurred to me as I was looking at these charts for CIFS and NFS last month. What if I plotted both results on the same chart?  Wouldn’t such a chart provide some additional rationale to our discussion on CIFS vs. NFS.

Sure enough, it did.

From my perspective this chart proves that CIFS is faster than NFS.  But, maybe a couple of points might clarify my analysis:

1. I have tried to eliminate any use of SSDs or NAND caching from this chart as they just confound the issue.  Also, all disk-based, NFS and CIFS benchmarks are represented on the above charts, not just those that have submitted both CIFS and NFS results on the same hardware.
2. There is an industry wide view that CIFS and NFS are impossible to compare because one is state-full (CIFS) and the other state-less (NFS).  I happen to think this is wrong.  Most users just want to know which is faster and/or better.  It would be easier to do analyze this if SPECsfs2008 reported data transfer rates rather than operations/second rates but they don’t.
3. As such, one potential problem with comparing the two on the above chart is that the percentage of “real” data transfers represented by “operations per second” may be different.  Ok, this would need to be normalized if they were a large difference between CIFS and NFS.  But when examining the SPECsfs2008 user’s guide spec., one sees that NFS read and write data ops is 28.0% of all operations and CIFS read and write data ops is 29.1% of all operations.  As they aren’t that different, the above chart should correlate well to the number of data operations done by each separate protocol. If anything, normalization would show an even larger advantage for CIFS, not less.
4. Another potential concern one needs to consider is the difference in the average data transfer size between the protocols.  The user guide doesn’t discriminate between access transfer rates for NFS or CIFS, so we assume it’s the same for the two protocols. Given that assumption, then the above chart provides a reasonable correlation to the protocols relative data transfer rates.
5. The one real concern on this chart is the limited amount of CIFS disk benchmarks.  At this time there are about 20 CIFS disk benchmarks vs. 40 NFS disk benchmarks. So the data is pretty slim for CIFS, nonetheless, 20 is almost enough to make this statistically significant.  So with more data the advantage may change slightly but I don’t think it will ever shift back to NFS.

Ok, now that I have all the provisos dealt with, what’s the chart really telling me.

One has to look at the linear regression equations to understand this but, CIFS does ~463.0 operations/second per disk drive and NFS does ~296.5 operations/second per disk drive.  What this says is, for all things being equal, i.e., the same hardware and disk drive count, CIFS does about 1.6X (463.0/296.5) more operations per second than NFS and correspondingly, CIFS provides ~1.6X more data per second than NFS does.

Comments?

—–

The full SPECsfs 2008 report went out to our newsletter subscribers last month. The above chart has been modified somewhat from a plot in our published report, but the data is the same (forced the linear equations to have an intercept of 0 to eliminate the constant, displayed the R**2 for CIFS, and fixed the vertical axis title).

A copy of the full SPECsfs report will be up on the dispatches page of our website later next month. However, you can get this information now and subscribe to future newsletters to receive these reports even earlier by just emailing us at SubscribeNews@SilvertonConsulting.com?Subject=Subscribe_to_Newsletter or using the signup form above and to the right.

As always, we welcome any suggestions on how to improve our analysis of SPECsfs or any of our other storage system performance discussions.