CIFS vs. NFS – Silverton Consulting

New SPECsfs2008 CIFS/SMB vs. NFS (again) – chart of the month

SPECsfs2008 benchmark results for CIFS/SMB vs. NFS protocol performance — SCISFS140326-001 (c) 2014 Silverton Consulting, All Rights Reserved

The above chart represents another in a long line of charts on the relative performance of CIFS[/SMB] versus NFS file interface protocols. The information on the chart are taken from vendor submissions that used the same exact hardware configurations for both NFS and CIFS/SMB protocol SPECsfs2008 benchmark submissions.

There are generally two charts I show in our CIFS/SMB vs. NFS analysis, the one above and another that shows a ops/sec per spindle count analysis for all NFS and CIFS/SMB submissions. Both have historically indicated that CIFS/SMB had an advantage. The one above shows the total number of NFS or CIFS/SMB operations per second on the two separate axes and provides a linear regression across the data. The above shows that, on average, the CIFS/SMB protocol provides about 40% more (~36.9%) operations per second than NFS protocol does with the same hardware configuration.

However, there are a few caveats about this and my other CIFS/SMB vs. NFS comparison charts:

The SPECsfs2008 organization has informed me (and posted on their website) that CIFS[/SMB] and NFS are not comparable. CIFS/SMB is a stateful protocol and NFS is stateless and the corresponding commands act accordingly. My response to them and my readers is that they both provide file access, to a comparable set of file data (we assume, see my previous post on What’s wrong with SPECsfs2008) and in many cases today, can provide access to the exact same file, using both protocols on the same storage system.
The SPECsfs2008 CIFS/SMB benchmark does slightly more read and slightly less write data operations than their corresponding NFS workloads. Specifically, their CIFS/SMB workload does 20.5% and 8.6% READ_ANDX and WRITE_ANDX respectively CIFS commands vs. 18% and 9% READ and WRITE respectively NFS commands.
There are fewer CIFS/SMB benchmark submissions than NFS and even fewer with the same exact hardware (only 13). So the statistics comparing the two in this way must be considered preliminary, even though the above linear regression is very good (R**2 at ~0.98).
Many of the submissions on the above chart are for smaller systems. In fact 5 of the 13 submissions were for storage systems that delivered less than 20K NFS ops/sec which may be skewing the results and most of which can be seen above bunched up around the origin of the graph.

And all of this would all be wonderfully consistent if not for a recent benchmark submission by NetApp on their FAS8020 storage subsystem. For once NetApp submitted the exact same hardware for both a NFS and a CIFS/SMB submission and lo and behold they performed better on NFS (110.3K NFS ops/sec) than they did on CIFS/SMB (105.1K CIFS ops/sec) or just under ~5% better on NFS.

Luckily for the chart above this was a rare event and most others that submitted both did better on CIFS/SMB. But I have been proven wrong before and will no doubt be proven wrong again. So I plan to update this chart whenever we get more submissions for both CIFS/SMB and NFS with the exact same hardware so we can see a truer picture over time.

For those with an eagle eye, you can see NetApp’s FAS8020 submission as the one below the line in the first box above the origin which indicates they did better on NFS than CIFS/SMB.

Comments?

~~~~

The complete SPECsfs2008 performance report went out in SCI’s March 2014 newsletter. But a copy of the report will be posted on our dispatches page sometime next quarter (if all goes well). However, you can get the latest storage performance analysis now and subscribe to future free newsletters by just using the signup form above right.

Even more performance information on NFS and CIFS/SMB protocols, including our ChampionCharts™ for file storage can be found in SCI’s recently (March 2014) updated NAS Buying Guide, on sale from our website.

As always, we welcome any suggestions or comments on how to improve our SPECsfs2008 performance reports or any of our other storage performance analyses.

Latest SPECsfs2008 results NFS vs. CIFS – chart-of-the-month

Posted on March 1, 2013August 19, 2016 by Ray in CIFS/SMB throughput, Ethernet, File Storage, Networking, NFS throughput, SPECsfs, Storage, Storage performance

SCISFS121227-010(001) (c) 2013 Silverton Consulting, Inc. All Rights Reserved

We return to our perennial quest to understand file storage system performance and our views on NFS vs. CIFS performance. As you may recall, SPECsfs2008 believes that there is no way to compare the two protocols because

CIFS/SMB is “statefull” and NFS is “state-less”
The two protocols are issuing different requests.

Nonetheless, I feel it’s important to go beyond these concerns and see if there is any way to assess the relative performance of the two protocols. But first a couple of caveats on the above chart:

There are 25 CIFS/SMB submissions and most of them are for SMB environments vs. 64 NFS submissions which are all over the map
There are about 12 systems that have submitted exact same configurations for CIFS?SMB and NFS SPECsfs2008 benchmarks.
This chart does not include any SSD or FlashCache systems, just disk drive only file storage.

All that being said, let us now see what the plot has to tell us. First the regression line is computed by Excel and is a linear regression. The regression coefficient for CIFS/SMB is much better at 0.98 vs NFS 0.80. But this just means that their is a better correlation between CIFS/SMB throughput operations per second to the number of disk drives in the benchmark submission than seen in NFS.

Second, the equation and slope for the two lines is a clear indicator that CIFS/SMB provides more throughput operations per second per disk than NFS. What this tells me is that given the same hardware, all things being equal the CIFS/SMB protocol should perform better than NFS protocol for file storage access.

Just for the record the CIFS/SMB version used by SPECsfs2008 is currently SMB2 and the NFS version is NFSv3. SMB3 was just released last year by Microsoft and there aren’t that many vendors (other than Windows Server 2012) that support it in the field yet and SPECsfs2008 has yet to adopt it as well. NFSv4 has been out now since 2000 but SPECsfs2008 and most vendors never adopted it. NFSv4.1 came out in 2010 and still has little new adoption.

So these results are based on older, but current versions of both protocols available in the market today.

So, given all that, if I had an option I would run CIFS/SMB protocol for my file storage.

Comments?

More information on SPECsfs2008 performance results as well as our NFS and CIFS/SMB ChampionsCharts™ for file storage systems can be found in our NAS Buying Guide available for purchase on our web site.

~~~~

The complete SPECsfs2008 performance report went out in SCI’s December newsletter. But a copy of the report will be posted on our dispatches page sometime this month (if all goes well). However, you can get the latest storage performance analysis now and subscribe to future free newsletters by just using the signup form above right.

As always, we welcome any suggestions or comments on how to improve our SPECsfs2008 performance reports or any of our other storage performance analyses.

Latest SPECsfs2008 benchmarks analysis, CIFS vs NFS corrected – chart of the month

Posted on November 4, 2011October 8, 2012 by Ray in CIFS/SMB throughput, Ethernet, File Storage, NFS throughput, SPECsfs, SPECsfs2008, Storage performance, System effectiveness

(SCISFS110929-001) (c) 2011 Silverton Consulting, All Rights Reserved

We made a mistake in our last post discussing CIFS vs. NFS results using SPECsfs2008 benchmarks by including some storage systems that had SSDs in this analysis. All of our other per spindle/disk drive analyses exclude SSDs and NAND cache because they skew per drive results so much. We have corrected this in the above chart which includes all the SPECsfs2008 results, up to the end of last month.

However, even with the corrections the results stand pretty much the way they were. CIFS is doing more throughput per disk drive spindle than NFS for all benchmark results not using SSDs or Flash Cache.

Dropping SSD results changed the linear regression equation. Specificall, the R**2 for CIFS and NFS dropped from 0.99 to 0.98 and from 0.92 to 0.82 and the B coefficient dropped from 463 to 405 and from 296 to 258 respectively.

I would be remiss if I didn’t discuss a few caveats with this analysis.

Now there are even less results in both CIFS and NFS groups, down to 15 for CIFS and 38 for NFS. For any sort of correlation comparison, more results would have better statistical significance.
In the NFS data, we include some NAS systems which have lots of DRAM cache (almost ~0.5TB). We should probably exclude these as well, which might drop the NFS line down some more (at least lower the B value).
There are not a lot of enterprise level CIFS systems in current SPECsfs resuslts, with or without SSD or NAND caching. Most CIFS benchmarks are from midrange or lower filers. Unclear why these would do much better on a per spindle basis than a wider sample of NFS systems, but they obviously do.

All that aside, it seems crystal clear here, that CIFS provides more throughput per spindle.

In contrast, we have shown in the past posts how for the limited number of systems that submitted benchmarks with both CIFS and NFS typically show roughly equivalent throughput results for CIFS and NFS. (See my other previous post on this aspect of the CIFS vs. NFS discussion).

Also, in our last post we discussed some of the criticism leveled against this analysis and provided our view to refute these issues. Mostly their concerns are due to the major differences between CIFS state-full protocol and NFS stateless protocol.

But from my perspective it’s all about the data. How quickly can I read a file, how fast can I create a file. Given similar storage systems, with similar SW, cache and hard disk drives, it’s now clear to me that CIFS provides faster access to data than NFS does, at least on a per spindle basis.

Nevertheless, more data may invalidate these results, so stay tuned.

—-

Why this is should probably be subject for another post but it may have a lot to do with the fact that it is stateless….

Comments?

CIFS vs. NFS the saga continues, recent SPECsfs2008 results- chart of the month

Posted on July 22, 2011October 8, 2012 by Ray in CIFS/SMB throughput, File Storage, NFS throughput, SPECsfs, SPECsfs2008, Storage, Storage performance

SCISFS110628-004A (c) 2011 Silverton Consulting, Inc., All Rights Reserved

When last we discussed this topic, the tides had turned and the then current SPECsfs 2008 results had shown that any advantage that CIFS had over NFS was an illusion.

Well there has been more activity for both CIFS and NFS protocols since our last discussion and it showed, once again that CIFS was faster than NFS but rather than going down that same path again, I decided to try something different.

As a result, we published the above chart which places all NFS and CIFS disk only submissions in stark contrast.

This chart was originally an attempt to refute many analysts contention that storage benchmarks are more of a contest as to who has thrown more disks at the problem rather than some objective truth about the performance of one product or another.

But a curious thought occurred to me as I was looking at these charts for CIFS and NFS last month. What if I plotted both results on the same chart? Wouldn’t such a chart provide some additional rationale to our discussion on CIFS vs. NFS.

Sure enough, it did.

From my perspective this chart proves that CIFS is faster than NFS. But, maybe a couple of points might clarify my analysis:

I have tried to eliminate any use of SSDs or NAND caching from this chart as they just confound the issue. Also, all disk-based, NFS and CIFS benchmarks are represented on the above charts, not just those that have submitted both CIFS and NFS results on the same hardware.
There is an industry wide view that CIFS and NFS are impossible to compare because one is state-full (CIFS) and the other state-less (NFS). I happen to think this is wrong. Most users just want to know which is faster and/or better. It would be easier to do analyze this if SPECsfs2008 reported data transfer rates rather than operations/second rates but they don’t.
As such, one potential problem with comparing the two on the above chart is that the percentage of “real” data transfers represented by “operations per second” may be different. Ok, this would need to be normalized if they were a large difference between CIFS and NFS. But when examining the SPECsfs2008 user’s guide spec., one sees that NFS read and write data ops is 28.0% of all operations and CIFS read and write data ops is 29.1% of all operations. As they aren’t that different, the above chart should correlate well to the number of data operations done by each separate protocol. If anything, normalization would show an even larger advantage for CIFS, not less.
Another potential concern one needs to consider is the difference in the average data transfer size between the protocols. The user guide doesn’t discriminate between access transfer rates for NFS or CIFS, so we assume it’s the same for the two protocols. Given that assumption, then the above chart provides a reasonable correlation to the protocols relative data transfer rates.
The one real concern on this chart is the limited amount of CIFS disk benchmarks. At this time there are about 20 CIFS disk benchmarks vs. 40 NFS disk benchmarks. So the data is pretty slim for CIFS, nonetheless, 20 is almost enough to make this statistically significant. So with more data the advantage may change slightly but I don’t think it will ever shift back to NFS.

Ok, now that I have all the provisos dealt with, what’s the chart really telling me.

One has to look at the linear regression equations to understand this but, CIFS does ~463.0 operations/second per disk drive and NFS does ~296.5 operations/second per disk drive. What this says is, for all things being equal, i.e., the same hardware and disk drive count, CIFS does about 1.6X (463.0/296.5) more operations per second than NFS and correspondingly, CIFS provides ~1.6X more data per second than NFS does.

Comments?

—–

The full SPECsfs 2008 report went out to our newsletter subscribers last month. The above chart has been modified somewhat from a plot in our published report, but the data is the same (forced the linear equations to have an intercept of 0 to eliminate the constant, displayed the R**2 for CIFS, and fixed the vertical axis title).

A copy of the full SPECsfs report will be up on the dispatches page of our website later next month. However, you can get this information now and subscribe to future newsletters to receive these reports even earlier by just emailing us at SubscribeNews@SilvertonConsulting.com?Subject=Subscribe_to_Newsletter or using the signup form above and to the right.

As always, we welcome any suggestions on how to improve our analysis of SPECsfs or any of our other storage system performance discussions.