A recent post by Stephen Foskett has revisted a blog discussion that Chuck Hollis and I had on commodity vs. special purpose hardware. It’s clear to me that commodity hardware is a losing proposition for the storage industry and for storage users as a whole. Not sure why everybody else disagrees with me about this.
It’s all about delivering value to the end user. If one can deliver equivalent value with commodity hardware than possible with special purpose hardware then obviously commodity hardware wins – no question about it.
But, and it’s a big BUT, when some company invests in special purpose hardware, they have an opportunity to deliver better value to their customers. Yes it’s going to be more expensive on a per unit basis but that doesn’t mean it can’t deliver commensurate benefits to offset that cost disadvantage.
Look around, one sees special purpose hardware everywhere. For example, just checkout Apple’s iPad, iPhone, and iPod just to name a few. None of these would be possible without special, non-commodity hardware. Yes, if one disassembles these products, you may find some commodity chips, but I venture, the majority of the componentry is special purpose, one-off designs that aren’t readily purchase-able from any chip vendor. And the benefits it brings, aside from the coolness factor, is significant miniaturization with advanced functionality. The popularity of these products proves my point entirely – value sells and special purpose hardware adds significant value.
One may argue that the storage industry doesn’t need such radical miniaturization. I disagree of course, but even so, there are other more pressing concerns worthy of hardware specialization, such as reduced power and cooling, increased data density and higher IO performance, to name just a few. Can some of this be delivered with SBB and other mass-produced hardware designs, perhaps. But I believe that with judicious selection of special purposed hardware, the storage value delivered along these dimensions can be 10 times more than what can be done with commodity hardware.
Special purpose HW cost and development disadvantages denied
The other advantage to commodity hardware is the belief that it’s just easier to develop and deliver functionality in software than hardware. (I disagree, software functionality can be much harder to deliver than hardware functionality, maybe a subject for a different post). But hardware development is becoming more software like every day. Most hardware engineers do as much coding as any software engineer I know and then some.
Then there’s the cost of special purpose hardware but ASIC manufacturing is getting more commodity like every day. Several hardware design shops exist that sell off the shelf processor and other logic one can readily incorporate into an ASIC and Fabs can be found that will manufacture any ASIC design at a moderate price with reasonable volumes. And, if one doesn’t need the cost advantage of ASICs, use FPGAs and CPLDs to develop special purpose hardware with programmable logic. This will cut engineering and development lead-times considerably but will cost commensurably more than ASICs.
Do we ever stop innovating?
Probably the hardest argument to counteract is that over time, commodity hardware becomes more proficient at providing the same value as special purpose hardware. Although this may be true, products don’t have to stand still. One can continue to innovate and always increase the market delivered value for any product.
If there comes a time when further product innovation is not valued by the market than and only then, does commodity hardware win. However, chairs, cars, and buildings have all been around for many years, decades, even centuries now and innovation continues to deliver added value. I can’t see where the data storage business will be any different a century or two from now…
Hi!
I saw the link from Stephen Foskett on twitter (and indeed, I commented on his tweet the other day on this topic).
I'm not sure anyone is arguing whether or not a customized highly tuned ASIC can beat a commodity chip…I think it's pretty well established that it can.
The reason there are so many vendors using commodity hardware (at least, this is my suspicion) is because the gains aren't enough for most people to invest the R&D money and time into it, when commodity hardware can provide a very good solution nearly off the shelf.
So many people are just running rebranded Dell OEM boxes and using in-house developed software because (again, my suspicion) it's easier to write good software than it is to design blazing hardware.
I think this is a case where "good enough" trumps "absolute raw performance" for most people.
Matt,Thanks for your comment. I suppose at the low end there will always be a place for commodity hardware. But, the IT industry and the rest of the economy is driven by high end products, systems, and solutions. The battle will always be where that line (between “good-enough/use commodity hardware” and “not-good enough/needs special hardware” is drawn. Value trumps commodity but the line is constantly moving upward and everyone has to keep on their game to stay ahead.Ray
[disclosure: I work for a storage-software company with a rigid commodity HW approach]
The reason software wins over hardware is agility. Even if the time to develop a new feature was roughly the same for both hardware and software, releasing software takes minutes while releasing a new hardware version takes +6 months [given you operate on a "global-level"]. You can't compensate the HW agility impedance. Sorry.
Martin,Thanks for your comment. However, I find that validation and verification activities for a new “release” of software takes just about the same amount of time as any new hardware design. The fact that you can distribute software to the field in minutes while hardware updates require swapping out components is one factor in software's agility favor but the actual release process which includes requirements analysis, development and validiation activities doesn't seem to be any shorter.Ray
Ray, the strongest argument for commodity hardware is not performance but economics and Moore's law.
It is hard to keep up with a company like Intel. Why produce special purpose processors, I/O means and interconnects, if commodity HW's performance doubles roughly 2 years?
Ultimately, commodity HW will win because the customer want's it. Corporate world has learned that non-commodity HW ultimately implies premium costs, lock-in, etc.
Martin,Thanks again for your comments. I am not proposing that you compete with Intel on processor chips, by all means if all you need is a micro-processor choose Intel's multi-core designs but if you want to compete on providing value you can combine other processor designs with state-machines, interfaces and other assorted logic which can beat Intel in performing a service of value to your customers. It's not about which one has the fastest chip/most cores but which one can provide the most value/functionality to the customer.Ray
While indeed capturing a clear trend in the storage business, Stephen Foskett’s extrapolation is too general and thus has led him to erroneous conclusions.
Three aspects are missing from his analysis: The essence of Intel’s roadmap, the voice of the customer who deploys such a system, and a time line.
Intel, the world largest chip company, is copying a page from its game book: Identify a mature and growing market and then embed the key peripherals into their chipsets. Does anyone remember that motherboards used to include only compute and core logic elements? Intel has identified graphics and networks as ubiquitous in the previous decades, and indeed most NIC providers and Graphic chips providers vanished.
Yet, by integrating peripherals into its chipsets Intel is aiming at the sweet spot of good enough. Companies who continued to innovate and focused on the needs of the users not satisfied with mediocrity continued to excel. Look at NVIDIA with their focus on high-end graphics and GPUs. Others, who elected to compete with Intel on cost, perished. The lesson is that when a segment matures and becomes a substantial market segment, expect commoditization, and innovate since one size does not fit all.
So, what did Intel add to their roadmap? SAS connectivity and core RAID engine. Indeed challenging times for HBA vendors and RAID vendors. But, how does such a trend impact a vendor of scalable high performance file systems?
Stephen Foskett carries his arguments further. Not only are all storage systems predicted to run on the same Intel platform from one’s favorite OEM, but even software will become just “general purpose software.” According to the post, efforts put into software development are a waste since Linux will eventually do just as well. This statement is fundamentally flawed. Stephen fails to distinguish between the underlying OS and the value-added software running on top of it. BlueArc has embraced Linux. BlueArc does not pretend to improve on Linux OS fundamentals. BlueArc innovates and creates value from marrying its scalable file system to Linux.
According to Stephen’s brave new world, channels, brands and size are all there is. Buy from EMC, HDS, IBM, HP or Dell, and you get the same product. Well, I bless my luck for being the last innovator standing.
If all storage users were the same, if all access patterns were identical, if all data sets and demands were similar, commodity would have sufficed. Luckily it isn’t so. Storage is not “just” for access. BlueArc’s customers deploy storage to serve their applications. Rendering movies, performing genomic research, deduping primary data at speed or hosting a multiplicity of VM images on a storage system demand performance and scalability. Commodity is not good enough.
Customer’s demands and usage patterns are not addressed by Stephen, and adding customer’s demands to the equation makes commodity less ubiquitous.
Ray from Silverton Consulting has already addressed some user related aspects in his post, and I am cognizant of not repeating his well articulated arguments.
So, where does the current commoditization trend fit in the continuum? Commoditizing NICs and Graphics is the past; Commoditizing HBAs and RAID is current. What’s next? In any scenario possible, BlueArc wins. Innovating at the file system level and delivering the only enterprise scalable file system assures success. Since the file system remains the answer to customer requirements, and since new applications and compute grids will only create more demands, being a provider of enterprise scalable NAS is a winning proposition. Should file systems ever get on the radar screen of future consolidation I know of no other vendor better positioned to benefit from such a future trend.
The end of innovation is not near. I would have suggested changing the title of Stephen’s post to “Innovate or get commoditized.”
Shmuel Shottan, CTO
BlueArc
Shmuel,Thanks for your expanded comment. Innovation is the key (in both hardware and software) to move the value bar upwards or one gets commoditized. I couldn't agree more.Ray Ray
But doesn't all hardware become a commodity in the end? Your comment with the iPad is valid but only for so long. As most customers demand prices to come down, and some demand newer technology….Therefore new tech which is popular becomes the commodity of the future.
The aim of the architect should be use commodity hardware where possible, and ensure the extra grunt can come from horizontal scalability rather than vertically?
Dave,Thanks for the comment. Yes, all of this is transient and temporary but as commodity hardware moves up in functionality and capability, companies doing special purpose hardware have to also move up in capabilities and functionality. It's a never ending dynamic but it keeps us all on our toes.Ray