FCoE – Silverton Consulting

Fall SNWUSA 2013

Posted on October 19, 2013 by Ray in Cloud services, Crowdsourcing, Data grid, File Storage, Object storage, Software Defined Network, Storage, Storage architecture, Strategic Inflection Points, Strategy, System effectiveness, Systems, Visionary leadershp

Here’s my thoughts on SNWUSA which occurred this past week in the Long Beach Convention Center.

First, it was a great location. I saw a number of users I haven’t seen at SNWUSA ever before, some of which I have known for years from other (non-storage) venues.

Second, the exhibit hall was scantly populated. There were no major storage vendors at the show at all. Gold sponsors included NEC, Riverbed, & Sepaton, representing the largest exhibiters presenn. Making up the next (Contributing) tier were Western Digital, Toshiba, Active Archive Alliance, and LTO consortium with a smattering of smaller companies. Finally, there were another 12 vendors with kiosks around the floor, with the largest there being Veeam Software.

I suspect VMWorld Europe happening the same time in Barcelona might have had something to do with the sparse exhibit floor but the trend has been present for the past few shows.

That being said there were still a few surprises in store, at least for me. Two of the most interesting ones were:

Coho Data who came out of stealth with a scale out, RAIN (Redundant array of independent nodes) based storage cluster, with distributed, mirrored customer data across nodes and software defined networking. They currently support NFS for VMware with a management UI reminiscent of IOS 7 sans touch support. The product comes as a series of nodes with SSDs, disk storage and SDN. The SDN allows Coho Data to relocate front-end (client) connections to where the customer data lies. The distributed, mirrored backend storage provides redundancy in the case of a node/disk failure, at which time the system understands what data is now at risk and rebuilds the now-mirorless data onto other nodes. It reminds me a lot of Bycast/Archivas like architectures, with SDN and NFS support. I suppose the reason they are supporting VMware VMDKs is that the files are fairly large and thus easier to supply.
Cloud Physics was not exhibiting but they sponsored a break. As such, they were there talking with analysts and the press about their product. Their product installs as a VMware VM service and propagates VMware management agents to ESX servers which then pipe information back to their app about how your VMware environment is running, how VMs are performing, how your network and storage are performing for the VMs running, etc. This data is then sent to the cloud, where it’s anonymized. In the cloud, customers can use apps (called Cards) to analyze this data in the cloud, which can help them understand problem areas, predict what configuration changes can do for them, show them how VMs are performing, etc. It essentially is logging all this information to the cloud and providing ways to analyze the data to optimize your VMware environment.

Coming in just behind these two was Jeda Networks with their Software Defined Storage Network (SDSN). They use commodity (OpenFlow compatible) 10GbE switches to support a software FCoE storage SAN. Jeda Networks say that over the past two years, most 10GbE switch hardware have started to support DCB in hardware and with that in place, plus OpenFlow compatibility, they can provide a SDSN on top of them just by emulating a control layer for FCoE switches. Of course one would still need FCoE storage and CNAs but with that in place one could use much cheaper switches to support FCoE.

CloudPhysics has a subscription based pricing model which offers three tiers:

Free where you get their Vapp, the management agents and a defined set of Free Card Apps for no cost;
Standard level where you get all the above plus a set of Card Apps which provide more VMware managability for $50/ESX server/Month; and
Enterprise level where you get all the above plus all the Card Apps presently available for $150/ESX server/Month.

Jeda networks and Coho Data are still developing their pricing and had none they were willing to disclose.

One of the CloudPhysics Card apps could predict how certain VMs would benefit from host based (PCIe or SSD) IO caching. They had a chart which showed working set inflection points for (I think) one VM running an OLTP application. I have asked for this chart to discuss further in a future post. But although CloudPhysics has the data to produce such a chart, the application shows three potential break points where say adding 500MB, 2000MB or 10000MB of SSD cache can speed up application performance by 10%, 30% or 50% (numbers here made up for example purposes and not off the chart they showed me).

A few other companies made announcements at the show. For example, Sepaton announced their new VirtuoSO, scale out hybrid reduplication appliance.

That’s about it. I would have to say that SNW needs to rethink their business model, frequency of stows or what they are trying to do at their conferences. However, on the plust side, most of the users I talked with came away with a lot of information and thought the show was worthwhile and I came away with a few surprises.

~~~~

Comments?

Is FC dead?!

Posted on August 9, 2011August 25, 2011 by Ray in Block Storage, Ethernet, FC, FCoE, Storage performance, Strategic Inflection Points, System effectiveness, System quality

SNIA Tech Center Computer Lab 2 switching hw (c) 2011 Silverton Consulting, Inc.

Was at the Pacific Crest/Mosaic annual conference cocktail hour last night surrounded by a bunch of iSCSI/NAS storage vendors and they made the statement that FC is dead.

Apparently, 40GbE is just around the corner and 10GbE cards have started a steep drop in price and are beginning to proliferate through the enterprise. The vendors present felt that an affordable 40GbE that does iSCSI and/or FCoE would be the death knell for FC as we know it.

As evidence they point to Brocade’s recent quarterly results that shows their storage business is in decline, down 5-6% YoY for the quarter. In contrast, Brocade’s Ethernet business is up this quarter 12-13% YoY (albeit, from a low starting point). Further confusing the picture, Brocade is starting to roll out 16Gbps FC (16GFC) while the storage market is still trying to ingest the changeover to 8Gbps FC.

But do we need the bandwidth?

One question is do we need 16GFC or even 40GbE for the enterprise today. Most vendors speak of the high bandwidth requirements for server virtualization as a significant consumer of enterprise bandwidth. But it’s unclear to me whether this is reality or just the next wave of technology needing to find a home.

Let’s consider for the moment what 16GFC and 40GbE can do for data transfer. If we assume ~10 bits pre byte then:

16GFC can provide 1.6GB/s of data transfer,
40GbE can provide 4GB/s of data transfer.

Using Storage Performance Council’s SPC-2 results the top data transfer subsystem (IBM DS8K) is rated at 9.7GB/s so with 40GbE it could use about 3 links and for the 16GFC it would be able to sustain this bandwidth using about 7 links.

So there’s at least one storage systems out there that can utilize the extreme bandwidth that such interfaces supply.

Now as for the server side nailing down the true need is a bit harder to do. Using Amdahl’s IO law, which states there is 1 IO for every 50K instructions, and with Intel’s Core I7 Extreme edition rated at 159KMips, it should be generating about 3.2M IO/s and at 4KB per IO this would be about 12GB/sec. So the current crop of high processors seem able to consume this level of bandwidth, if present.

FC or Ethernet?

Now the critical question, which interface does the data center use to provide that bandwidth. The advantages of FC are becoming less so over time as FCoE becomes more widely adopted and any speed advantage that FC had should go away with the introduction of data center 40GbE.

The other benefit that Ethernet offers is a “single data center backbone” which can handle all network/storage traffic. Many large customers are almost salivating at the possibility of getting by with a single infrastructure for everything vs. having to purchase and support separate cabling, switches and server cards to use FC.

On the other hand, having separate networks, segregated switching, isolation between network and storage traffic can provide better security, availability, and reliability that are hard to duplicate with a single network.

To summarize, one would have to say is that there are some substantive soft benefits to having both Ethernet and FC infrastructure but there are hard cost and operational advantages to having a single infrastructure based on 10GbE or hopefully, someday 40GbE.

—-

So I would have to conclude that FC’s days are numbered especially when 40GbE becomes affordable and thereby, widely adopted in the data center.

Comments?

Why Open-FCoE is important

Posted on January 28, 2011 by Ray in Block Storage, Ethernet, FC, FCoE, Networking, Server virtualization, Storage, storage economics, Storage performance, Strategic Inflection Points

FCoE Frame Format (from Wikipedia, http://en.wikipedia.org/wiki/File:Ff.jpg)

I don’t know much about O/S drivers but I do know lots about storage interfaces. One thing that’s apparent from yesterday’s announcement from Intel is that Fibre Channel over Ethernet (FCoE) has taken another big leap forward.

Chad Sakac’s chart of FC vs. Ethernet target unit shipments (meaning, storage interface types, I think) clearly indicate a transition to ethernet is taking place in the storage industry today. Of course Ethernet targets can be used for NFS, CIFS, Object storage, iSCSI and FCoE so this doesn’t necessarily mean that FCoE is winning the game, just yet.

WikiBon did a great post on FCoE market dynamics as well.

The advantage of FC, and iSCSI for that matter, is that every server, every OS, and just about every storage vendor in the world supports them. Also there are plethera of economical, fabric switches available from multiple vendors that can support multi-port switching with high bandwidth. And there many support matrixes, identifying server-HBAs, O/S drivers for those HBA’s and compatible storage products to insure compatibility. So there is no real problem (other than wading thru the support matrixes) to implementing either one of these storage protocols.

Enter Open-FCoE, the upstart

What’s missing from 10GBE FCoE is perhaps a really cheap solution, one that was universally available, using commodity parts and could be had for next to nothing. The new Open-FCoE drivers together with the Intels x520 10GBE NIC has the potential to answer that need.

But what is it? Essentially Intel’s Open-FCoE is an O/S driver for Windows and Linux and a 10GBE NIC hardware from Intel. It’s unclear whether Intel’s Open-FCoE driver is a derivative of the Open-FCoe.org’s Linux driver or not but either driver works to perform some of the FCoE specialized functions in software rather than hardware as done by CNA cards available from other vendors. Using server processing MIPS rather than ASIC processing capabilities should make FCoE adoption in the long run, even cheaper.

What about performance?

The proof of this will be in benchmark results but it’s quite possible to be a non-issue. Especially, if there is not a lot of extra processing involved in a FCoE transaction. For example, if Open-FCoE only takes let’s say 2-5% of server MIPS and bandwidth to perform the added FCoE frame processing then this might be in the noise for most standalone servers and would showup only minimally in storage benchmarks (which always use big, standalone servers).

Yes, but what about virtualization?

However real world, virtualized servers is another matter. I believe that virtualized servers generally demand more intensive I/O activity anyway and as one creates 5-10 VMs, ESX server, it’s almost guaranteed to have 5-10X the I/O happening. If each standalone VM requires 2-5% of a standalone processor to perform Open-FCoE processing, then it could easily represent 5-7 X 2-5% on a 10VM ESX server (assumes some optimization for virtualization, if virtualization degrades driver processing, it could be much worse), which would represent a serious burden.

Now these numbers are just guesses on my part but there is some price to pay for using host server MIPs for every FCoE frame and it does multiply for use with virtualized servers, that much I can guarantee you.

But the (storage) world is better now

Nonetheless, I must applaud Intel’s Open-FCoE thrust as it can open up a whole new potential market space that today’s CNAs maybe couldn’t touch. If it does that, it introduces low-end systems to the advantages of FCoE then as they grow moving their environments to real CNAs should be a relatively painless transition. And this is where the real advantage lies, getting smaller data centers on the right path early in life will make any subsequent adoption of hardware accelerated capabilities much easier.

But is it really open?

One problem I am having with the Intel announcement is the lack of other NIC vendors jumping in. In my mind, it can’t really be “open” until any 10GBE NIC can support it.

Which brings us back to Open-FCoE.org. I checked their website and could see no listing for a Windows driver and there was no NIC compatibility list. So, I am guessing their work has nothing to do with Intel’s driver, at least as presently defined – too bad

However, when Open-FCoE is really supported by any 10GB NIC, then the economies of scale can take off and it could really represent a low-end cost point for storage infrastructure.

Unclear to me what Intel has special in their x520 NIC to support Open-FCoE (maybe some TOE H/W with other special sauce) but anything special needs to be defined and standardized to allow broader adoption by other Vendors. Then and only then will Open-FCoE reach it’s full potential.

—-

So great for Intel, but it could be even better if a standardized definition of an “Open-FCoE NIC” were available, so other NIC manufacturers could readily adopt it.

Comments?

Top 10 storage technologies over the last decade

Posted on December 30, 2010April 10, 2012 by Ray in Block Storage, Cloud services, Data, Data compression, Disk storage, Ethernet, FC, File Storage, Networking, Storage, Storage Backup, Storage density, Storage Features, Storage performance, Strategic Inflection Points, System effectiveness, Systems, Tape storage

Aurora's Perception or I Schrive When I See Technology by Wonderlane (cc) (from Flickr)

Some of these technologies were in development prior to 2000, some were available in other domains but not in storage, and some were in a few subsystems but had yet to become popular as they are today. In no particular order here are my top 10 storage technologies for the decade:

NAND based SSDs – DRAM and other technology solid state drives (SSDs) were available last century but over the last decade NAND Flash based devices have dominated SSD technology and have altered the storage industry forever more. Today, it’s nigh impossible to find enterprise class storage that doesn’t support NAND SSDs.
GMR head– Giant Magneto Resistance disk heads have become common place over the last decade and have allowed disk drive manufacturers to double data density every 18-24 months. Now GMR heads are starting to transition over to tape storage and will enable that technology to increase data density dramatically
Data Deduplication – Deduplication technologies emerged over the last decade as a complement to higher density disk drives as a means to more efficiently backup data. Deduplication technology can be found in many different forms today, ranging from file and block storage systems, backup storage systems, to backup software only solutions.
Thin provisioning – No one would argue that thin provisioning emerged last century but it took the last decade to really find its place in the storage pantheon. One almost cannot find a data center class storage device that does not support thin provisioning today.
Scale-out storage – Last century if you wanted to get higher IOPS from a storage subsystem you could add cache or disk drives but at some point you hit a subsystem performance wall. With scale-out storage, one can now add more processing elements to a storage system cluster without having to replace the controller to obtain more IO processing power. The link reference talks about the use of commodity hardware to provide added performance but scale-out storage can also be done with non-commodity hardware (see Hitachi’s VSP vs. VMAX).
Storage virtualization – server virtualization has taken off as the dominant data center paradigm over the last decade but a counterpart to this in storage has also become more viable as well. Storage virtualization was originally used to migrate data from old subsystems to new storage but today can be used to manage and migrate data over PBs of physical storage dynamically optimizing data placement for cost and/or performance.
LTO tape – When IBM dominated IT in the mid to late last century, the tape format dejour always matched IBM’s tape technology. As the decade dawned, IBM was no longer the dominant player and tape technology was starting to diverge into a babble of differing formats. As a result, IBM, Quantum, and HP put their technology together and created a standard tape format, called LTO, which has become the new dominant tape format for the data center.
Cloud storage – Unclear just when over the last decade cloud storage emerged but it seemed to be a supplement to cloud computing that also appeared this past decade. Storage service providers had existed earlier but due to bandwidth limitations and storage costs didn’t survive the dotcom bubble. But over this past decade both bandwidth and storage costs have come down considerably and cloud storage has now become a viable technological solution to many data center issues.
iSCSI – SCSI has taken on many forms over the last couple of decades but iSCSI has the altered the dominant block storage paradigm from a single, pure FC based SAN to a plurality of technologies. Nowadays, SMB shops can have block storage without the cost and complexity of FC SANs over the LAN networking technology they already use.
FCoE – One could argue that this technology is still maturing today but once again SCSI has taken opened up another way to access storage. FCoE has the potential to offer all the robustness and performance of FC SANs over data center Ethernet hardware simplifying and unifying data center networking onto one technology.

No doubt others would differ on their top 10 storage technologies over the last decade but I strived to find technologies that significantly changed data storage that existed in 2000 vs. today. These 10 seemed to me to fit the bill better than most.

Comments?