Tag: CNA

Prior to last week’s VMworld, I had never heard of IO virtualization products before – storage virtualization yes but never IO virtualization. Then at last week’s VMworld I met with two vendors of IO virtualization products Aprius and Virtensys.

IO virtualization shares the HBAs/CNAs/NICs that a server tower would normally have plugged into each server and creates a top-of-rack box that shares these IO cards. The top-of-rack IO is connected to each of the tower servers by extending each server’s PCI-express bus.

Each individual server believes it has a local HBA/CNA/NIC card and acts accordingly. The top-of-rack box handles the mapping of each server to a portion of the HBA/CNA/NIC cards being shared. This all seems to remind me of server virtualization, using software to share the server processor, memory and IO resources across multiple applications. But with one significant difference.

How IO virtualization works

Aprius depends on the new SRIOV (Single Root I/O virtualization [requires login]) standards. I am no PCI-express expert but what this seems to do is allow a HBA/CNA/NIC PCI-express card to be a shared resource among a number of virtual servers executing within a physical server. What Aprius has done is sort of a “P2V in reverse” and allows a number of physical servers to share the same PCI-express HBA/CNA/NIC card in the top-of-rack solution.

Virtensys says it’s solution does not depend on SRIOV standards to provide IO virtualization. As such, it’s not clear what’s different but the top-of-box solution could conceivably share the hardware via software magic.

From a FC and HBA perspective there seems to be a number of questions as to how all this works.

• Does the top-of-box solution need to be powered and booted up first?
• How is FC zoning and LUN masking supported in a shared environment?

Similar networking questions should arise especially when one considers iSCSI boot capabilities.

Economics of IO virtualization

But the real question is one of economics. My lab owner friends tell me that a CNA costs about $800/port these days. Now when you consider that one could have 4-8 servers sharing each of these ports with IO virtualization the economics become clearer. With a typical configuration of 6 servers

• For a non-IO virtualized solution, each server would have 2 CNA ports at a minimum so this would cost you $1600/server or $9600.
• For an IO virtualized solution, each server requires PCI-extenders, costing about $50/server or $300, at least one CNA (for the top-of-rack) costing $1600 and the cost of their top-of-rack box.

If the IO virtualization box cost less than $7.7K it would be economical. But, IO virtualization providers also claim another savings, i.e, less switch ports need to be purchased because there are less physical network links. Unclear to me what a 10Gbe port with FCOE support costs these days but my guess may be 2X what a CNA port costs or another $1600/port or for the 6 server dual ported configuration ~$19.2K. Thus, the top-of-rack solution could cost almost $27K and still be more economical. When using IO virtualization to reduce HBAs and NICs then the top-of-rack solution could be even more economical.

Although the economics may be in favor of IO virtualization – at the moment – time is running out. CNA, HBA and NIC ports are coming down in price as vendors ramp up production. These same factors will reduce switch port cost as well. Thus, the savings gained from sharing CNAs, HBAs and NICs across multiple servers will diminish over time. Also the move to FCOE will eliminate HBAs and NICs and replace them with just CNAs so there are even less ports to amortize.

Moreover, PCI-express extender cards will probably never achieve volumes similar to HBAs, NICs, or CNAs so extender card pricing should remain flat. In contrast, any top-of-rack solution will share in overall technology trends reducing server pricing so relative advantages of IO virtualization over top-of-rack switches should be a wash.

The critical question for the IO virtualization vendors is can they support a high enough fan-in (physical server to top-of-rack) to justify the additional costs in both capital and operational expense for their solution. And will they be able to keep ahead of the pricing trends of their competition (top-of-rack switch ports and server CNA ports).

On one side as CNAs, HBAs, and NICs become faster and more powerful, no single application can consume all the throughput being made available. But on the other hand, server virtualization are now running more applications on each physical server and as such, amortizing port hardware over more and more applications.

Does IO virtualization make sense today at HBAs@8GFC, NICs and CNAs@10Gbe, would it make sense in the future with converged networks? It all depends on port costs. As port costs go down eventually these products will be squeezed.

The significant difference between server and IO virtualization is the fact that IO virtualization doesn’t reduce hardware footprint – one top-of-box IO virtualization appliance replaces a top-of-box switch and server PCI-express slots used by CNAs/HBAs/NICs are now used by PCI-extender cards. In contrast, server virtualization reduced hardware footprint and costs from the start. The fact that IO virtualization doesn’t reduce hardware footprint may doom this product.