Diablo Technologies

Commodity hardware loses again …

Posted on May 27, 2015 by Ray in PCIe, Storage performance, Strategic Inflection Points, System effectiveness

It seemed only a a couple of years back that everyone was touting how hardware engineering no longer mattered anymore.

What with Intel and others playing out Moore’s law, why invest in hardware engineering when the real smarts were all in software?

We said then that hardware engineered solutions still had a significant place to play but few believed me (see my posts: Commodity hardware always loses and Commodity hardware debate heats up ).

Well hardware’s back, …

A few examples;

EMC DSSD – at EMCWorld2015 a couple of weeks back, EMC demoed a new rack-scale flash storage system that was targeted at extremely high IOPS and very low latency. DSSD is a classic study on how proprietary hardware could enable new levels of performance. The solution connected to servers over a PCIe switched network, which didn’t really exist before and used hardware engineered, Flash Modules which were extremely dense, extremely fast and extremely reliable. (See my EMCWorld post on DSSD and our Greybeards on Storage (GBoS) podcast with Chad Sakac for more info on DSSD)
Diablo Memory Channel Storage (MCS) /SanDisk UltraDIMMs – Diablo’s MCS is coming out in SanDisk’s UltraDIMM NAND storage that plugs into DRAM slots and provides a memory paged access to NAND storage. The key is that the hardware logic provides overheads that are ~50 μsecs to access NAND storage. (We’ve wrote about MCS and UltraDIMMs here).
Hitachi VSP G1000 storage and their Hitachi Accelerated Flash (HAF) – recent SPC-1 results showed that a G1000 outfitted with HAF modules could generate over 2M IOPS and had very low latency (220 μsecs). (See our announcement summary on the Hitachi G1000 here).

Diablo ran into some legal problems but that’s all behind them now, so the way forward is clear of any extraneous hurdles.

There are other examples of proprietary hardware engineering from IBM FlashSystems, networking companies, PCIe flash vendors and others but these will suffice to make my point.

My point is if you want to gain orders of magnitude of better performance, you need to seriously consider engaging in some proprietary hardware engineering. Proprietary hardware may take longer than software-only solutions (although that’s somewhat of a function of the resources you throw at), but the performance gains are sometimes unobtainable any other way.

~~~~

Chad made an interesting point on our GBoS podcast, hardware innovation is somewhat cyclical. For a period of time, commodity hardware is much better than any storage solution really needs, so innovation swings to the software arena. But over time, software functionality comes up to speed and maxes out the hardware that’s available and then you need more hardware innovation to take performance to the next level. Then the cycle swings back to hardware engineering. And the cycle will swing back and forth again a lot more times before storage is ever through as an IT technology.

Today when it seems that there’s a new software defined storage solution coming out every month we seem very close to peak software innovation with little left for performance gains, but there’s still plenty left if we open our eyes to consider proprietary hardware.

Welcome to the start of the next hardware innovation cycle – take that commodity hardware.

Comments?

MCS, UltraDIMMs and memory IO, the new path ahead – part 2

Posted on June 24, 2014June 25, 2014 by Ray in Block Storage, DAS, data protection, SSD storage, Storage

In part 1 (see previous post here), we discussed the underlying technology for SanDisk‘s UltraDIMMs based on Diablo Technologies MCS hardware and software. IBM will be shipping UltraDIMMs in their high end servers later this year as their new eXFlash.

In this segment we will discuss what SanDisk has put on top of the Diablo Technology’s MCS to supply SSD storage.

SanDisk UltraDIMM SSD storage

In the UltraDIMM package, SanDisk supports 200 or 400GB of 19nm MLC NAND SSD storage that is accessed via SATA [corrected after this went out, Ed.] internally, but the main interface is the 1600MHz, DDR3 to the UltraDIMMs. As each UltraDIMM card plugs into any DDR3 memory slot you can potentially support multiples of these cards in a single server. I believe the maximum number is ~~7 UltraDIMMs, not sure if IBM supports this many~~ [corrected after this went out, Ed.] dependent on the number of memory slots in your server. IBM on their x3850 and x3950 can support up to 32 UltraDIMMs per server.

SanDisk uses their Guardian Technology to enhance NAND endurance beyond what’s possible with native NAND controllers. One of the things that Guardian Technology does is to vary the voltage used to program the NAND bits over the life of the bit cells/pages. So early on when the cell is fresh, they can use less voltage and as it ages they increase the voltage to insure that the bits are properly programmed. With other NAND controllers, using the same voltage across the whole NAND lifetime it will unduly stress the NAND bits early on and later as they age, it will be unable to program properly and will need to be flagged as bad. The NAND chips/bits are characterized so that SanDisk Guardian Technology can use an optimum voltage curve over the chips lifetime.

The UltraDIMMs also have powerloss protection. This means that any write to an UltraDIMM memory that’s been acknowledged to the server is guaranteed to have sufficient power to make it all the way to the SSD storage.

Another thing that MCS memory interface brings to the picture is Error Correction Circuitry (ECC). Data written to UltraDIMMs has ECC protection throughout the data path up from the server DRAM memory, through the DIMM socket, all the way to the SSD flash.

As discussed extensively in Part 1 of this post, access times for UltraDIMM storage is on the order 7µsec, which is ~7X faster than best of class PCIe Flash storage and a single UltraDIMM card is capable of sustaining 20GB/second of data throughput. I know of enterprise class storage systems that can’t do half that in throughput.

On the other hand, one problem with UltraDIMM storage is that they are not hot swappable. This is primarily a memory interface problem and not an UltraDIMM issue but nonetheless, you can’t swap an UltraDIMM module until the server is powered down. And who would want to do such a thing when the server is powered anyway?

SanDisk long history in NAND

As you can see from the three photos at right SanDisk seems to have been involved in flash/NAND technology innovation since the early 1990’s. At the time NOR and NAND were competing for almost the same market.

But sometime in the mid to late 1990’s NAND found a niche in consumer cameras and never looked back. Not sure where NOR marketis today but it’s a drop in the bucket compared to the NAND market

UltraDIMMs is just the latest platform to support NAND storage access. It happens to be one with blazingly fast access times and high IO parallelism, but in the end it just represents another way to obtain the benefits of NAND for IT customers.

Also, SanDisk’s commercial NAND (Memory Card) business seems to be very healthy. What with higher resolution photos/video/audio coming online over the next decade or so it doesn’t seem to be going away anytime soon.

SanDisk is in a new joint venture (JV) with Toshiba to produce 3D NAND flash. But in the mean time they are still using 2D flash for their current SSD storage. Toshiba and SanDisk in their current JV together manufacture about 1/2 the NAND bits in the world today.

The rest of SanDisk NAND business also seem to be doing well. And the aforementioned JV with Toshiba on 3D NAND looks positioned to take all of this NAND to the next level of density as well which should make all of us happy.

SanDisk acquiring FusionIO

SanDisk was in the news lately as they have recently filed to acquire FusionIO, a prominent and early PCIe flash supplier that in recent years has broadened their portfolio to include enterprise storage with their acquisition of NexGen storage (renamed IO Control).

When FusionIO IPO’d the stock sold at ~$19/share and SanDisk is purchasing the company in an all cash deal for $11.25/share almost a 40% reduction in share price in 3 years (June’11 IPO) – ouch. At IPO the company was valued at ~$2B, (some pundits said this was ~$1.5B, so there’s some debate on the original valuation). SanDisk is buying the company for ~$1.1B in cash. Any way you look at it, they paid significantly less than what the company was worth at IPO. Granted, it was valued at 41X earnings then and its recent stock price at $11.59 represents a 3.3P/E (ttm).

Not exactly certain what happened. Analysts seem to indicate that Apple and Facebook, FusionIO’s biggest customers were buying less FusionIO product. I also happen to think that the PCIe flash space has gotten pretty crowded over the last 3 years with entrants from Micron Technologies, Intel, LSI, Verident/Western Digital, and others.

In addition, for PCIe flash to broaden its market there’s a serious need to surround it with sophisticated caching software to enable a more general purpose IO solution (see Pernix Data, Proximal Data, and others). These general purpose, caching solutions have finally reached high levels of sophistication and just now are becoming more widely available.

~~~~

Originally, part 3 of this series was going to be on IBM’s release of the UltraDIMM technology as their new eXFlash. However, I am somewhat surprised not to see other vendors taking up the MCS/UltraDIMM technology but IBM may have a limited exclusivity to it.

The only other thing thats this interesting happening in solid state storage is HP’s Memristor Machine which is still a ways off.

Nonetheless, a new much faster memory card based SSD is hitting the market and if history is any indication, it won’t be long until the data storage world will sit up and take notice.

Comments?

MCS, UltraDIMMs and memory IO, the new path ahead – part 1

Posted on May 22, 2014 by Ray in Block Storage, DAS, SSD storage, Storage, Storage architecture, Storage performance

I was at Storage Field Day 5 (SFD5) last month and got a chance to talk with SanDisk and Diablo Technologies. It turns out that SanDisk’s UltraDIMM product is based on Diablo Technologies MCS hardware. So the two of them provided a pretty deep dive into the technology and where they want to go with it. Before we go any deeper the UltraDIMMs will be released to the field by IBM under the eXFlash name.

The team at Diablo have been focusing on the x86 standard memory channel for a while now and lately have been trying out different sorts of technologies to connect as CPU memory. The first Memory Channel Storage (MCS) product converts Memory Channel IO to SATA IO. This allows any SATA device to be attached as memory and enjoy lightening fast, memory access times. Access times are clocked at 7µsec. Most PCIe Flash cards have an access latency at 50µsec or more, so this is 7X faster that PCIe Flash. They also claim the MCS is capable of 20GB/sec. I know enterprise class storage systems that can’t do that. Also, the MCS utilizes 2 memory channels.

Diablo delivers a chip (that converts MemIO to SATA IO) and software that provides a block IO access to the MCS device. Customers of MCS supply their own SATA flash storage device and presumably package it all together in a DIMM compatible card.

But the main problem is that the whole MCS chip and SATA IO flash device has to fit in the form factor of a DIMM. And cannot draw any more power than a memory device can draw, ~10-15W with its corresponding thermal load.

But this seems plenty for a small flash drive. The MCS is configured as a 4GB DDR3 DIMM. There is a requirement to patch the BIOS so that it doesn’t run diagnostic memory tests on the MCS device and their software needs to be loaded to access the device as a block device. I believe they currently support Linux O/S with more O/Ss on the way.

Diablo has looked at other applications for their technology including providing an Memory IO accessed Ethernet NIC was mentioned. But it seems flash storage would be a great first application of their technology. Not clear to me but SAS would also be something that could be done.

Whatever happens after NAND with the next generation semiconductor storage (see my The end of NAND is near post, it seems to me that accessing it as Memory IO would make an awful lot of sense. makes a lot of sense. Using MCS as the access channel would seem to be a logical next step.

Part 1 of this story is on Diablo Technologies, Part 2 will be on SanDisk and I am not sure but maybe there will be a Part 3 on IBM eXFlash. So stay tuned.

Comments?