What really drives storage innovation

Ongoing waves of consolidation remind me of what really drives storage innovation – companies willing to experiment. Startups can only succeed when their products can engage the marketplace.

Startups risk everything to develop technology an innovation or two that can change the world. But what they ultimately discover, what they truly need is some large and/or small company’s willingness to experiment with new and untried technology. Such market engagement is essential to understand their technology’s rough edges, customer requirements, and distribution options.

Recently, I was informed that some large companies prefer to work with startups because they can better control any emerging technology direction. Also, their problems are big enough that typically no one solution can solve them. Startups allow them to cobble together (multiple) solutions that ultimately can solve their problem.

From the small company’s perspective the question becomes how to attract and begin the dialogue with innovative customers willing to invest time and money in startups. But, the real problem is knowing enough about a customer’s environment to know if suitable prospects for their technology exist. Armed with this knowledge, targeted marketing approaches can be applied to ultimately get a hearing with the customer.

However, what’s missing is a forum for large and small companies to describe their environment and more importantly, their serious, chronic problems. Mostly, this has been done informally or on an ad hoc basis in the past, but some formality around this could really benefit storage innovation at least from startups.

I see many possibilities to solve this, ways that companies could provide information on their environment and identify problems needing solutions. Such possibilities include:

  • an electronic forum something like Innocentive.com where companies could post problems and solicit solutions
  • an award to solve a particularly pressing problem like Xprize.org where a group of companies, perhaps in one vertical combine together to offer a significant award to help solve a particular nasty storage/IT problem
  • an organization of sorts like SNIA end user council that could provide anonymous information on IT environments and problems needing solutions.
  • a Small Business Innovation Research-like (see SBIR.gov) that could provide a list of problems soliciting solutions

The problem with SNIA end user council and SBIR-like approaches is the lack of anonyminity, the problems with an Xprize-like award is the inability for any one organization to fund the award. All of which is why I prefer an innocentive.com-like approach, maybe better targeted to IT issues and less targeted on basic and materials science. Finally, perhaps another, unforeseen approach that might even work better – comments?

Why big storage vendors can’t be enticed to work on something like this is another conundrum and probably subject for a future post.

HDS Dynamic Provisioning for AMS

HDS announced support today for their thin provisioning (called Dynamic Provisioning) feature to be available in their mid-range storage subsystem family the AMS. Expanding the subsystems that support Thin provisioning can only help the customer in the long run.

It’s not clear whether you can add dynamic provisioning to an already in place AMS subsystem or if it’s only available on a fresh installation of an AMS subsystem. Also no pricing was announced for this feature. In the past, HDS charged double the price of a GB of storage when it was in a thinly provisioned pool.

As you may recall, thin provisioning is a little like a room with a bunch of inflatable castles inside. Each castle starts with it’s initial inflation amount. As demand dictates, each castle can independently inflate to whatever level is needed to support the current workload up to that castles limit and the overall limit imposed by the room the castles inhabit. In this analogy, the castles are LUN storage volumes, the room the castles are located in, is the physical storage pool for the thinly provisioned volumes, and the air inside the castles is the physical disk space consumed by the thinly provisioned volumes.

In contrast, hard provisioning is like building permanent castles (LUNS) in stone, any change to the size of a structure would require major renovation and/or possible destruction of the original castle (deletion of the LUN).

When HDS first came out with dynamic provisioning it was only available for USP-V internal storage, later they released the functionality for USP-V external storage. This announcement seems to complete the roll out to all their SAN storage subsystems.

HDS also announced today a new service called the Storage Reclamation Service that helps
1) Assess whether thin provisioning will work well in your environment
2) Provide tools and support to identify candidate LUNs for thin provisioning, and
3) Configure new thinly provisioned LUNs and migrate your data over to the thinly provisioned storage.

Other products that support SAN storage thin provisioning include 3PAR, Compellent, EMC DMX, IBM SVC, NetApp and PillarData.

Latest SPC-1 IOPS vs LRT Chart Of The Month

SPC-1* IOPS v LRT for storage subsystems under $100/GB with subsystem price ($K) as bubble size
SPC-1* IOPS v LRT with subsystem cost as bubble size, (C) 2009 Silverton Consulting, Inc.
This chart was included in our last months newsletter and shows relative costs of subsystem storage as well as subsystems performance on two axis SPC-1 IO operations per second and measured Least Response Time.

Having the spreadsheet, I can easily tell which bubble is which subsystem but have yet to figure out an easy way for Excel to label the bubbles. For example the two largest bubbles with highest IOPs performance are the IBM SVC4.3 and 3PAR Inserv T800 subsystems.

The IBM SVC is a storage virtualization engine which has 16-DS4700 storage subsystems behind it with 8-SVC nodes using 1536-146GB drives at a total cost of $3.2M. Whereas the 3PAR has 8 T-Series controller nodes with 1280-146GB drives at a total cost of $2.1M.

I am constantly looking for new ways to depict storage performance data and found that other than the lack of labels, this was almost perfect. It offered both IOPS and LRT performance metrics as well as subsystem price in one chart.

This chart and others like it were sent out in last months SCI newsletter. If you are interested in receiving your own copy of next months newsletter please drop me an email

*Information for this chart is from the Storage Performance Council and can be found StoragePerformance.org

Norton Online Backup ships with HP computers

Symantec announced today that Norton Online Backup software will be shipping with HP PCs and Laptops. Norton Online Backup is a cloud storage solution which can be used to backup your data on your PC.

Norton Online currently has about 32PB of consumer data and is growing by about 5PB/Qtr and is currently number one in online backup market. Also Norton online has about 8M users today growing 100% each year. With the HP announcement today all of these metrics will just increase even faster.

Consumers create over 70% of the worlds digital data with a 60% CAGR. Roughly about 2% of consumers use online backup services and ~25% of never backup at all. Norton Online Backup, EMC’s Mozy, Carbonite and others are attempting to entice these backup shy users to start backing up their data online and forgo onsite headaches of doing it yourself.

Apparently with Norton Online one can back up up to 5 machines and they can be located anywhere. So if you wanted to backup your kid’s pc at college and your parent’s pc at their retirement village you could do this with one Norton online license (as long as the total machine count < =5). Once backed up the data can be restored to any machine and takes just a few clicks. Backing up your pc is easy to setup and once done can be forgotten. Then whenever you are on the internet and the machine is not busy, the data just trickles out to the Norton Online backup service. The Norton Online Backup service is renewed yearly and cost is based on storage quantity backed up. How Symantec stores and records 32PB of user backup data is non-trivial but I am told it is all done using commodity hardware and commodity disk drives with nary a SAN in sight. They have multiple data centers, professionally managed, supporting Symantec developed/acquired cloud storage services. Apparently, Norton Online Backup is an outgrowth of Symantec's SwapDrive acquisition from last year. Symantec appears to be the leader in cloud storage applications and this would seem to be just the start of the services that Symantec will deploy via the cloud. Now if they only had something for the Mac... Technorati Profile

EMC Better At Acquisitions?

I was talking with an EMCer the other day about the Data Domain deal and he said that EMC does very well with acquisitions. Just about every EMC product line other than Symmetrix (and possibly Celerra, Invista, PowerPath and maybe others) came from an acquisition in EMC’s past.

The list goes something like this Clariion from Data General, Centerra from FilePool, Control Center from BMC, Networker from Legato, RainFinity, Avamar, Documentum, RSA all from companies of the same name. There are other examples as well but these should suffice for now. One almost starts to forget about all these separate companies that existed prior to EMC’s acquisitions. Over time EMC manages to succeed in advancing and integrating the various technologies and products into their portfolio.

On the other extreme is Sun. They have almost a perfect record of acquiring companies and burying the technology away. Often the technology does emerge after a gestation period in another Sun product somewhere else but just as often it just fades away never to be seen again.

Today’s companies have to do acquisitions well. They can no longer afford the luxury to acquire companies and then see their investment die away. Those days are long gone

What makes EMC so successful while others can do so poorly? One thing I have learned is that EMC leaves a new acquisition pretty much alone for 12 months or so. During that time presumably they are assessing the current management team for EMC cultural fit and determining the best way to sell, advance and integrate the acquired technology into the rest of EMC’s product and services portfolio.

The other thing I have noticed is that EMC’s most recentr acquisitions have retained at least portions of their original brand names. Networker, RainFinity, Documentum, and RSA are examples here.

I don’t know what it is about retaining a brand name but 1) it makes it harder to let it fade away because it’s so visible, 2) employees who have a personal interest in the brand fight to keep it alive and advancing, and 3) customer base and loyalty is retained better.

Just pieces of the puzzle but no doubt there is more to this than is visible externally.

How well NetApp will do as an Acquirer is another question. I know they have acquired Spinnaker, Alacritus, Decru, Topio, and Onaro over the past five years. Most of these products are still being sold. Rumors point to Spinnaker technology being merged into NetApp’s mainline product soon. All in all, I would have to say that although NetApp has retained the product names for most of these products Onaro’s SANScreen, Decru’s DataFort and others, they haven’t necessarily done a good job keeping the brandnames alive.

What NetApp will do with Data Domain however, is another matter entirely. First, the price being paid is much higher than any previous acquisitions. Second, the market share that Data Domain currently enjoys is much larger than any previous acquisition. Finally, it’s crucial to NetApp’s future revenue growth to do this one right. Given all that, I truly believe they will do a much better job with retaining Data Domain’s brand and product names, thereby keeping the product alive and well for the foreseeable future.


HDS High Availability Manager(HAM)

What does HAM look like to the open systems end user. We need to break this question up into two parts – one part for USP-V internal storage and the other part for external storage.

It appears that for internal storage first you need data replication services such as asynch or synchronous replication between the two USP-V storage subsystems. But here you still need some shared External storage used as a quorum disk. Then once all this is set up under HAM the two subsystems can automatically failover access to the replicated internal and shared external storage from one USP-V to the other.

For external storage it appears that this storage must be shared between the two USP-V systems and whenever the primary one fails the secondary one can take over (failover) data storage responsibilities for the failing USP-V frontend.

What does this do for data migration? Apparently, using automated failover with HAM one can migrate date between two different storage pools and then failover server access from one to the other non-disruptively.

Obviously all the servers accessing storage under HAM control would need to be able to access both USP-Vs in order for this to all work properly.

Continuous availability is a hard nut to crack. HDS seems to have taken a shot at doing this from a purely storage subsystem perspective. This might be very useful for data centers running heterogeneous server environments. Typically server clustering software is OS specific like MSCS. Symantec being the lone exception with VCS which supports multiple OSs. Such server clustering can handle storage outages but also depend on storage replication services to make this work.

Unclear to me which is preferable but when you add the non-disruptive data migration – it seems that HAM might make sense.

Data Domain bidding war

It’s unclear to me what EMC would want with Data Domain (DD) other than to lockup deduplication technology across the enterprise. EMC has Avamar for Source dedupe, has DL for target dedupe, has Celerra Dedupe and the only one’s missing are V-Max, Symm & Clariion dedupe.

My guess is that EMC sees Data Domain’s market share as the primary target. It doesn’t take a lot of imagination to figure that once Data Domain is a part of EMC, EMC’s Disk Library (DL) offerings will move over to DD technology. Which probably leaves FalconStor/Quantum technology used in DL today as outsiders.

EMC’s $100M loan to Quantum last month probably was just insurance to keep a business partner afloat until something better came along or they could make it on their own. The DD deal would leave Quantum parntership supporting EMC with just Quantum’s tape offerings.

Quantum deduplication technology doesn’t have nearly the market share that DD has in the enterprise but they have won a number of OEM deals not the least of which is EMC and they were looking to expand. But if EMC buys DD, this OEM agreement will end soon.

I wonder if DD is worth $1.8B in cash what could Sepaton be worth. They seem to be the only pure play dedupe appliance left standing out there.

Not sure whether NetApp will up their bid but they always seem to enjoy competing with EMC. Also unclear how much of this bid is EMC wanting DD or EMC just wanting to hurt NetApp, either way DD stockholders win out in the end.

Data Domain and NetApp

Data Domain has been a longtime partner of NetApp’s, which is probably one reason that NetApp finally decided to make them an offer. Another reason why it’s right to do this now is that in today’s economy, NetApp could get the best price.

The final reason that NetApp and Data Domain should hook up is that there are not many other major storage vendors that don’t already have a dedicated deduplication appliance or two. If Sun were still around it might make sense for them to think about buying Data Domain but they are out of the picture until Oracle figures out what to do with their storage business. EMC has bought Avamar and invested significantly in Quantum. IBM has purchased Diligent, Symantec has PureDisk. HP already has a deduplication product. The only major vendor without dedupe today is HDS.

Data Domain had a lot going for them. They practically defined the target deduplication appliance market. Diligent (now with IBM), Quantum, Sepaton, and others notwithstanding, Data Domain had the largest market share out there and was continuing to experience rapid growth. The fact that Data Domain both supported NAS as well as VTL access modes coupled with their excellent market share made them a prime acquisition on many fronts.

NetApp, of course, has their own deduplication technology which has been very successful in supporting virtual server environments primary storage and was also used to support secondary storage. No doubt over time these two technologies could conceivably merge into one. But don’t hold your breath, some companies have way more than two distinct deduplication technologies which are used for their various products and NetApp may not feel its worthwhile to combine the two technologies in the near future given there two different markets.

All in all, consolidation is necessary evil today.