Backup – Silverton Consulting

SCI’s Latest ESRP (v3) Performance Analysis for Over 5K mailboxes – chart of the month

Posted on February 14, 2012October 8, 2012 by Ray in Block Storage, Disk storage, ESRP, ESRPv3/Exchange 2010, Storage, Storage Backup, Storage performance, System effectiveness

Bar chart showing ESRP Top 10 total database backup throughput results — (SCIESRP120125-001) (c) 2012 Silverton Consulting, All Rights Reserved

This chart comes from our analysis of Microsoft Exchange Reviewed Solutions Program (ESRP) v3 (Exchange 2010) performance results for the over 5000 mailbox category, a report sent out to SCI Storage Intelligence Newsletter subscribers last month.

The total database backup throughput reported here is calculated based on the MB read per second per database times the number of databases in a storage configuration. ESRP currently reports two metrics for database backups the first used in our calculation above is the backup throughput on a database basis and the second is backup throughput on a server basis. I find neither of these that useful from a storage system perspective so we have calculated a new metric for our ESRP analysis which represents the total Exchange database backup per storage system.

I find three submissions (#1, #3 & #8 above) for the same storage system (HDS USP-V) somewhat unusual in any top 10 chart and as such, provides a unique opportunity to understand how to optimize storage for Exchange 2010.

For example, the first thing I noticed when looking at the details behind the above chart is that disk speed doesn’t speed up database throughput. The #1, #3 and #8 submissions above (HDS USP-V using Dynamic or thin provisioning) had 7200rpm, 15Krpm and 7200rpm disks respectively with 512 disk each.

So what were the significant differences between the USP-V submissions (#1, #3 and #8) aside from mailbox counts and disk speed:

Mailboxes per server differed from 7000 to 6000 to 4700 respectively, with the top 10 median = 7500
Mailboxes per database differed from 583 to 1500 to 392, with the top 10 median = 604
Number of databases per host (Exchange server) differed from 12 to 4 to 12, with the top 10 median = 12
Disk size differed from 2TB to 600GB to 2TB, with the top 10 median = 2TB
Mailbox size differed from 1GB to 1GB to 3GB, with the top 10 median = 1.0 GB
% storage capacity used by Exchange databases differed from 27.4% to 80.0% to 55.1%, with the top 10 median = 60.9%

If I had to guess, the reason the HDS USP-V system with faster disks didn’t backup as well as the #1 system is that its mailbox databases spanned multiple physical disks. For instance, in the case of the (#3) 15Krpm/600GB FC disk system it took at least 3 disks to hold a 1.5TB mailbox database. For the #1 performing 7200rpm/2TB SATA disk system, a single disk could hold almost 4-583GB databases on a single disk. The slowest performer (#8) also with 7200rpm/2TB SATA disks could hold at most 1-1.2TB mailbox database per disk.

One other thing that might be a factor between the #1 and #3 submissions is that the data being backed up per host was significantly different. Specifically for a host in the #1 HDS USP-V solution they only backed up ~4TB but for a host in the #3 submission they had to backup ~9TB. However, this doesn’t help explain the #8 solution, which only had to backup 5.5TB per host.

How thin provisioning and average space utilization might have messed all this up is another question entirely. RAID 10 was used for all the USP-V configurations, with a 2d+2d disk configuration per RAID group. The LDEV configuration in the RAID groups was pretty different, i.e., for #1 & #8 there were two LDEVs one 2.99TB and the other .58TB whereas for the #3 there was one LDEV of 1.05TB. These LDEVs were then added to Dynamic Provisioning pools for database and log files. (There might be something in how the LDEVs were mapped to V-VOL groups but I can’t seem to figure it out.)

Probably something else I might be missing here but I believe a detailed study of these three HDS USP-V systems ESRP performance would illustrate some principles on how to accelerate Exchange storage performance.

I suppose the obvious points to come away with here are to keep your Exchange databases smaller than your physical disk sizes and don’t overburden your Exchange servers.

~~~~

The full ESRP performance report went out to our newsletter subscriber’s this past January. A copy of the full report will be up on the dispatches page of our website sometime next month (if all goes well). However, you can get the full ESRP performance analysis now and subscribe to future newsletters by just sending us an email or using the signup form above right.

For a more extensive discussion of block or SAN storage performance covering SPC-1&-2 (top 30) and ESRP (top 20) results please consider purchasing SCI’s SAN Storage Buying Guide available on our website.

As always, we welcome any suggestions on how to improve our analysis of ESRP results or any of our other storage performance analyses.

Comments?

EMC NetWorker 7.6 SP1 surfaces

Posted on October 8, 2010 by Ray in data protection, Storage Backup

This week EMC releases NetWorker 7.6 SP1 with new Boost support for Data Domain (DD) appliances which allows NetWorker’s storage node (media server) and the DD appliance to jointly work on providing deduplication services. Earlier this year EMC DD announced the new Boost functionality which at the time only worked with Symantec’s OST interface. But with this latest service pack (SP1), NetWorker also offers this feature and EMC takes another step to integrate DD systems and functonality across their product portfolio.

DD Boost integration with NetWorker

DD Boost functionality resides on the NetWorker storage node which transfers data to backend storage. Boost offloads the cutting up of data into segments fingerprinting segments and passing the fingerprints to DD. Thereafter NetWorker only passes unique data between the storage node and the DD appliance.

Doing this reduces the processing workload on DD appliance, uses less network bandwidth, and on the NetWorker storage node itself, reduces the processing requirements. While this later reduction may surprise some, realize the storage node primarily moves data and with DD Boost, it moves less data, consuming less processing power. All in all, with NetWorker-DD Boost vs. NetWorker using DD in NFS mode there is a SIGNIFICANT improvement in data ingest performance/throughput.

DD cloning controlled by NetWorker

Also the latest SP incorporates DD management integration, such that an admin can control DataDomain replication from the NetWorker management console alone. Thus, the operator no longer needs to use the DD management interface to schedule, monitor, and terminate DD replication services.

Additionally, NetWorker can now be aware of all DD replicas and as such, can establish separate retention periods for each replica all from the NetWorker management interface. Another advantage is that now tape clones of DD data can be completely managed from the NetWorker management console.

Furthermore, one can now configure new DD appliances as a NetWorker resource using new configuration wizards. NetWorker also supports monitoring and alerting on DD appliances through the NetWorker management console which includes capacity utilization and dedupe rates.

Other enhancements made to NetWorker

NetWorker Cloning – scheduling of clones no longer requires CLI scripts and is now can be managed within the GUI as well. NetWorker cloning is the process which replicates save sets to other storage media.
NetWorker Checkpoint/Restart- resuming backups from known good points after a failure. Checkpoint/Restart can be used for very large save sets which cannot complete within a window.

New capacity based licensing for NetWorker

It seems like everyone is simplifying their licensing (see CommVault’s Simpana 9 release). With this version of NetWorker, EMC now supports a capacity based licensing option in addition to their current component- and feature-based licensing. With all the features of the NetWorker product, component-based licensing has become more complex and cumbersome to use. The new Capacity License Option charges on the amount of data being protected and all NetWorker features are included at no additional charge.

The new licensing option is available worldwide, with no tiers of capacity based licensing for feature use, i.e., one level of capacity based licensing. Capacity based licensing can be more cost effective for those using advanced NetWorker features, should be easier to track, and will be easier to install. Anyone under current maintenance can convert to the new licensing model but it requires this release of NetWorker software.

—-

NetWorker’s 7.6 SP1 is not a full release but substantial nonetheless. Not the least of which is the DD Boost and management integration being rolled out. Also, I believe the new licensing option may appeal to a majority of their customer base but one has to do the math. Probably some other enhancements I missed here but these seem the most substantial.

What do you think?

Cloud storage, CDP & deduplication

Posted on August 20, 2010 by Ray in Block Storage, Cloud services, Data, Data efficiency, data protection, File Storage, Storage architecture, Storage Backup

Strange Clouds by michaelroper (cc) (from Flickr)

Somebody needs to create a system that encompasses continuous data protection, deduplication and cloud storage. Many vendors have various parts of such a solution but none to my knowledge has put it all together.

Why CDP, deduplication and cloud storage?

We have written about cloud problems in the past (eventual data consistency and what’s holding back the cloud) despite all that, backup is a killer app for cloud storage. Many of us would like to keep backup data around for a very long time. But storage costs govern how long data can be retained. Cloud storage with its low cost/GB/month can help minimize such concerns.

We have also blogged about dedupe in the past (describing dedupe) and have written in industry press and our own StorInt dispatches on dedupe product introductions/enhancements. Deduplication can reduce storage footprint and works especially well for backup which often saves the same data over and over again. By combining deduplication with cloud storage we can reduce the data transfers and data stored on the cloud, minimizing costs even more.

CDP is more troublesome and yets still worthy of discussion. Continuous data protection has always been sort of a step child in the backup business. As a technologist, I understand it’s limitations (application consistency) and understand why it has been unable to take off effectively (false starts). But, in theory at some point CDP will work, at some point CDP will use the cloud, at some point CDP will embrace deduplication and when that happens it could be the start of an ideal backup environment.

Deduplicating CDP using cloud storage

Let me describe the CDP-Cloud-Deduplication appliance that I envision. Whether through O/S, Hypervisor or storage (sub-)system agents, the system traps all writes (forks the write) and sends the data and meta-data in real time to another appliance. Once in the CDP appliance, the data can be deduplicated and any unique data plus meta data can be packaged up, buffered, and deposited in the cloud. All this happens in an ongoing fashion throughout the day.

Sometime later, a restore is requested. The appliance looks up the appropriate mapping for the data being restored, issues requests to read the data from the cloud and reconstitutes (un-deduplicates) the data before copying it to the restoration location.

Problems?

The problems with this solution include:

Application consistency
Data backup timeframes
Appliance throughput
Cloud storage throughput

By tieing the appliance to a storage (sub-)system one may be able to get around some of these problems.

One could configure the appliance throughput to match the typical write workload of the storage. This could provide an upper limit as to when the data is at least duplicated in the appliance but not necessarily backed up (pseudo backup timeframe).

As for throughput, if we could somehow understand the average write and deduplication rates we could configure the appliance and cloud storage pipes accordingly. In this fashion, we could match appliance throughput to the deduplicated write workload (appliance and cloud storage throughput)

Application consistency is more substantial concern. For example, copying every write to a file doesn’t mean one can recover the file. The problem is at some point the file is actually closed and that’s the only time it is in an application consistent state. Recovering to a point before or after this, leaves a partially updated, potentially corrupted file, of little use to anyone without major effort to transform it into a valid and consistent file image.

To provide application consistency, one needs to somehow understand when files are closed or applications quiesced. Application consistency needs would argue for some sort of O/S or hypervisor agent rather than storage (sub-)system interface. Such an approach could be more cognizant of file closure or application quiesce, allowing a synch point could be inserted in the meta-data stream for the captured data.

Most backup software has long mastered application consistency through the use of application and/or O/S APIs/other facilities to synchronize backups to when the application or user community is quiesced. CDP must take advantage of the same facilities.

Seems simple enough, tie cloud storage behind a CDP appliance that supports deduplication. Something like this could be packaged up in a cloud storage gateway or similar appliance. Such a system could be an ideal application for cloud storage and would make backups transparent and very efficient.

What do you think?

Backup is for (E)discovery too

Posted on December 18, 2009 by Ray in Data discovery, Data search, Systems

Electronic Discovery Reference Model (from EDRM.net)

There has been lot’s of talk in twitterverse and elsewhere on how “backup is used for restore and archive is for e-discovery”, but I beg to differ.

If one were to take the time to review the EDRM (Electronic Discovery Reference Model) and analyze what happens during actual e-discovery processes, one would see that nothing is outside the domain of court discovery requests. Backups have and always will hold discoverable data just as online and user desktop/laptop storage do. In contrast, archives are not necessarily a primary source of discoverable data.

In my view, any data not in archive, by definition is online or on user desktop/laptop storage. Once online, data is most likely being backed up periodically and will show up in backups long before it’s moved to archive. Data deletions and other modifications can often be reconstructed from backups much better than from archive (with the possible exception of records management systems). Also, reconstructing data proliferation, such as who had a copy of what data when, is often crucial to court proceedings and normally, can only be reconstructed from backups.

Archives have a number of purposes but primarily it’s to move data that doesn’t change off company storage and out of its backup stream. Another popular reason for archive is to be used to satisfy compliance regimens that require companies to hold data for periods of time, such as mandated by SEC, HIPPA, SOX, and others. For example, SEC brokerage records must be held long after an account goes inactive, HIPPA health records must be held long after a hospital visit, SOX requires corporate records to be held long after corporate transactions transpire. Such records are more for compliance and/or customer back-history request purposes than e-discovery but here again any data stored by the corporation is discoverable.

So I believe it’s wrong to say that Backup is only for restore and archive is only for discovery. Information, anywhere within a company is discoverable. However, I would venture to say that a majority of e-discovery data comes from backups rather than elsewhere.

Now, as for using backups for restore,…

Problems solved, introduced and left unsolved by cloud storage

Posted on November 17, 2009 by Ray in Disaster Recovery, File Storage, Storage, Storage Backup, storage economics, Storage performance, Systems

Cloud whisps (sic) by turtlemom4bacon (cc) (from flickr)

When I first heard about cloud storage I wondered just what exactly it was trying to solve. There are many storage problems within the IT shop nowadays ~~day~~s, cloud storage can solve a few of them but introduces more and leaves a few unsolved.

Storage problems solved by cloud storage

Dynamic capacity – storage capacity is fixed once purchased/leased. Cloud storage provides an almost infinite amount of storage for your data. One pays for this storage, in GB or TB per month increments, with added storage services (multi-site replication, high availability, etc.) at extra charge. Such capacity can be reduced or expanded at a moments notice.
Offsite DR – disaster recovery for many small shops is often non-existent or rudimentary at best. Using cloud storage, data can be copied to the cloud and accessed anywhere via the internet. Such data copies can easily support rudimentary DR for a primary data center outage.
Access anywhere – storage is typically local to the IT shop and can normally only be accessed at that location. Cloud storage can be accessed from any internet access point. Applications that are designed to operate all over the world can easily take advantage of such storage.
Data replication – data should be replicated for high availability. Cloud storage providers can replicate your data to multiple sites so that if one site goes down other sites can still provide service.

Storage problems introduced by the cloud

Variable access times – local storage access times vary from 1 and 100 milleseconds. However, accessing cloud storage can take from 100’s of milleseconds to minutes depending on network connectivity. Many applications cannot endure such variable access times.
Different access protocols – local storage support fairly standard access protocols like FC, iSCSI, NFS, and/or CIFS/SMB. Barring the few (but lately increasing) cloud providers that provide NFS access protocol, most cloud storage requires rewriting applications to use new protocols such as REST to store and access cloud file data.
Governance over data – local storage is by definition all located inside one data center. Many countries do not allow personal and/or financial data to be stored outside the country of origin. Some cloud storage providers will not guarantee that data stored in the cloud couldn’t be stored outside the country and jurisdiction of a single country.

Storage problems not solved by the cloud:

Data backups – data protection via some form of backup is essential. Nothing says that cloud storage providers cannot provide backup of data in the cloud but few if any provide such service. See my Are backups needed in the cloud post.
Data security – data security remains an ongoing problem for the local data center moving the data to the cloud just makes security more difficult. Many cloud storage providers provide rudimentary security for data stored but none seem to have integrated strong authentication and encryption services that might provide true data security.
Energy consumption – today’s storage consumes power and cooling. Although, cloud storage can be more efficient than onsite storage, this does not eliminate the environmental cost of storage.
Data longevity – data stored in the cloud can just as easily go obsolete as data stored locally.

Probably some I have missed here but these are a good start.

Protecting the Yottabyte archive

Posted on November 11, 2009January 27, 2010 by Ray in Data index, Networking, Storage, Storage Backup, Storage density, Storage performance, Systems

blinkenlights by habi (cc) (from flickr)

In a previous post I discussed what it would take to store 1YB of data in 2015 for the National Security Agency (NSA). Due to length, that post did not discuss many other aspects of the 1YB archive such as ingest, index, data protection, etc. Thus, I will attempt to cover each of these in turn and as such, this post will cover some of the data protection aspects of the 1YB archive and its catalog/index.

RAID protecting 1YB of data

Protecting the 1YB archive will require some sort of parity protection. RAID data protection could certainly be used and may need to be extended to removable media (RAID for tape), but that would require somewhere in the neighborhood of 10-20% additional storage (RAID5 across 10 to 5 tape drives). It’s possible with Reed-Solomon encoding and using RAID6 that we could take this down to 5-10% of additional storage (RAID 6 for a 40 to a 20 wide tape drive stripe). Possibly other forms of ECC (such as turbo codes) might be usable in a RAID like configuration which would give even better reliability with less additional storage.

But RAID like protection also applies to the data catalog and indexes required to access the 1YB archive of data. Ditto for the online data itself while it’s being ingested, indexed, or readback. For the remainder of this post I ignore the RAID overhead but suffice it to say with today’s an additional 10% storage for parity will not change this discussion much.

Also in the original post I envisioned a multi-tier storage hierarchy but the lowest tier always held a copy of any files residing in the upper tiers. This would provide some RAID1 like redundancy for any online data. This might be pretty usefull, i.e., if a file is of high interest, it could have been accessed recently and therefore resides in upper storage tiers. As such, multiple copies of interesting files could exist.

Catalog and indexes backups for 1YB archive

IMHO, RAID or other parity protection is different than data backup. Data backup is generally used as a last line of defense for hardware failure, software failure or user error (deleting the wrong data). It’s certainly possible that the lowest tier data is stored on some sort of WORM (write once read many times) media meaning it cannot be overwritten, eliminating one class of user error.

But this presumes the catalog is available and the media is locatable. Which means the catalog has to be preserved/protected from user error, HW and SW failures. I wrote about whether cloud storage needs backup in a prior post and feel strongly that the 1YB archive would also require backups as well.

In general, backup today is done by copying the data to some other storage and keeping that storage offsite from the original data center. At this amount of data, most likely the 2.1×10**21 of catalog (see original post) and index data would be copied to some form of removable media. The catalog is most important as the other two indexes could potentially be rebuilt from the catalog and original data. Assuming we are unwilling to reindex the data, with LTO-6 tape cartridges, the catalog and index backups would take 1.3×10**9 LTO-6 cartridges (at 1.6×10**12 bytes/cartridge).

To back up this amount of data once per month would take a gaggle of tape drives. There are ~2.6×10**6 seconds/month and each LTO-6 drive can transfer 5.4×10**8 bytes/sec or 1.4X10**15 bytes/drive-month but we need to backup 2.1×10**21 bytes of data so we need ~1.5×10**6 tape transports. Now tapes do not operate 100% of the time because when a cartridge becomes full it has to be changed out with an empty one, but this amounts to a rounding error at these numbers.

To figure out the tape robotics needed to service 1.5×10**6 transports we could use the latest T-finity tape library just announced by Spectra Logic . The T-Finity supports 500 tape drives and 122,000 tape cartridges, so we would need 3.0×10**3 libraries to handle the drive workload and about 1.1×10**4 libraries to store the cartridge set required, so 11,000 T-finity libraries would suffice. Presumably, using LTO-7 these numbers could be cut in half ~5,500 libraries, ~7.5×10**5 transports, and 6.6×10**8 cartridges.

Other removable media exist, most notably the Prostor RDX. However RDX roadmap info out to the next generation are not readily available and high-end robotics are do not currently support RDX. So for the moment tape seems the only viable removable backup for the catalog and index for the 1YB archive.

Mirroring the data

Another approach to protecting the data is to mirror the catalog and index data. This involves taking the data and copying it to another online storage repository. This doubles the storage required (to 4.2×10**21 bytes of storage). Replication doesn’t easily protect from user error but is an option worthy of consideration.

Networking infrastructure needed

Whether mirroring or backing up to tape, moving this amount of data will require substantial networking infrastructure. If we assume that in 2105 we have 32GFC (32 gb/sec fibre channel interfaces). Each interface could potentially transfer 3.2GB/s or 3.2×10**9 bytes/sec. Mirroring or backing up 2.1×10**21 bytes over one month will take ~2.5×10**6 32GFC interfaces. Probably should have twice this amount of networking just to not have any one be a bottleneck so 5×10**6 32GFC interfaces should work.

As for switches, the current Brocade DCX supports 768 8GFC ports and presumably similar port counts will be available in 2015 to support 32GFC. In addition if we assume at least 2 ports per link, we will need ~6,500 fully populated DCX switches. This doesn’t account for multi-layer switches and other sophisticated switch topologies but could be accommodated with another factor of 2 or ~13,000 switches.

Hot backups require journals

This all assumes we can do catalog and index backups once per month and take the whole month to do them. Now storage today normally has to be taken offline (via snapshot or some other mechanism) to be backed up in a consistent state. While it’s not impossible to backup data that is concurrently being updated it is more difficult. In this case, one needs to maintain a journal file of the updates going on while the data is being backed up and be able to apply the journaled changes to the data backed up.

For the moment I am not going to determine the storage requirements for the journal file required to cover the catalog transactions for a month, but this is dependent on the change rate of the catalog data. So it will necessarily be a function of the index or ingest rate of the 1YB archive to be covered in a future post.

Stay tuned, I am just having too much fun to stop.

Sidekick's failure, no backups

Posted on October 12, 2009 by Ray in Storage, Storage Backup, Storage Quality, Storage reliability

Sidekick 2 "Skinit" by grandtlairdjr (cc) (from flickr)

I believe I have covered this ground before but apparently it needs reiterating. Cloud storage without backup cannot be considered a viable solution. Replication only works well if you never delete or logically erase data from a primary copy. Once that’s done the data is also lost in all replica locations soon afterwards.

I am not sure what happened with the sidekick data, whether somehow a finger check deleted it or some other problem but from what I see looking in from the outside – there were no backups, no offline copies, no fall back copies of the data that weren’t part of the central node and it’s network of replicas. When that’s the case disaster is sure to ensue.

At the moment the blame game is going around to find out who is responsible and I hear that some of the data may be being restored. But that’s not the problem. Having no backups that are not part of the original storage infrastructure/environment are the problem. Replicas are never enough. Backups have to be elsewhere to count as backups.

What would have happened if they had backups is that the duration of the outage would have been the length of time it took to retrieve and restore the data and some customer data would have been lost since the last backup but that would have been it. It wouldn’t be the first time backups had to be used and it won’t be the last. But without backups at all, then you have a massive customer data loss that cannot be recovered from.

This is unacceptable. It gives IT a bad name, puts a dark cloud over cloud computing and storage and makes the IT staff of sidekick/danger look bad or worse ~~incompetent~~ naive.

All of you cloud providers need to take heed. You can do better. Backup software/services can be used to backup this data and we will all be better served because of it.

BBC and others now report that most of the Sidekick data will be restored. I am glad that they found a way to recover their “… data loss in the core database and the back up.” and have “… installed a more resilient back-up process” for their customer data.

Some are saying that the backups just weren’t accessible but until the whole story comes out I will withhold judgement. Just glad to have another potential data loss be prevented.

What if there were no backup?

Posted on September 10, 2009 by Ray in Storage, Storage Backup, Strategy, Systems

If backup didn’t exist and you had to start over to protect your data how would you do it today?

I think four things are important to protect data in today’s data center:

Any data ever created in the data center or on-the-road needs to be protected,
Data restores must be under end-user control,
Data needs to be copied/replicated/mirrored offsite to support disaster recovery,
Multiple data copies should exist only to satisfy some data protection policy – one copy is mandatory, two copies (not co-located) would be required to support higher availability, and
Data protection activities should not interfere with or interrupt ongoing data center operations

All this can and is being done with backup and other systems today but most of these products and features grew out of earlier phases of computing. With today’s technology many of these capabilities may no longer be necessary today if one could just rethink data protection from the ground up.

Data Versioning

I think some form of data/file/block easily versioning could easily support the requirement of restoring any data ever created. Versioning systems have existed in the past and could certainly be re-constituted today with some sort of standards. The cost of storing all that data might be a concern but storage costs continue to decrease and if multiple copies retained for data protection can be eliminated, it might just be a wash. Versioning could just as easily be provided for the labtop and once new versions of data are created old versions could be moved off the laptop to the data center for safekeeping and to free up space.

End-user visiblility

End-user restoration requires some facility to explore the end-users data protection file-name and block space. Once this is available, identifying which version needs to be restored and where to restore it should be straightforward. All backup applications provide a backup directory and a few even allow end-user access to perform data restores. While all this works well with files, having an end-user do this for block storage would require more sophistication. Nonetheless, both file and block restores seems entirely feasible once data versioning is in place.

Ubiquitous replication

The requirement to have data copies offsite is certainly feasible today. Replication can be done in hardware or software today, synchronously, semi-synchronously, and/or asynchronously. Replication today can solve this problem but replicating to separate data centers cost too much. Enter the storage cloud. With the storage cloud we could pay just for the data bandwidth and storage to support our data protection needs and no more. Old data versions could be replicated as new versions are created. Protecting data written to a new version is more problematic but some sort of write splitter (ala CDP) could be used to create a replica of this data as well.

Policy driven

Having a policy driven data protection system that only stores a minimal number of copies of data seems to be difficult to support. Yet, this seems to be what incremental-only backup software and archive products support today. For other backup software, if one uses a deduplicating VTL this can be very similar. Adding some policy sophistication to coordinate multiple data protection copies across multiple (potentially Cloud) nodes and deduplicating all the un-necessary copies seems entirely feasible.

Operationally transparent

Not interrupting ongoing operations also seems to be tough to crack. Yet, many storage vendors provide snapshot technologies that copy block and/or file data without interrupting operations. However, coordinating vendor snapshot technologies from some central data protection manager is an essential integration but continues to be lacking.

Can pieceparts solve the problem?

Yes, most of these features are purchasable as separate product offerings (except data versioning) but what’s missing is any one product that pulls all of this together and offers one integrated solution to data protection as I have described it.

The problem, of course, is that such functionality probably best belongs as part of the O/S or a hypervisor but they long ago relinquished any responsibility for data protection. Aside from the anti-trust and non-competitive nature of such a future data protection O/S offering, I only see isolated steps and no coordinated attack on today’s overall data protection problem.

Backup software vendors do a great job with what they have under their control, but they can’t do it all, ditto for VTL providers, CDP vendors, replication products, etc. Piecemeal solutions can only take us so far down this path but it’s all we have today and I fear for the forseeable future.

Dream time over for now, gotta backup some data…