Microsoft MVP Summit 2010 by David McCarter (cc) (From Flickr)
I missed the MVP summit last month in Redmond, but I heard there was some more discussion of the Server Message Block v2.2 (SMB2.2, also known previously as CIFS) coming in Windows Server (R) 8.
As reported last fall at the SNIA Developer Conference SMB2.2 using RDMA over InfiniBand reached over 3.7GB/sec with no server configuration changes using two QDR cards and 160K IOPs (the IOPs are from an SQLIO run using 8KB IOs, not SPECsfs2008). The pre-beta, SMB2.2 code was running on commodity server hardware using 32Gbps InfiniBand links. I couldn’t find any performance numbers with ROCE or iWARP but I would suspect running on 10GbE these would be much slower than InfiniBand.
Hints are that performance gets even better with the released versions of the code coming out in Windows Server 8.
SMB2.2 gets even faster than NFS
We have noted in the past that SMB (CIFS) on average, shows better throughput (IOPS) performance than NFS in SPECsfs2008 results (for example, see our latest Chart-of-the-Month post on SPECsfs results). However, those results were all at best SMB2 or even SMB1 results, and commonly using Ethernet links.
NFS already supports InfiniBand but I am unsure whether it makes use of RDMA. Nevertheless, the significant speed up shown here for SMB2.2 will potentially take SPECsfs2008 SMB2.2 performance up to a whole new level.
Why InfiniBand?
As you may recall, InfinBand is primarily deployed as a server to server interface and used extensively in the past for high performance computing environments. However nowadays, we find storage clusters, such as EMC Isilon, HP X9000 (Ibrix), IBM XIV and others using InfiniBand for their inter-node communications. The use of InfiniBand in these storage clusters is probably due primarily to its superior latency over Ethernet.
But InfiniBand has another advantage, fast data throughput, when using RDMA it can transfer data faster than almost any other networking protocol alive today. SMB2.2 takes advantage of this throughput boost by using RDMA only for large blocks of data and avoiding it for smaller blocks of data. Not sure what the cutoff is, but this would certainly help in large SQL database queries, disk copies, and any other large file data transfer operations.
Of course with 56Gbps FDR InfiniBand available today and faster transfer rates coming (see IBTA roadmap), there appears to be every reason to believe that superior throughput performance will continue at least for the foreseeable future. Better latency is also certain to be retained as well
Now that Intel’s pushing it, Mellanox continuing to push Infiniband and storage cluster’s using it more frequently, we may start to see more storage protocols supporting it.
We thought that FC only had Ethernet to worry about, with SMB2.2 moving to InfiniBand, NFS already supporting it, can a fully functional FCoIB be far behind?
[long post 945 wds] HP held their (annual?) HP Tech Days in Fort Collins, Colorado this last week. We had presentations from a number of HP product managers and got to meet a number of new and old bloggers there.
Craig Nunes VP of Marketing, HP Storage got up and led off the day’s discussion talking about recent results. HP disk storage is up 11% for the quarter, 3par is growing by triple digit growth (QoQ maybe YoY?) and channel sales are growing by 10%. HP storage is gaining market share, grew 3% for the quarter. Also, HP is #2 is shipped backup appliances (1H11). The current focus for HP storage is in three areas:
Invest in established platforms, MSA and EVA (with a 100K customers)
Invest in converged storage aimed at new data centers, 3PAR, Lefthand, IBRIX and StoreOnce.
Invest in converged systems knocking down barriers between servers, storage and networking with Virtual Systems.
Craig spent most of his time talking about converged storage. HP’s converged storage includes:
built in autonomic storage automating operations with one pain of glass and an orchestration layer on top to oversee everything.
scale out storage providing simpler ways to grow storage.
built on standardized platforms using off the shelf server platform technology
Craig ended up discussing HP’s Virtual System, their response to VCE’s Vblock, NetApp’s FlexPod and Dell’s vStart Bundle. HP’s Virtual System was announced earlier last year and has been doing well in the market.
Brad Katz, Product Manager got up next and talked about Lefthand storage solutions. Lefthand’s portfolio now ranges from the Virtual Storage Appliance (VSA) all the way up to a P4800 SAN storage blade with P4300 and P4500 rackmountable storage systems between those two. Lefthand systems provide a clustered, scale-out IP/SAN and NAS storage. Cluster data is striped across all disks in all storage nodes.
The VSA runs as a virtual machine and utilizes any ESX (direct or SAN attached) storage. The P4800 operates as a storage blade in an HP blade server and uses storage in the blade system. The two rackmount systems P4300 and P4500 connect to SAS attached, external disk shelves.
Steve Johnson on StoreOnce
Steve Johnson and Mat Jacoby talked next about the StoreOnce deduplicating backup appliance product line. StoreOnce is an HP R&D Labs home grown, deduplication technology which provides balanced ingest-restore rates and memory efficient deduplication. The current product line spans D2D25xx, D2D41xx, D2D43xx and the recently announced, B6200 backup storage blade.
StoreOnce use a variable block, 4K chunksize and a sparse index which saves on server memory size which both lead to great deduplication rates. Most deduplication functionality is memory intensive making it hard to scale without increasing memory or using different dedupe engines across a product line. StoreOnce’s sparse indexing fixed that issue and as such, can use the same deduplication engine across their entire product line.
JR talking about 3PAR advantages
Jim Richardson or JR, a 3PAR SE from the start, got up and discussed 3PAR. Early on, 3PAR brought to the market three characteristics that differentiated it from other enterprise storage products:
Multi-tennancy – today’s cloud service providers and just about anyone running enterprise storage needs to support mixed workloads on shared storage. 3PAR’s ASIC allows data to be placed on any storage node and be serviced at direct access speeds to better support these multi-application environments.
Thin provisioning – although certainly not the first to support thin provisioning (Iceberg was the first), 3PAR did much to popularize it. Once again the ASIC provides automated support for thin provisioning.
Autonomic functionality – optimization of storage performance across nodes and tiers of storage was also helped by their ASIC’s ability to transfer data without involving processor interaction. Also 3PAR, tried to take the drudgery out of administration by automatically wide striping and making provisioning easier.
Jim Hankins and Chris Duffy came up next and talked about the X9000 IBRIX storage system. Ibrix has intrinsic scale out NAS support and provides automatic failover across dual processing nodes called couplets. The B6200 backup system (see above) is based on Ibrix technology. Ibrix supports a 15PB single name space that is segmented across cluster couplets. Ibrix also comes in a gateway configuration using shared SAN storage behind it.
HP X5000 NAS system
Robert Thompson got up and talked about the X5000 Windows Server WSS based NAS product. It is the industry’s first two node file system with active/active clustering in a box. As the product runs Windows Server, one can run Anti-Virus or other server applications directly on the storage and is customer maintainable. Robert pulled out every replaceable unit in the system. Apparently the E5000, HP Storage’s Exchange Appliance is also based on the same hardware. The two servers in the storage system are clustered together using MSCS.
HPer showing off intelligent floor tiles
In the afternoon we went on a lab tour and got to see some of HP’s storage and data center cooling technology on display.
On the second day, Mike Koponen got up and discussed HP’s Virtual System (or Vblock competitor) and Aboubacar Diare gave some of his opinions on VMware VAAI & VASA integration from his testing perspective. Finally, Calvin Zito wrapped up the two day event and everyone (except me and a few others) went on a brewery tour.
~~~~
All in all, we had a good time with HP. Too bad, I didn’t get to go on the New Belgium Brewery tour, perhaps next time.
Recent research from MIT on a Streamlining Chip Design was in the news today. The report described work was done by Nyrav Dave PhD and Myron King to create a new programming language, BlueSpec that can convert specifications into hardware chip design (Verilog) or compile it into software programming (C++).
BlueSpec designers can tag (annotate) system modules to be hardware or software. The intent of the project is to make it easier to decide what is done in hardware versus software. By specifying this decision using a language attribute, it should make architectural hardware-software tradeoffs much easier to do and as a result, delay that decision until much later in the development cycle.
Hardware-software tradeoffs
Making good hardware-software tradeoffs are especially important in mobile handsets where power efficiency and system performance requirements often clash. It’s not that unusual in these systems that functionality is changed from hardware to software implementations or vice versa.
The problem is that the two different implementations (hardware or software) use different design languages and would typically require a complete re-coding effort to change, delaying system deployment significantly. Which makes such decisions all the more important to get right early on in system architecture.
In contrast, with BlueSpec, all it would take is a different tag to have the language translate the module into Verilog (chip design language) or C++ (software code).
Better systems through easier hardware design
There is a long running debate around commodity hardware versus special purpose hardware designed systems in storage systems (see Commodity Hardware Always Loses and Commodity Hardware Debate Heats-up Again). We believe that there will continuing place for special purpose built hardware in storage. Also, I would go on to say this is likely the case in networking, server systems as well as telecommunications handsets/back-office equipment.
The team at MIT specifically created their language to help create more efficient mobile phone hand sets. But from my perspective it has an equally valid part to play in storage and other systems.
Hardware and software design, more similar than different
Nowadays, hardware and software designers are all just coders using different languages.
Yes hardware engineers have more design constraints and have to deal with the real, physical world of electronics. But what they deal with most, is a hardware design language and design verification tools tailored for their electronic design environment.
Doing hardware design is not that much different from software developers coding in a specific language like C++ or Java. Software coders must also be able to understand their framework/virtual machine/OS environment their code operates in to produce something that works. Perhaps, design verification tools don’t work or even exist in software as much as they should but that is more a subject for research than a distinction between the two types of designers.
—-
Whether BlueSpec is the final answer or not isn’t as interesting as the fact that it has taken a first step to unify system design. Being able to decide much later in the process whether to make a module hardware or software will benefit all system designers and should get products out with less delay. But getting hardware designers and software coders talking more, using the same language to express their designs can’t help but result in better/tighter integrated designs which end up benefiting the world.
Toyota Hybrid Synergy Drive Decal: RAC Future Car Challenge by Dominic's pics (cc) (from Flickr)
I saw where Seagate announced the next generation of their Momentus XT Hybrid (SSD & Disk) drive this week. We haven’t discussed Hybrid drives much on this blog but it has become a viable product family.
I am not planning on describing the new drive specs here as there was an excellent review by Greg Schulz at StorageIOblog.
However, the question some in the storage industry have had is can Hybrid drives supplant data center storage. I believe the answer to that is no and I will tell you why.
Hybrid drive secrets
The secret to Seagate’s Hybrid drive lies in its FAST technology. It provides a sort of automated disk caching that moves frequently accessed OS or boot data to NAND/SSD providing quicker access times.
Storage subsystem caching logic has been around in storage subsystems for decade’s now, ever since the IBM 3880 Mod 11&13 storage control systems came out last century. However, these algorithms have gotten much more sophisticated over time and today can make a significant difference in storage system performance. This can be easily witnessed by the wide variance in storage system performance on a per disk drive basis (e.g., see my post on Latest SPC-2 results – chart of the month).
Enterprise storage use of Hybrid drives?
The problem with using Hybrid drives in enterprise storage is that caching algorithms are based on some predictability of access/reference patterns. When you have a Hybrid drive directly connected to a server or a PC it can view a significant portion of server IO (at least to the boot/OS volume) but more importantly, that boot/OS data is statically allocated, i.e., doesn’t move around all that much. This means that one PC session looks pretty much like the next PC session and as such, the hybrid drive can learn an awful lot about the next IO session just by remembering the last one.
However, enterprise storage IO changes significantly from one storage session (day?) to another. Not only are the end-user generated database transactions moving around the data, but the data itself is much more dynamically allocated, i.e., moves around a lot.
Backend data movement is especially true for automated storage tiering used in subsystems that contain both SSDs and disk drives. But it’s also true in systems that map data placement using log structured file systems. NetApp Write Anywhere File Layout (WAFL) being a prominent user of this approach but other storage systems do this as well.
In addition, any fixed, permanent mapping of a user data block to a physical disk location is becoming less useful over time as advanced storage features make dynamic or virtualized mapping a necessity. Just consider snapshots based on copy-on-write technology, all it takes is a write to have a snapshot block be moved to a different location.
Nonetheless, the main problem is that all the smarts about what is happening to data on backend storage primarily lies at the controller level not at the drive level. This not only applies to data mapping but also end-user/application data access, as cache hits are never even seen by a drive. As such, Hybrid drives alone don’t make much sense in enterprise storage.
Maybe, if they were intricately tied to the subsystem
I guess one way this could all work better is if the Hybrid drive caching logic were somehow controlled by the storage subsystem. In this way, the controller could provide hints as to which disk blocks to move into NAND. Perhaps this is a way to distribute storage tiering activity to the backend devices, without the subsystem having to do any of the heavy lifting, i.e., the hybrid drives would do all the data movement under the guidance of the controller.
I don’t think this likely because it would take industry standardization to define any new “hint” commands and they would be specific to Hybrid drives. Barring standards, it’s an interface between one storage vendor and one drive vendor. Probably ok if you made both storage subsystem and hybrid drives but there aren’t any vendor’s left that does both drives and the storage controllers.
~~~~
So, given the state of enterprise storage today and its continuing proclivity to move data around accross its backend storage, I believe Hybrid drives won’t be used in enterprise storage anytime soon.
HDS CEO Jack Domme shares the company’s vision and strategy with Influencer Summit attendees #HDSday by HDScorp
Attended #HDSday yesterday in San Jose. Listened to what seemed like the majority of the executive team. The festivities were MCed by Asim Zaheer, VP Corp and Product Marketing, a long time friend and employee, that came to HDS with the acquisition of Archivas five or so years ago. Some highlights of the day’s sessions are included below.
The first presenter was Jack Domme, HDS CEO, and his message was that there is a new, more aggressive HDS, focused on executing and growing the business.
Jack said there will be almost a half a ZB by 2015 and ~80% of that will be unstructured data. HDS firmly believes that much of this growing body of data today lives in silos, locked into application environments and can’t become truly information until it can be liberated from this box. Getting information out of the unstructured data is one of the key problems facing the IT industry.
To that end, Jack talked about the three clouds appearing on the horizon:
infrastructure cloud – cloud as we know and love it today where infrastructure services can be paid for on a per use basis, where data and applications move seemlessly across various infrastructural boundaries.
content cloud – this is somewhat new but here we take on the governance, analytics and management of the millions to billions pieces of content using the infrastructure cloud as a basic service.
information cloud – the end game, where any and all data streams can be analyzed in real time to provide information and insight to the business.
Jack mentioned the example of when Japan had their earthquake earlier this year they automatically stopped all the trains operating in the country to prevent further injury and accidents, until they could assess the extent of track damage. Now this was a specialized example in a narrow vertical but the idea is that the information cloud does that sort of real-time analysis of data streaming in all the time.
For much of the rest of the day the executive team filled out the details that surrounded Jack’s talk.
For example Randy DeMont, Executive VP & GM Global Sales, Services and Support talked about the new, more focused sales team. On that has moved to concentrate on better opportunities and expanded to take on new verticals/new emerging markets.
Then Brian Householder, SVP WW Marketing and Business Development got up and talked about some of the key drivers to their growth:
Current economic climate has everyone doing more with less. Hitachi VSP and storage virtualization is a unique position to be able to obtain more value out of current assets, not a rip and replace strategy. With VSP one layers better management on top of your current infrastructure, that helps get more done with the same equipment.
Focus on the channel and verticals are starting to pay off. More than 50% of HDS revenues now come from indirect channels. Also, healthcare and life sciences are starting to emerge as a crucial vertical for HDS.
Scaleability of their storage solutions is significant. Used to be a PB was a good sized data center but these days we are starting to talk about multiple PBs and even much more. I think earlier Jack mentioned that in the next couple of years HDS will see their first 1EB customer.
MarkMike Gustafson, SVP & GM NAS (former CEO BlueArc) got up and talked about the long and significant partnership between the two companies regarding their HNAS product. He mentioned that ~30% of BlueArc’s revenue came from HDS. He also talked about some of the verticals that BlueArc had done well in such as eDiscovery and Media and Entertainment. Now these verticals will become new focus areas for HDS storage as well.
John Mansfield, SVP Global Solutions Strategy and Developmentcame up and talked about the successes they have had in the product arena. Apparently they have over 2000 VSPs intsalled, (announced just a year ago), and over 50% of the new systems are going in with virtualization. When asked later what has led to the acceleration in virtualization adoption, the consensus view was that server virtualization and in general, doing more with less (storage efficiency) were driving increased use of this capability.
Hicham Abdessamad, SVP, Global Services got up and talked about what has been happening in the services end of the business. Apparently there has been a serious shift in HDS services revenue stream from break fix over to professional services (PS). Such service offerings now include taking over customer data center infrastructure and leasing it back to the customer at a monthly fee. Hicham re-iterated that ~68% of all IT initiatives fail, while 44% of those that succeed are completed over time and/or over budget. HDS is providing professional services to help turn this around. His main problem is finding experienced personnel to help deliver these services.
After this there was a Q&A panel of John Mansfield’s team, Roberto Bassilio, VP Storage Platforms and Product Management, Sean Moser, VP Software Products, and Scan Putegnat, VP File and Content Services, CME. There were a number of questions one of which was on the floods in Thailand and their impact on HDS’s business.
Apparently, the flood problems are causing supply disruptions in the consumer end of the drive market and are not having serious repercussions for their enterprise customers. But they did mention that they were nudging customers to purchase the right form factor (LFF?) disk drives while the supply problems work themselves out.
Also, there was some indication that HDS would be going after more SSD and/or NAND flash capabilities similar to other major vendors in their space. But there was no clarification of when or exactly what they would be doing.
After lunch the GMs of all the Geographic regions around the globe got up and talked about how they were doing in their particular arena.
Jeff Henry, SVP &GM Americas talked about their success in the F500 and some of the emerging markets in Latin America. In fact, they have been so successful in Brazil, they had to split the country into two regions.
Niels Svenningsen, SVP&GM EMAE talked about the emerging markets in his area of the globe, primarily eastern Europe, Russia and Africa. He mentioned that many believe Africa will be the next area to take off like Asia did in the last couple of decades of last century. Apparently there are a Billion people in Africa today.
Kevin Eggleston, SVP&GM APAC, talked about the high rate of server and storage virtualization, the explosive growth and heavy adoption of Cloud pay as you go services. His major growth areas were India and China.
The rest of the afternoon was NDA presentations on future roadmap items.
—-
All in all a good overview of HDS’s business over the past couple of quarters and their vision for tomorrow. It was a long day and there was probably more than I could absorb in the time we had together.
T-Finity library at SpectraLogic's test facility (c) 2011 Silverton Consulting, All Rights Reserved
Today I met with Spectra Logic execs and some of their Media and Entertainment (M&E) customers, and toured their manufacturing, test labs and briefing center. The tour was a blast and the customers Kyle Knack from National Geographic (Nat Geo) Global Media, Toni Perez from Medcom (Panama based entertainment company) and Lee Coleman from Entertainment Tonight (ET) all talked about their use of the T-950 Spectra Logic tape libraries in the media ingest, editing and production processes.
Mr. Collins from ET spoke almost reverently about their T-950 and how it has enabled ET to access over 30 years of video interviews, movie segments and other media they can now use to put together clips on just about any entertainment subject imaginable.
He talked specifically about the obit they did for Michael Jackson and how they were able to grab footage from an interview they did years ago and splice it together with more recent media to show a more complete story. He also showed a piece on some early Eddie Murphy film footage and interviews they had done at the time which they used in a recent segment about his new movie.
All this was made possible by moving to digital file formats and placing digital media in their T-950 tape libraries.
Spectra Logic T-950 (I think) with TeraPack loaded in robot (c) 2011 Silverton Consulting, All Rights Reserved
Mr. Knack from Nat Geo Media said every bit of media they get anymore, automatically goes into the library archive and becomes the “original copy” of the media used in case other copies are corrupted or lost. Nat Geo started out only putting important media in the library but found it just cost so much less to just store it in the tape archive that they decided it made more sense to just move all media to the tape library.
Typically they keep two copies in their tape library and important media is also copied to tape and shipped offsite (3 copies for this data). They have a 4-frame T-950 with around 4000 slots and 14 drives (combination of LTO-4 and -5). They use FC and FCoE storage for their primary storage and depend on 1000s of SATA drives for primary storage access.
He said they only use SSDs for some metadata support for their web site. He found that SATA drives can handle their big block sequential and provide consistent throughput and especially important to M&E companies consistent latency.
3D printer at Spectra Logic (for mechanical parts fabrication) (c) 2011 Silverton Consulting, All Rights Reserved
Mr. Perez from MedCom had much the same story. They were in the process of moving off of proprietary video tape format (Sony Betacam) to LTO media and digital files. The process is still ongoing although they are more than halfway there for current production.
They still have a lot of old media in Betacam format which will take them years to convert to digital files but they are at least starting this activity. He said a recent move from one site to another revealed that much of the Betacam tapes were no longer readable. Digital files on LTO tape should solve that problem for them when they finally get there.
Matt Starr Spectra Logic CTO talked about the history of tape libraries at Spectra Logic which was founded in 1998 and has been laser focused on tape data protection and tape libraries.
I find it pleasantly surprising that a company today can just supply tape libraries with software and make a ongoing concern of it. Spectra Logic must be doing something right, revenue grew 30% YoY last year and they are outgrowing their current (88K sq ft) office, lab, and manufacturing building they just moved into earlier this year and have just signed to occupy another building providing 55K sq ft of more space.
T-Series robot returning TeraPack to shelf (c) 2011 Silverton Consulting, All Rights Reserved
Molly Rector Spectra Logic CMO talked about the shift in the market from peta-scale (10**15 bytes) storage repositories to exa-scale (10**18 bytes) ones. Ms. Rector believed that today’s cloud storage environments can take advantage of these large tape based, archives to provide much more economical storage for their users without suffering any performance penalty.
Matt said that there are about a dozen large HPSS sites (HPSS website shows at least 30 sites using it) that store a significant portion of the worlds 1ZB (10**21 bytes) of digital data created this past year (see my 3.3 exabytes of data a day!? post). Later that day talking with Nathan Thompson Spectra Logic CEO, he said these large HPSS sites probably store ~10% of the worlds data, or 100EB. I find that difficult to comprehend that much data at only ~12 sites but the national labs do have lots of data on hand.
Nowadays you can get a Spectra Logic T-Finity tape complex with 122K slot, using LTO-4/-5 or IBM TS1140 (enterprise class) tape drives. This large a T-Finity has 4 rows of tape libraries which uses the ‘Skyway’ to transport a terapack of tape cartridges between one library row to the another. All Spectra Logic libraries are built around a tape cartridge package they call the TeraPack which contains 10 LTO cartridges or (I think) 9-TS1140 tape cartridges (they are bigger than LTO tapes). The TeraPack is used to import or export tapes from the library and all the tape slots in the library.
The software used to control all this is called BlueScale and is used in their T50e, a small, 50 slot library all the way up to the 122K T-Finity tape complex. There are some changes for configuration, robotics and other personalization for each library type but the UI looks exactly the same across any of their libraries. Moreover, BlueScale offers the same enterprise level of functionality (e.g., drive and media life management) services for all Spectra Logic tape libraries.
Day 1 for SpectraPRDay closed with the lab tour and dinner. Day 2 will start discussing futures and will be under NDA so there won’t be much to talk about right away. But from what I can see, Spectra Logic seems to be breaking down the barriers inhibiting tape use and providing tape library systems, that people almost revere.
I haven’t seen that sort of reaction about a tape library since the STK 4400 first came out last century.
Functional Fusion? by Cain Novocaine (cc) (from Flickr)
One can only be perplexed by the seemingly overwelming adoption of server virtualization and contrast that with the ho-hum, almost underwelming adoption of storage virtualization. Why is there this significant a difference?
I think the problem is partly due to the lack of an common understanding of storage performance utilization.
Why server virtualization succeeded
One significant driver of server virtualization is the precipitous drop in server utilization that occurred over the last decade when running single applications on a physical server. It was nothing to see real processor utilization of less than 10% and consequently it was easy to envision that executing 5-10 applications on the single server. And what’s more each new generation of server kept getting more powerful, handling double the MIPs every 18 months or so driven by Moore’s law.
The other factor was that application workloads weren’t increasing that much. Yes new applications would come online but they seldom consumed an inordinate amount of MIPs and were often similar to what was already present. So application processing growth while not flatlining, was expanding at a relatively slow speed.
Why storage virtualization has failed
Data on the other hand continues its never ending exponential growth. Doubling every 3-5 years or less. And the fact that you have more data, almost always requires more storage hardware to support the IOPs being required to support it.
In the past the storage IOP rates was intrinsically tied to the number of disk heads available to service the load. Although disk performance grew it wasn’t doubling every 18 months, and real per disk performance was actually going down over time, measured as the amount of IOPS per GB.
This drove proliferation of disk spindles and as such, storage subsystems in the data center. Storage virtualization couldn’t reduce the number of spindles required to support the workload.
Thus, if you look at storage performance from the perspective of % IOPS one could support per disk, most sophisticated systems were running anywhere from 75% to 150% (based on DRAM caching).
Paradigm shift ahead
But SSDs can change this dynamic considerably. A typical SSD can sustain 10-100K IOPs and there is some liklihood that this will increase with each generation that comes out but the application requirements will not increase as fast. Hence, , there is a high liklihood that normal data center utilisation of SSD storage perfomance will start to drop below 50% or more, when that happens. -torage virtualization may start to make a lot more sense.
Maybe when (SSD) data storage starts moving more in line with Moore’s law, storage virtualization will become a more dominant paradigm for data center storage use.
—
Any bets on who the VMware of storage virtualization will be?
A head assembly on a Seagate disk drive by Robert Scoble (cc) (from flickr)
An interesting guest post on Claus’s Blog (Claus Mikkelsen of HDS) by Ian Vogelesang of HGSTprovided some technical/economic insights on why specific disk drives are more economically feasible than others.
It’s a bit hard going and more technical than a typical blog post, but it certainly makes a number of interesting points.
There is an interaction between recording density, performance and $/GB when introducing a new, smaller form factor. Most often drive vendors are trying to maximize GB per drive while IO performance is not as much of a concern. So they most often try to first come out with their densest drive they can in any new form factor. I think this is what we have seen with the SFF disks today, i.e., most vendors came out with 10Krpm drives, leaving their faster drives to the LFF. As recording density for a new technology continues to improve, GB/drive is no longer the driving factor which is when performance rises to the top. At that point then we see the introduction of higher speed drives in a form factor.
Enterprise SATA drives perform worse than equivalent capacity SAS drives. In HDS’s case there are two reasons for this: 1) For enterprise storage they append ECC plus other LBA integrity checks to each 512 byte block however, SATA doesn’t support anything but 2**n block size, thus multiple IOs are required to read/validate a block and 2) SAS hardware supports a larger tag command queue than SATA and thus, a better optimized IO queue for multiple IO requests.
Global access density requirements are 600IOPs/TB of storage. This is stated as a matter of fact in the post without any background information but is another key factor driving disk changes.
I would love to know more about that last point 600 IOPS/TB. But there wasn’t much else there. (It seem to me this should have changed over time. It’s certainly worthy of a research study if anybody’s listening out there.)
Shingled writes
One other thing I found interesting is a few statements at the end regarding emerging disk recording technology. It seems thermally assisted recording (TAR) is not coming along as fast as everyone in the industry thought it would. As such, the disk industry is considering moving to shingled writes (see my post Sequential Only Disk) which may cause them to abandon random writes.
But there is another solution to non-random writes besides sequential only disk and that is implementing a log structured file for blocks on the disk. Similar to NetApp’s Data ONTAP, where the system supports random writes but actually writes data on disk drives sequentially.
This requires more smarts in the drive controller but it’s nothing like what’s in SSDs today for wear leveling and is a viable alternative. The nice thing about a log structured file on disk, is that there is no need to change any IO drivers as the disk drive continues to support random writes (from the server/storage system perspective) but the drives write sequential on the platter.
I would suspect most drive vendors considering shingled writes are busily working on doing something similar to this and it wouldn’t surprise me to see the next generation disks support shingled writes using an onboard log structured file.
What this will do for read sequential IO is another question entirely.
Luckily, data that is read sequentially is often written sequentially and even with a log structured file layout on disk, will more than likely be positioned close together on a disk platter.
