Project Natick’s Phase 2 underwater data center had 12 racks with 864 servers and 27.5PB of disk storage and was connected to the nearby Orkney island’s power grid (250Kw) and networking infrastructure. The Orkney’s islands are located off the NE coast of the Scotland and its power grid is 100% renewable, using tidal, solar and wind power. During the data center test, Orkney was able was able to power the data center, the islands and still provide power back to the Scottish power grid.
More reliable underwater
According to early reports, the servers in the underwater data center had 1/8th the failures that a control data center, on land, had. Microsoft attributes the enhanced server reliability to the use of a 100% Nitrogen (at 1 atmosphere pressure) rather than normal air and the lack of any humans to jostle the equipment/disturb the environment.
It’s also likely that the temperature variability present in a normal, on the surface of the earth, data center was measurably less than for a data center on the sea floor. If this were true, that could also help explain its better reliability.
Why underwater?
It’s all about cooling modern servers (and storage). According to NREL ( USA National Renewable Energy Lab), most data centers operate at 1.8 PUE (power use efficiency) that is, using 180% of the power required for the servers, storage and networking equipment. The other 80% is used mainly for cooling electronics, but also includes lighting, HVAC, and other essential services for humans. NREL says that high efficiency data centers can achieve a PUE of 1.2.
PUE for Project Natick Phase 2 data center was reported to be 1.07. The only additional electricity needed would probably be power for cooling.
Cooling for the Project Natick Phase 2 data center used seawater pumped through the back of server racks. The data center was placed on the seafloor at 35m (117ft) deep.
It kind looked like a submarine. According to Microsoft, the data center was contracted for, built and deployed in under 90 days. The intent was to have the data center be smaller than a standard ISO shipping container. The data center was driven ontop of an 18 wheeler, from where it was built to the Orkney Island, including ferry crossings. It was placed on a triangular support, towed out to see and deposited on the seafloor.
While 864 servers and 27.5PB of storage seem like a lot to most of us, for Microsoft Azure it’s too small to be used as a regional zone. But for (large) edge deployments. something this size or (10X) smaller might be just the thing.
Microsoft notes that 1/2 the world’s population lives within 200km (120mi) of the ocean. So there’s a ready supply of people and businesses that could take advantage of any underwater data center.
And of course, such a structure when laid on the bottom of the ocean floor, could create an artificial reef (if left in place long enough). Artificial reefs have been made out of ocean oil rigs, sunken war ships and large chunks of steel/concrete. So a underwater data center could do so just as well. And maybe the heating coming from the data center cooling pumps would foster even more coral life.
Microsoft plans Project Natick Phase 3 to be a full Azure AZ that will be deployed underwater which will include about 12 Phase 2 datacenter pressurized units.
Read an article this past week in IEEE Spectrum (The Software Defined Power Grid is here) about a company that has been implementing software defined power grids throughout USA and the world to better integrate and utilize renewable energy alongside conventional power generation equipment.
Moreover, within the last year or so, Tesla has installed a Virtual Power Plant (VPP) using residential solar and grid scale batteries to better manage the electrical grid of South Australia (see Tesla’s Australian VPP propped up grid during coal outage). VPP use to offset power outages would necessitate something like a software defined power grid.
Software defined power grid
Not sure if there’s a real definition somewhere but from our perspective, a software defined power grid is one where power generation and control is all done through the use of programatic automation. The human operator still exists to monitor and override when something goes wrong but they are not involved in the moment to moment control of which power is saved vs. fed into the grid.
About a decade ago, we wrote a post about smart power meters (Smart metering’s data storage appetite) discussing the implementation of smart meters for home owners that had some capabilities to help monitor and control power use. But although that technology still exists, the software defined power grid has moved on.
The IEEE Spectrum article talks about a phasor measurement units (PMUs) that are already installed throughout most power grids. It turns out that most PMUs are capable of transmitting phasor power status at 60 times a second granularity and each status report is time stamped with high accuracy, GPS synchronized time.
On the other hand, most power grids today use SCADAs (supervisory control and data acquisition) to monitor and manage the power grid. But SCADAs only send data every 2-4 seconds. PMU’s are also installed in most power grids, but their information is not as important as SCADA to the monitoring, management and control of most (non-software defined) power grids.
One software defined power grid
PXiSE, the company in the IEEE Spectrum article, implemented their first demonstration project in Hawaii. That power grid had reached the limit of wind and solar power that it could support with human management. The company took their time and implemented a digital simulation of the power grid. But with the simulation in hand, battery storage and a off the shelf PC, the company was able to manage the grids power generation mix in real time with complete automation.
After that success, the company next turned to a micro-grid (building level power) with electronic vehicles, battery and solar power. Their software defined power grid reduced peak electricity demand within the building, saving significant money. With that success the company took their software defined power grid on the road to South Korea, Chile, Mexico and a number of other locations the world.
Tesla’s VPP
The Tesla VPP in South Australia, is planned to consists of up to 50K houses with solar PV panels and 13.5Kwh of batteries, able to deliver up to 250Mw of power generation and 650Mwh of power storage.
At the present time, the system has ~1000 house systems installed but even with that limited generation and storage capability it has already been called upon at least twice to compensate for coal generation power outage. To manage each and every household, they’d need something akin to the smart meters mentioned above in conjunction with a plethora of PMUs.
Puerto Rico’s power grid problems and solutions
There was an article not so long ago about the disruption to Puerto Rico’s power grid caused by Hurricanes Irma and Maria in IEEE Spectrum (Rebuilding Puerto Rico’s Power Grid: The Inside Story) and a subsequent article on making Puerto Rico’s power grid more resilient to hurricanes and other natural disasters (How to harden Puerto Rico’s power grid). The later article talked about creating micro grids, community PV and battery storage that could be disconnected from the main grid in times of disaster but also used to distribute power generation throughout the island.
Although the researchers didn’t call for the software defined power grid, it is our understanding that something similar would be an outstanding addition to their work there.
~~~~
As the use of renewables goes up and the price of batteries decreases while their capabilities go up over time, more and more power grids will need to become software defined. In the end, more software defined power grids with increasing renewables power generation and storage will make any power grid, more resilient and more fault tolerant.
Read an article the other day in TechReview (How Google Docs became the social media of the resistance) about how Google Docs was being used to help coordinate and promote the resistance surrounding the recent Black Lives Matter movement.
Protests could be the killer app for shared Google Docs. Facebook and other social media sites are better used for documenting the real time interactions during protests, but coordinating, motivating and informing the protests and protestors is better accomplished using Google Docs, a simple web based, document editor and sharing service.
In pre-internet days, I suppose all this would have been done on hand copied, typeset printed, carbon copied or photocopied theses/phamplets/fliers/printouts. For example, Luther’s list of grievances nailed to the cathedral door, Common Sense pamphlet during the USA revolutionary war to countless fliers during the 60’s protests, all these used the technology of the day to promote protest and revolution.
Nowadays all it takes is a shared Google Doc and a Google (drive) account.
Google Docs are everywhere
The high school that one of my kids went to uses Google Docs for sharing and submitting homework assignments.
Google Docs are shareable because they are hosted on Google Drives. Docs is just one component of the Google (G-)suite of web based apps that includes Google Sheets (spreadsheets), Google Slides (presentations) and Google Drives (object storage).
Moreover, any Google Doc, Sheet or Slide file can be shared and edited by anyone. And Google services like Docs, Sheets, and Slides are useable anonymously, Anyone onlin, can make a change to a shareable/editable doc, sheet, or slide and their changes are automatically saved to the google drive file.
Another thing is that any Google Doc can be shared with just a URL. And they can also be made read-only (or uneditable) by their owner at any time. And of course any Google Doc is backed up automatically by Google drive services.
Owners of documents can revert to previous versions of a Doc file. So if someone incorrectly (or maliciously) changes a doc, the originator can revert it back to a prior version.
Why not use a Wiki
I would think a Wiki would be better to use to coordinate, motivate and inform a protest. Once a Wiki is setup and started, it can be much easier to navigate, as easy to update, and can become a central repository of all information about a movement/protest.
But it takes a lot more effort and IT-web knowledge to set up a Wiki. And it has to have it’s own web address.
Another problem with a Wiki, is that it can become a central point which can be more easily attacked or disturbed. And Wiki edit wars are pretty common, so they too are not immune to malicious behavior.
But with 10s to 100s of Google Docs, spread across user a similar number of user Google drives, Google Docs are a much more distributed resource, less prone to single point of attack. And they can be created and edited almost on a whim. And the only thing it takes is a Google log in and Google drive.
~~~~
Photo copiers were a controlled technology in the old Soviet Union and even today facebook and twitter are restricted in China and other authoritarian states.
But Google Doc’s seems to have become a much more ubiquitous tool and have become the latest technology, to aid, abet and support social resistance.
Read an article the other day in PNAS (A scaleable pipeline for designing reconfigurable organisms) which described an approach to designing and constructing living organisms to perform real world actions. One could call these living machines or biological robots (biobots). There’s an appendix to the paper which providessupplementary information.
The intention of the pipeline is to expand the modern design space from construction materials, chemical process, electronics and mechanical devices to the domain of living things. Thereby create objects that perform functions for mankind, that are operate well with living things, are more resilient and have a benign impact on the environment.
The Biobot design pipeline stage 1
The design process begins using an evolutionary algorithm which takes as input an organism goal or action (i.e., moving so many body lengths for minutes) and the cell types to be used in constructing the organism and randomly generates potential organism designs.
In the current process there are two cell types (red and cyan) one is passive (scaffolding) and the other is active and provides movement power.
Designing and manufacturing reconfigurable organisms. A behavioral goal (e.g., maximize displacement), along with structural building blocks [here, contractile (red) and passive (cyan) voxels], are supplied to an evolutionary algorithm. The algorithm evolves an initially random population and returns the best design that was found.
Once a set of randomized designs using the two cell types have been determined, each undergoes a computerized simulation (in a physics engine that simulates gravity and liquid environment) to see how well the possible organism perform.
All designs are ranked in how well the achieve they performe and the best of these are used as seeds for another round of evolutionary design exploration. This uses these good designs and randomly changes some aspect of them to create another set of organism designs to test out.
Designing reconfigurable organisms. For a given goal, 100 independent evolutionary trials were conducted in silico (A–C). Each colored line represents the velocity of the fastest-moving design within its clade. Each genome (D) dictates anatomy and behavior by determining where and how voxels are combined, and whether they are passive (cyan) or contractile (red; E).
At some point, the evolutionary design exploration-simulation process stops when it has determined a set of workable organism designs which can achieve the goals set out for them.
The workable organism designs are then subjected to two rounds of filtering. The first filter is tests the designs for resilience to noise. This is done by putting the designs through another set of computer simulations that include noise. Some of the workable organism designs will still perform well in noisy environment and others will not.
The organism designs that perform well in noisy environments, are deemed resilient and are then fed into the next filtering stage of the pipeline.
The resilient workable organism designs are then filtered by whether they can be constructed with the current processes. Even though all the organisms are made up of the two cell types, not all of them can be realized given the current process.
After this point we have a set of designs that
a) Achieve the requested goal in simulation;
b) Perform well in noisy environment simulations; and
c) Can be constructed with the current processes and cell types.
The Biobot design pipeline stage 2
The next steps in the organism design pipeline all take place in the real world. The set of selected designs are constructed/manufactured and set into a petri dish to see how well they perform in real life .
The two cell types used in the current process are derived from the Xenobus (frog) embryos and consist of stem cells (passive) and heart muscle cells (active). The building of organism designs is done through layering of stem cells and then surgically or using cauterization to remove cells not part of the design. After the stem cells are placed then heart muscle cells can be layered on in a similar fashion.
There’s no control mechanism whatsoever other than the surfaces designed for the organism. Xenobus heart muscle cells automatically contract and when combined with other heart muscle cells, all the muscle cells contract in waves.
The design of the organism is such that the contractions propel the organism to move and explore the environment (the intended goal). The designed organisms are placed in a Petri dish and then observed over a period of time to see how well the perform the desired action.
Successful designs can then be seeded back into the start of the evolutionary exploration to generate even better designs. Simulations can also be adjusted with feedback from the real world behavior of the designed organisms. At some point the best designed organisms can be used in the real world.
Why biobots
Although the example had a goal to explore its environment. other goals could be readily used as well. Some of the ones mentioned in the paper are manipulating and gathering together some compounds/elements/particles in a volume. These could be used to clear a viscous solution of some impurities.
Another organism could be designed to have a pouch within which they can store and transport objects (or drugs).
Designed organisms could operate together in a solution with some organisms performing one function while others perform other functions. Organism designs could eventually be combined into one organism that performs more functions.
One nice aspect of biobots is that they can be squeezed, perturbed in many ways including being cut and they repair themselves and continue to operate.
One could design an organism to reproduce in a suitable environment or even designed to age and die after a specific time period.
The end goal seems to be to create living machines that can be used to operate in the environment or a body. Biobots could be designed to clear away plaque from a blood vessels or to dismantle malignant tumors. They could conceivably be constructed from a person’s own cells to operate for days-weeks-months in a body and then dissolve to be reused/disposed of just like any other biological material in a human.
So now we can design biobots.
~~~~
Just in case you wanted to try your hand at designing living organisms yourself. The researchers have all open sourced thei code for the evolutionary design exploration, computerized simulation, noise and build ability filtering which is available ongithub. The actual manufacturing/construction of the designed organism would need to be done in a lab.
Read an article a couple of weeks back (An internet of tires?… IEEE Spectrum) and can’t seem to get it out of my head. Pirelli, a European tire manufacturer was demonstrating a smart tire or as they call it, their new Cyber Tyre.
The Cyber Tyre includes accelerometer(s) in its rubber, that can be used to sense the pavement/road surface conditions. Cyber Tyre can communicate surface conditions to the car and using the car’s 5G, to other cars (of same make) to tell them of problems with surface adhesion (hydroplaning, ice, other traction issues).
Presumably the accelerometers in the Cyber Tyre measure acceleration changes of individual tires as they rotate. Any rapid acceleration change, could potentially be used to determine whether the car has lost traction due and why.
They tested the new tires out at a (1/3rd mile) test track on top of a Fiat factory, using Audi A8 automobiles and 5G. Unclear why this had to wait for 5G but it’s possible that using 5G, the Cyber Tyre and the car could possibly log and transmit such information back to the manufacturer of the car or tire.
Accelerometers have become dirt cheap over the last decade as smart phones have taken off. So, it was only a matter of time before they found use in new and interesting applications and the Cyber Tyre is just the latest.
Internet of Vehicles
Presumably the car, with Cyber Tyres on it, communicates road hazard information to other cars using 5G and vehicle to vehicle (V2V) communication protocols or perhaps to municipal or state authorities. This way highway signage could display hazardous conditions ahead.
Audi has a website devoted to Car to X communications which has embedded certain Audi vehicles (A4, A5 & Q7), with cellular communications, cameras and other sensors used to identify (recognize) signage, hazards, and other information and communicate this data to other Audi vehicles. This way owning an Audi, would plug you into this information flow.
Pirelli’s Cyber Car Concept
Prior to the Cyber Tyre, Pirelli introduced a Cyber Car concept that is supposedly rolling out this year. This version has tyres with real time pressure, temperature, (static) vertical load and a Tyre ID. Pirelli has been working with car manufacturers to roll out Cyber Car functionality.
The Tyre ID seems to be a file that can include anything that the tyre or automobile manufacturer wants. It sort of reminds me of a blockchain data blocks that could be used to validate tyre manufacturing provenance.
The vertical load sensor seems more important to car and tire manufacturers than consumers. But for electrical car owners, knowing car weight could help determine current battery load and thereby more precisely know how much charge is left in a battery.
Pirelli uses a proprietary algorithm to determine tread wear. This makes use of the other tyre sensors to predict wear and perhaps uses an AI DL algorithm to do this.
~~~
ABS has been around for decades now and tire pressure sensors for over 10 years or so. My latest car has enough sensors to pretty much drive itself on the highway but not quite park itself as of yet. So it was only a matter of time before something like smart tires would show up.
But given their integration with car electronics systems, it would seem that this would only make sense for new cars that included a full set of Cyber Tyres. That is until all tire AND car manufacturers agreed to come up with a standard protocol to communicate such information. When that happens, consumers could chose any tire manufacturer and obtain have similar if not the same functionality from them.
I suppose someone had to be first to identify just what could be done with the electronics available today. Pirelli just happens to be it for now in the tire industry.
I just don’t want to have to upgrade tires every 24 months. And, if I have to wait a long time for my car to boot up and establish communications with my tires, I may just take a (dumb) bike.
1) Metal alloys – because of micro-gravity, the mixture of metals that go into metal alloys should be much more even and as a result, should create a purer mixture of the metal alloy at the end of the process.
2) Fibre optical cables – the article says, ZBLAN, which is a heavy-metal fluoride glass fibre could have 1/10th the signal loss of current cable but is hard to manufacture on earth due to micro-crystal formation. Apparently, when manufactured (mixed-drawn) in micro-gravity, there’s less of this defect in the glass.
3) Printed human organs – the problem with printing biological organs, hearts, lungs, livers, etc. is they require scaffolding for the cells to adhere to that needs to be bio-degradeable and in the form of whatever organ is needed. However, in micro-gravity there should be less of a need for any scaffolding.
4) Artificial meat – similar to human organs above, by being able to build (3D print) biological products, one could create a steak or other cuts of meat that biological #D printing.
Problems with space manufacture
One problem with manufacturing metal alloys and fibre optic cable in space, is the immense heat required. Glass melts at 1400C, metals anywhere from 650C to 3400C. Getting rid of all that heat in space could present a significant problem. Not to mention the vessels required to hold molten materials weigh a lot.
And metal and glass manufacturing processes can also create waste, such as hot metal/glass particulates that settle on the floor on earth, but who knows where in space. To manufacture metal or glass on ISS would require a very heat tolerant, protected environment or capsule, lots of power to provide heat and radiator surfaces to release said heat.
And of course, delivering raw materials for metals and glass to space (LEO) would cost a lot (SpaceX $2.7K/kg , Atlas V $13.2K/kg). As such, the business case for metal alloy manufacturing in space doesn’t appear positive.
But given the reduced product weight and potentially higher prices one can charge for the product, fibre pptical glass may make business sense. Especially, if you could get by with 1/10th the glass because it has 1/10th the signal loss.
And if you don’t have to ship raw materials from earth (using the moon or asteroids instead), it would improvesboth business cases. That is, assuming raw material discovery and shipping costs are 1/6th or less as much as shipping from earth.
As for organs, as they can’t be manufactured on earth (yet), it could be the “killer app’ for made in space. But it’s sort of a race against time. Doing this in space may be a lot easier today but more research is going on to create organs on earth than in space. But eventually, manufacturing these on earth could be a lot cheaper and just as effective.
But I don’t see a business case for meat in space unless it’s to support making food for astronauts on ISS. Even then, it might be cheaper to just ship them some steak.
Products hard to make in space
I would think anything that doesn’t require gravity to work, should be easier to produce in space.
But that eliminates distillation, e.g., fossil fuel refining, fermentation, and many other chemical distillation processes (see Wikipedia article on Distillation).
But gravity is also used in depositing and holding multiple layers onto one another. So manufacturing paper, magnetic/optical disk platters, magnetic tapes, or any other product built up layer by layer, may not be suitable for space manufacture.
Not sure about semiconductors, as deposition steps can make use of chemical vapors. And that seems to require gravity. But it’s conceivable that in the absence of gravity, chemicals may still adhere to the wafer surface, as it’s an easier location to combine with than other surfaces in the chamber. On the other hand, they may just as likely retain their mixture in the vapor.
Growing extremely pure silicon ingots may be something better done in space. However, it may suffer from the same problems as metal alloy manufacturing. Given the need for extreme purity and the price paid for pure silicon, I would think this would be something to research ahead of metal alloys.
For further research
But in the end, if and when we become a space fairing people, we will need to manufacture everything in space. As well as grow or find raw materials easier than shipping them from the earth.
So, some research ought to be directed on how to perform distillation and multi-layer product manufacturing in space/micro-gravity. Such processes could potentially be done in a centrifuge, if they truly can’t be gone without gravity.
It’s also unclear how to boil any liquid in 0g or micro-g without convection (see Bizarre Boiling NASA Science article). According to the article, it creates one big bubble that stays where it is formed. Providing some way to extract this bubble in place would seem difficult. Boiling liquids in a centrifuge may work.
In any case, I’m sure the ISS crew would be more than happy to do any research necessary to figure out how to brew beer, let alone, distill vodka in space.
Read a recent article about Intel’s Pohoiki Beach neuromorphic system and their Loihi chips, that has scaled up to 8M neurons in IEEE Spectrum (Intel’s neuromorphic system hits 8 M neurons). In the last month or so I wrote up about a two startups one of which seemed (?) to be working on a neuromorphic chip development (see my Photonics computing sees the light of day post).
But first please take our new poll:
I’ve been writing about neuromorphic chips since 2011, 8 long years (see IBM SyNAPSE chip post from 2011 or search my site for “neuromorphic”) and none have been successfully reached the market. The problems with neurmorphic architectures have always been twofold, scaling AND software.
Scaling up neurons
The human brain has ~86B neurons (see wikipedia human brain article). So, 8 million neuromorphic neurons is great, but it’s about 10K X too few. And that doesn’t count the connections between neurons. Some human neurons have over 1000 connections between nerve cells (can’t seem to find this reference anymore?).
To get from a single chip with 125K neurons to their 8M neuron system, Intel took 64 chips and put them on a couple of boards. To scale that to 86B or so would take ~690, 000 of their neuromorphic chips. Now, no one can say if there’s not some level below 85B neuromorphic neurons, that could support a useful AI solution, but the scaling problem still exists.
Then there’s the synapse connections between neuromorphic neurons problem. The article says that Loihi chips are connected in a heirarchical routing network, which implies to me that there are switches and master switches (and maybe a really big master switch) in their 8M neuromorphic neuron system. Adding another 4 orders of magnitude more neuromorphic neurons to this may be impossible or at least may require another 4 sets of progressively larger switches to be added to their interconnect network. There’s a question of how many hops and the resultant latency in connecting two neuromorphic neurons together but that seems to be the least of the problem with neuromorphic architectures.
Missing software abstractions
The first time I heard about neuromorphic chips I asked what the software looks like and the only thing I heard was that it was complex and not very user friendly and they didn’t want to talk about it.
I keep asking about software for neuromorphic chips and still haven’t gotten a decent answer. So, what’s the problem. In today’s day and age, software is easy to do, relatively inexpensive to produce and can range from spaghetti code to a hierarchical masterpieces, so there’s plenty of room to innovate here.
But whenever I talk to engineers about what the software looks like, it almost seems like a software version of an early plug board unit-record computer (essentially card sorters). Only instead of wires, you have software neuromorphic network connections and instead of electro-magnetic devices, one has software spiking neuromorphic neuron hardware.
The way we left plugboards behind was by building up hardware abstractions such as adders, shifters, multipliers, etc. and moving away from punch cards as a storage medium. Somewhere along this transition, we created programing languages like (macro) Assemblers, COBOL, FORTRAN, LISP, etc. It’s the software languages that brought computing out of the labs and into the market.
It’s been at least 8 years now, and yet, no-one has built a spiking neuromorphic computer language yet. Why not?
I think the problem is there’s no level of abstraction above a neuron. Where’s the aritmetic logic unit (ALU) or register equivalents in neuromorphic computers? They don’t exist as far as I can see.
Until we can come up with some higher levels of abstraction, coding neuromorphic chips is going to be an engineering problem not a commercial endeavor.
But neuromorphism has advantages
The IEEE article states a couple of advantages for neuromorphic computing: less energy to perform inferencing (and possibly training) and the ability to train on incremental data rather than having to train across whole datasets again.
And the incremental training issue doesn’t seem any easier when you have ~80B neurons, with an occasional 1000s of connections between them to adjust correctly. From my perspective, its training advantage seems illusory at best.
Another advantage of neuromorphism is that it simulates the real analog logic of a human brain. Again, that’s great but a brain takes ~22 years to train (college level). Maybe because neuromorphic chips are electronic perhaps training can be done 100 times faster. But there’s still the software issue
~~~~
I hate to be the bearer of bad news. There’s been some major R&D spend on neuromorphism and it continues today with no abatement.
I just think we’d all be better served figuring out how to program the beast than on –spending more to develop more chip hardware..
This is hard for me to say, as I have always been a proponent of hardware innovation. It’s just that neuromorphic software tools don’t exist yet. And I’m afraid, I don’t see any easy way forward to make any progress on this.
Was at a conference last month where there was discussion of the “cloudless” future. This is so wrong, clouds are a threat to every IT hardware and software vendor out there and it’s not going away
The hardware side is easy to see.
Clouds threat to IT hardware vendors
On the storage side, the big hyperscalers have adopted software defined storage from the git go. Smaller ones are migrating that way as well and it’s even impacting data centers as the big virtualization software vendors release more and more functionality in SwDefStorage
And on the networking side, the clouds were an early adopter of Openflow, software defined networking. OpenFlow gear still requires specialized hardware but mostly it’s just a server with PCIe accelerator cards that perform high speed switching. Ditto the prior paragraph here as the virtualization vendors are also moving their networking to SwDefNetworking.
Luckily for servers there’s no such thing as a SwDefServer, yet. But server offerings are under just as big a threat from the cloud. Hyper-scalars are sophisticated enough to design their own server hardware and have it manufactured to spec. The smaller ones can make use of whitebox servers. Both of them, at the volumes they consume servers, can force a race to the bottom on pricing.
So server vendors are being relegated to the data center for the most part. And as data center servers become more powerful, virtualized environments need less of them.
The threat to IT software vendors
Make no mistake about it, software is under just as much threat as hardware. AWS and Oracle was probably the best example of how this works. Oracle was once a profitable niche market on AWS. Today, Oracle is not even available on AWS marketplace anymore.
This sort of dynamic can happen to any solution where acceptable open source alternatives exist. With the cloud’s sophistication and volumes they can easily take the sting out of using open source by providing ease of deployment, use and maintenance along with performance scalability. That makes running open source on clouds as easy as any packaged solution.
Internet Splat Map by jurvetson (cc) (from flickr)
Albeit, maybe the cloud may not offer the support or hand-holding one obtains with packaged software. But open source can be very responsive to bugs/security exposures. Cloud providers can take the time to make their open source solutions bullet proof. And with 1000s to 10,000s of users, running them at scale, it should be easy enough to find any high profile bugs.
Even all those software vendors that make software that executes only on the cloud, to make it run better, more secure or to add some unique functionally are at risk. All these vendors ultimately will suffer by “death from marketplace success“. As they become successful and cloud vendors know inherently how successful they are, they become more interesting to the cloud. Over time more successful solutions will attract cloud provider functionally-equivalent, open source alternatives, that will push them out of the clouds marketplace.
Dealing with the threat to hardware vendors
Hardware vendors have four grand strategies to address the cloud threat.
Stick head in sand, hope it goes away (or at least takes a long time to kill them off). There are still some major vendors with this mindset. Yes, slowly but surely they are coming around to see the light but they think they have a long enough runway to hold on until something better comes along.
Co-opt the cloud by providing unique, hardware capabilities in their cloudenvironment. There are a few hardware vendors that have adopted this strategy. This buys them more time as they can depend on current data center revenues and over time augment this with cloud revenues.
Join the race to the bottom to become a supplier to clouds. Most hardware vendors started out in a highly competitive environment, but over time they have lost their way (found a higher profitability niche). But lurking in their past somewhere, there’s a competitiveness streak that’s dying to come out. The race to the bottom may never be as profitable as data centers but there’s significant revenue to be had here.
Co-opt the cloud by providing services that span multiple clouds. Not exactly creating a hybrid cloud but rather providing a multi-cloud hardware service. Hardware functionality that can be accessed from multiple clouds can enjoy some advantages of the cloud but at the same time generate data center like revenues..
I may be missing some grand hardware vendor strategies but as I’ve talked with hardware vendors over time these seem to be the main ones moving ahead.
I’ve tried a couple of times to talk to vendors in the #1 mindset above about the futility of their approach. Mostly, I get ignored or at best politely brushed off as being alarmist. Their main hope is that the data center continues on in the present environment and that they can retain their dominance there.
Maybe they have a point. The 1960smainframe environment still exists today. And IBM still remains dominant there, and generates profits there. But it just doesn’t matter that much to IT anymore. IT has moved on. .
Richard (Dick) Nafzger with Apollo data tape by Goddard Photo and Video (cc) (from flickr)
Something similar will happen to IT’s data center. Yes it will still exist forever, and perhaps some vendors can continue to profit there.
But the vast majority of IT workloads will be moving to the cloud over time, relegating this to a smaller (proportionally) niche market. They’ve been saying tape is dead since 1967, but it’s still alive, it’s just moved from being a large market to a smaller one (proportionally).
The #2 mindset vendors have a clearer view of wha’s happening with the cloud. They are moving select hardware functionality out to the cloud as soon as they are able. Some are even placing their hardware in cloud provider availability zones (data centers) to support this. We all hope they enjoy lasting success doing this.
But ultimately they to, shall suffer the same fate as software vendors above, due to the cloud’s death by marketplace success. The more successful they become, the higher the likelihood that the cloud providers will go after them with their own functionally-equivalent, software defined solution.
I’m not privy to the contracts between hardware vendors and cloud providers bit perhaps this later transition, to outright competition, can be forestalled enough to make the cloud providers reluctant to compete with them. But hardware success can only lead to more cloud interest and no contract can protect against every contingency.
Those vendors adopting the #3 mindset have to return to their competitiveness roots. Doing this will never be as profitable as today’s data center. So the transition will be painful, but they need to do this soon, while they still have some profits coming from data center sales. The sooner they can deploy these $s to fix supply chains, manufacturing quality/production, drastically slim down marketing and sales, the faster they can start supplying the clouds with appropriate hardware. Profitability will suffer early on but it may never fully recover.
The #4 mindset applies equally well to software vendors as well as hardware vendors but the hardware group seems to be doing this already. Many storage vendors have multi-cloud solutions with hardware positioned in cloud-adjacent facilities that can be accessed from multiple clouds. Such services have to be consumable like any cloud service. But once in place they have a unique value proposition, the ability to move work and data from one cloud to another.
But the only thing stopping cloud providers doing something similar is that they don’t want to help any current user to use a different cloud. Again, depending on how successful this multi-cloud approach becomes, there’s nothing prohibiting the cloud providers from providing similar functionality.
Dealing with the threat to software vendors
Software vendors see 4 grand strategies to deal with the cloud threat:
If you can’t beat them, join them, and create their own cloud. IBM exemplifies this best but one can see this with Microsoft, Oracle, SAP and others. If they can create their own cloud, they can start to compete with cloud providers on an equal footing. Yes they will be smaller but they can enjoy many of the same benefits of bigger clouds, just not as much. .
Offer their software services/stack on the cloud providers. This is similar to the hardware vendors #2 mindset. Yet this has suffered from death by marketplace success since the inception.
Co-opt the cloud by providing services that fuse the data center and the cloud environments. Thus creating hybrid cloud solutions that span the data center-cloud environment which seem to have a longer runway. But this lasts only as long as the data center is a significant market.
Co-opt the cloud by providing services that span cloud provider vendors. Multi-cloud solutions are more apparent for hardware, but nothing prohibits a software vendor from offering services that spans clouds.
I may be missing a few grand strategies here but these seem to be the major ones software vendors are using to deal with the cloud. And just like hardware vendors above, much of the success of these strategies (at least #2,3 &4) depends on flying under the radar of cloud providers. Limiting your success may give you some time to eek out a decent revenue/profitability stream, while the cloud provider kills off the more successful solutions ahead of you. But you’re all living on borrowed time.
The most interesting one is #1. Yes economies of scale will matter, which will make their long term viability a concern. But at least you can be on the same playing field. Most of these companies have sizable treasure chests and if invest serious money on their own clouds, they may have a chance to survive.
Cloud providers are taking their time
The other thing that’s prolonging the data center and correspondingly vendors existence is cloud providers expenses. With all their hardware volumes, use of white box or own designed hardware and open source software, does it make any sense that IT could provide matching services in data centers by themselves.
But something is chewing up their revenues, Maybe it’s marketing, customer acquisition, software/hardware development or support expenses. I tend to think it’s trying to keep pace with customer growth. They end up having to anticipate this growth ahead of time and position hardware, software and services before the customers exist to use them.
I don’t think there’s anything more mysterious to their lack of profitability than that. They all want all the customers they can get. They are have significant growth and they are all charging a premium for their service. However, I may be wrong.
But how long can such hyper-growth last. At some point, as more and more IT organizations move to the cloud this growth will slow, prices will start to come down and it will set off a vicious cycle, more cloud success brings more volumes, less overhead and should lower prices which brings more cloud success.
More cloud success brings less volumes for hardware and software vendors, more overhead and ultimately higher prices.
None of the above solutions seem that attractive to hardware or software vendors but I see only a few ways forward for all of them.
In part 2, I’ll discuss some out of the box strategies that move beyond the data center and the cloud that may be just the way forward for hardware and software vendors need to take the cloud on.
