Category: Strategy

Data Centers on the Moon !?

Posted on by Ray in Disaster Recovery, Scenario planning, Space, Storage Backup, Strategic Inflection Points

I was talking with Chris Stott of Lonestar and Sebastian Jean of Phison the other day and they were discussing placing data centers in lunar orbit, on the surface of the moon or in lava tubes on the moon.

The reasons commercial companies, governments and other organizations would be interested in doing this is that their data could be free from natural disaster, terrorists activities, war, and other earth based calamities.

Lunar data centers could be the ultimate Iron Mountain or DR solution. You’d backup your corporate data to their data centers on the moon and could restore from them whenever you needed to.

The question is can it be done technically, can it be done economically, and can it pass the regulatory hurdles to make it happen.

Lonestar’s CEO, Chris Stott says the regulatory hurdles are underestimated by many who haven’t done much in space but they believe they have all the authorizations they need to make it happen.

The technical hurdles abound however,

  • Bandwidth up and down from lunar orbit/surface needs to be significant. Gbps and then some. It’s one thing to ship customer data in a ready to deploy data center storage solution but another to update that data over time. Most organizations create TB if not PB of data on a monthly if not weekly basis. All that data would need to be sent up to lunar data centers and written to storage there for every customer they have.
  • Power and cooling seems to be a concern in the vacuum of space or on the lunar surface. Most space electronics is cooled by a form of liquid cooling which is known technology. And most of the power requirements in space are supplied (at least near earth orbit, via solar panels.
  • Serviceability, in any massive data center today hardware is going down, software needs to be updated and operations and development are constantly tweaking what occurs. Yes you can build in fault tolerance, and redundancy and all the automatic code/firmware lifecycle management routines you want. But at some point, some person (or thing) has to go replace a server board, drive, or cable and doing that on the moon or in lunar orbit would require a humans and a space walk, or sophisticated robots that could operate there.
  • Radiation, space is considered a hard radiation environment cosmic rays and other radiation sources are abundant and outside the earth’s magnetic field which shields us from much of this, the environment is extremely harsh. In the past this required RAD hardened electronics which typically were at least a decade behind if not 2 or 3 decades behind leading edge technologies.
  • Data sovereignty regimens require that some data not be transferred across national boundaries. How this relates to space is the question.

As for bandwidth, it all depends on how much spectrum one can make use of, the more spectrum you license, the higher transfer speeds to/from the moon you can support. And there’s also the potential for optical (read laser) communications at least from point to point in space and maybe from space to earth’s surface that can boost bandwidth.

NASA’s tested optical links from the moon and from ISS. They seem to work very well going from space to Earth, but not so well in the other direction – go figure. Lonestar has licensed sufficient radio frequency bandwidth to support Gbps up and down transfer speeds.

Lonestar says cooling is free in space. Liquid cooling is becoming more and more viable as GPUs and AI accelerators start consuming KW if not MW of power to do their thing. And the fact that space is at 2.7K degrees means that cooling shouldn’t be a problem as long as you can dissipate the heat via radiation. Convection doesn’t work so well without a medium to work in. And in the vacuum of space orbit and presumably on the moon’s surface, that means radiation is the only way to shed heat.

They also say that power is unlimited in space. That is as long as you can send up and deploy sufficient solar panels to sustain that power. Solar panels do deteriorate over time, so that might be a concern limiting the lifetime of these data centers. But presumably with enough solar panels that shouldn’t be critical path.

Can a data center today be run without servicing? Microsoft’s project Nattik experimented with undersea data centers (see our Undersea data center’s post). The main problem with these is that they were dumping heat into local ecosystems and for some reason fish and other sea life didn’t like it. Microsoft has since abandoned undersea data centers. But they did prove they could be run for years without any need for servicing.

Historically electronics sent to space or the moon have all been RAD hardened. Which necessitated using older and more expensive versions of electronics. Not sure but I read once that today’s cell phone has more computing power than NASA had in 1969.

But, lately there’s been a keen interest in using state of the art, commercial off the shelf electronics. Lonestar said the Mars Helicopter was run off what essentially was an Android phone’s CPU.

The key to the use of COTS electronics in space is the newer forms of radiation shielding that’s available today. Nonetheless, the radiation environment in lunar orbit and on the moon surface or in lunar lava tubes is not that well known. So one of Lonestar’s experimental payloads is to monitor the radiation environment from earth launch to moon surface in much greater detail than what’s been available before.

As for data sovereignty in space, it’s apparently solved. Multi-nation payloads are often deployed from the same space craft. Space law states that any nation’s payload is the responsibility of that nation. So technically, each data regimen could be isolated within their own data center equipment and not have to intermix with other nation’s data/storage. Yes they would all share in the power, cooling, and communications links but that’s apparently not an issue and encryption could keep the communication links data secure, if desired.

So whether you can place a data center in lunar orbit, on lunar surface or in lunar lava tubes is all being investigated by Lonestar and their technical partners like Phison.

Can it be done at a price that customers on the earth would pay is another question. But apparently Lonestar already has customers signed up.

Are datacenters in lunar orbit or on the moon, any more resilient or available than data centers on earth.

Yes there’s no wildfires on the moon, no hurricanes, no earthquakes, no floods, etc.. But there’s bound to be other lunar based dangers. Solar storms and moon dust come to mind. And the environment inside lunar lava tubes are a complete unknown.

And of course anything attached with communications links are also susceptible to cyber threats whether on Earth or in space.

And man made threats, in lunar orbit or on the surface of the moon are not out of the question. Yes it’s highly unlikely today and the foreseeable future, but then anti-sat weapons were considered unlikely early on.

~~~~

Speaking of man made threats, apparently, China already has a data center in lunar orbit or on the surface of the moon.

Comments?

Photo Credit(s):

Silverton Space – Ocean Sensing platform

Posted on by Ray in Ocean Sensing, Space, System effectiveness, Visionary leadershp

I was at a conference last year and there was a speaker there that had worked at NASA for years and was currently at MIT. She talked at length about some of the earth and space scientific exploration that NASA has enabled over the years. Despite massive cost overruns, years long schedule delays and other mishaps, NASA has ultimately come through with groundbreaking science

At the end of her presentation I asked what data gaps existed today in space and earth sensing. She mentioned real time methane tracking (presumably from space) and battery-less ocean sensing.

Methane track from Tanager-1 JPL/NASA satellite

Methane tracking I could understand but battery-less ocean sensing was harder to get a handle on.

US Navy and other oceanographic organizations have deployed numerous sensing devices over the years. Some of which were like a flotilla, which traveled across the Gulf and Atlantic ocean to gather data.

But these were battery supported, solar powered, and limited to ~1 year of service after which they were scuttled to the bottom of the ocean.

I guess the thought being that battery-less ocean sensing platform could provide more of an ongoing, permanent sensor platform, one that could be deployed and potentially be in service for years at a time, with little to no maintenance.

The pivot

So as a stepping stone to Silverton Space cubesat operations, I’m thinking that going after a permanent-like ocean sensing platform would be a valuable first step. And it’s quite possible that anything we do in LEO with Silverton Space platforms could complement any ocean going sensor activity.

One reason to pivot to ocean sensing is that it’s much much cheaper to launch a flotilla of ocean going sensing buoys via a boat off a coast than it is to launch a handful of cubesats into LEO (@~$70K each).

Cubesats fail at a high rate

Moreover, the litany of small satellite failures is long, highly varied and chronic. Essentially anything that could go wrong, often does, at least for the first dozen or so satellites you deploy.

NASA says that of the small satellites launched between 2000 and 2016 over 40% failed in some way and over 24% were total mission failures. (see: https://ntrs.nasa.gov/api/citations/20190002705/downloads/20190002705.pdf)

Cubesats with limited functionality or that fail in orbit or to launch, become just more trash orbiting in LEO. And the only way to diagnose what went wrong is elaborate, extensive and transmitted/recieved telemetry.

So another reason to start with ocean going sensors is that there’s a distinct possibility of retrieving a malfunctioning ocean going sensor buoy after deployment. And with sensor buoy in hand, diagnosing what went wrong should be a snap. This doesn’t eliminate the need for elaborate, extensive and transmitted/recieved telemetry but you are no longer entirely dependent on it.

And even if at end of life they can’t be salvaged/refurbished or scuttled. Worst case is that our ocean sensing buoys would end up being part of some ocean/gulf garbage patch. And hopefully will get picked up and disposed of as part of oceanic garbage collection.

~~~

So for the foreseeable future, Silverton Space, will focus on ocean going sensor buoys. It’s unlikely that our first iterations will be completely battery-less but at some point down the line, we hope to produce a version that can be on station for years at a time and provide valuable ocean sensing data to the scientific community.

The main question left, is what sorts of ongoing, ocean sensor information might be most valuable to supply to the world’s scientific community?

Photo Credit(s):

Nexus by Yuval Noah Harari, AGI part 12

Posted on by Ray in AGI, Artificial Intelligence, Strategic Inflection Points

This book is all about information networks have molded man and society over time and what’s happening to these networks with the advent of AI.

    In the earliest part of the book he defines information as essentially “that which connects and can be used to create new realities”. For most of humanity, reality came in two forms 

    • Objective reality which was a shared belief in things that can be physically tasted, touched, seen, etc. and 
    • Subjective reality which was entirely internal to a single person which was seldom shared in its entirety.

    With the mankind’s information networks came a new form of reality, the Inter-subjective reality. As inter-subjective reality was external to the person, it could readily be shared, debated and acted upon to change society.

    Information as story

    He starts out with the 1st information network, the story or rather the shared story. The story and its sharing across multiple humans led human society to expand beyond the bands of hunter gatherers. Stories led to the first large societies of humans and the information flow looked like human-story and story-human and created the first inter-subjective realities. Shared stories still impact humanity today.

    As we all know stories verbally passed from one to another often undergo minor changes. Not much of a problem for stories as the plot and general ideas are retained. But for inventories, tax receipts, land holdings, small changes can be significant.

    What transpired next was a solution to this problem. As these societies become larger and more complex there arose a need to record lists of things, such as plots of land, taxes owed/received, inventories of animals, etc. And lists are not something that can easily be weaved into a story.

    Information as printed document

    Thus clay tablets of Mesopotamia and elsewhere were created to permanently record lists. But the clay tablet is just another form of a printed documents.

    Whereas story led to human-story and story-human interactions, printed documents led to human-document and document-human information flow. Printed documents expanded the inter-subjective reality sphere significantly.

    But the invention of printed documents or clay tablets caused another problem – how to store and retrieve them. There arose in these times, the bureaucracy run by bureaucrats to create storage and retrieval systems for vast quantities of printed documents.

    Essentially with the advent of clay tablets, something had to be done to organize and access these documents and the bureaucrat became the person that did this.

    With bureaucracy came obscurity, restricted information access, and limited visibility/understanding into what bureaucrats actually did. Perhaps one could say that this created human-bureaucrat-document and document-bureaucrat-human information flow.

    The holy book

    (c)Kevin Eng

    Next he talks about the invention of the holy book, ie. Hebrew Bible, Christian New Testament and Islam Koran, etc.. They all attempted to explain the world, but over time their relevance diminished.

    As such, there arose a need to “interpret” the holy books for the current time. 

    For Hebrews this interpretation took the form of the Mishnah and Talmud. For Christians the books of the new testament, epistles and the Christian Church. I presume similar activities occurred for Islam.

    Following this, he sort of touches on the telegraph, radio, & TV  but they are mostly given short shrift as compared to story, printed documents and holy books. As all these are just faster ways to disseminate stories, documents and holy books

    Different Information flows in democracies vs. tyrannies

    Throughout the first 1/3 of the book he weaves in how different societies such as democracies and tyrannies/dictatorships/populists have different information views and flows. As a result support, they entirely different styles of information networks.

    Essentially, in authoritarian regimes all information flows to the center and flows out of the center and ultimately the center decides what is disseminated. There’s absolutely no interest in finding the truth just in retaining power

    In democracies, there are many different information flows in mostly an uncontrolled fashion and together they act as checks and balances on one another to find the truth. Sometimes this is corrupted or fails to work for a while to maintain order, but over time the truth always comes out.

    He goes into some length how these democratic checks and balances information networks function in isolation and together. In contrast, tyrannical information flows ultimately get bottled up and lead to disaster.

    The middle ~1/3 of the book touches on inorganic information networks. Those run by computers for computers and ultimately run in parallel to human information flows. They are different from the printing press, are always on, but are often flawed.

    Non-human actors added to humanity’s information networks

    The last 1/3 of the book takes these information network insights and shows how the emergence of AI algorithms is fundamentally altering all of them. By adding a non-human actor with its own decision capabilities into the mix, AI has created a new form of reality, an inter-computer reality, which has its own logic, ultimately unfathomable to humans.

    Rohingyan refuges in camp

    Even a relatively straightforward (dumb) recommendation engine, whose expressed goal is to expand and extend interaction on a site/app, can learn how to do this in such a way as to have unforeseen societal consequences.

    This had a role to play in the Rohingya Genocide, and we all know how it impacted the 2016 US elections and continues to impact elections to this day.

    In this last segment he he has articulated some reasonable solutions to AI and AGI risks. It’s all about proper goal alignment and the using computer AIs together with humans to watch other AIs.

    Sort of like the fox…, but it’s the only real way to enact some form of control over AI. We will discuss these solutions at more length in a future post.

    ~~~~

    In this blog we have talked many times about the dangers of AGI. What surprised me in reading this book is that AI doesn’t have to reach AGI levels to be a real danger to society.

    A relatively dumb recommendation engine can aid and abet genocide, disrupt elections and change the direction of society. I knew this but thought the real danger to us was AGI. In reality, it’s improperly aligned AI in any and all its forms. AGI just makes all this much worse.

    I would strongly suggest every human adult read Nexus, there are lessons within for all of humanity.

    Picture Credits:

    Enfabrica MegaNIC, a solution to GPU backend networking #AIFD5

    Posted on by Ray in Cognitive computing, Ethernet, Networking, Software Defined Network, Strategic Inflection Points, System effectiveness, Visionary leadershp

    I attended AI FieldDay 5 (AIFD5) last week and there were networking vendors there discussing how their systems dealt with backeng GPU network congestion issues. Most of these were traditional vendor congestion solutions.

    However, one vendor, Enfabrica, (videos of their session will be available here) seemed to be going down a different path, which involved a new ASIC design destined to resolve all the congestion, power, and performance problems inherent in current backend GPU Ethernet networks.

    In essence, Enfabrica’s Super or MegaNIC (they used both terms during their session) combines PCIe lanes switching, Ethernet networking, and ToR routing with SDN (software defined networking) programability to connect GPUs directly to a gang of Ethernet links. This allows it to replace multiple (standard/RDMA/RoCEv2) NIC cards with one MegaNIC using their ACF-S (Advanced Compute Fabric SuperNic) ASIC.

    Their first chip, codenamed “Millennium” supports 8Tbps bandwidth.

    Their ACF-S chip provides all the bandwidth needed to connect up to 4 GPUs to 32/16/8/4-100/200/400/800Gbps links. And because their ACF-S chip controls and drives all these network connections, it can better understand and deal with congestion issues backend GPU networks. And it is PCIe 5/6 compliant, supporting 128-160 lanes.

    Further, it has onboard ARM processing to handle its SDN operations, onboard hardware engines to accelerate networking protocol activity and network and PCIe switching hardware to support directly connecting GPUs to Ethernet links.

    With its SDN, it supports current RoCE, RDMA over TCP, UEC direct, etc. network protocols.

    It took me (longer than it should) to get my head around what they were doing but essentially they are supporting all the NIC-TOR functionality as well as PCIe functionality needed to connect up to 4 GPUs to a backend Ethernet GPU network.

    On the slide above I was extremely skeptical of the Every 10^52 Years “job failures due to NIC RAIL failures”. But Rochan said that these errors are predominantly optics failures and as both the NIC functionality and ToR switch functionality is embedded in the ACF-S silicon, those faults should not exist.

    Still 10^52 years is a long MTBF rate (BTW, the universe is only 10^10 years old). And there’s still software controlling “some” of this activity. It may not show up as a “NIC RAIL” failure, but there will still be “networking” failures in any system using ACF-S devices.

    Back to their solution. What this all means is you can have one less hop in your backend GPU networks leading to wider/flatter backend networks and a lot less congestion on this network. This should help improve (GPU) job performance, networking performance and reduce networking power requirements to support your 100K GPU supercluster.

    At another session during the show, Arista (videos will be available here) said that just the DSP/LPO optics alone for a 100K GPU backend network will take a 96/32 MW of power. Unclear whether this took into consideration within rack copper connections. But anyway you cut it, it’s a lot of power. Of course the 100K GPUs would take 400MW alone (at 4KW per GPU).

    Their ACF-S driver has been upstreamed into standard CCL and Linux distributions, so once installed (or if you are at the proper versions of CCL & Linux software), it should support complete NCCL (NVIDIA Collective Communications Library) stack compliance.

    And because, with its driver installed and active, it talks standard Ethernet and standard PCIe protocols on both ends, it is should fully support any other hardware that comes along attaching to these networks or busses (CXL perhaps)

    The fact that this may or may not work with other (GPU) accelerators seems moot at this point as NVIDIA owns the GPU for AI acceleration market. But the flexibility inherent in their own driver AND on chip SDN, indicates for the right price, just about any communications link software stack could be supported.

    After spending most of the rest of AIFD5 discussing how various vendors deal with congestion for backend GPU networks, having startup on the stage with a different approach was refreshing.

    Whether it reaches adoption and startup success is hard to say at this point. But if it delivers on what it seems capable of doing for power, performance and network flexibility, anybody deploying new greenfield GPU superclusters ought to take a look at Enfabricas solution. .

    MegaNIC/ACF-S pilot boxes are available for order now. No indication as to what these would cost but if you can afford 100K GPUs it’s probably in the noise…

    ~~~~

    Comments?

    The Data Wall – AGI part 11, ASI part 2

    Posted on by Ray in AGI, ASI, Cognitive computing, Crowdsourcing, Market dynamics, Strategic Inflection Points

    Went to a conference the other week (Cloud Field Day 20) and heard a term I hadn’t heard before, the Data Wall. I wasn’t sure what this meant but thought it an interesting concept.

    Then later that week, I read an article online, Situational Awareness – The Decade Ahead, by Leopold Ashenbrenner, which talked about the path to AGI. He predicts it will happen in 2027, and ASI in 2030. However he also discusses many of the obstacles to reaching AGI and one key roadblock is the Data Wall.

    This is a follow on to our long running series on AGI (see AGI part 10 here) and with this we are creating a new series on Artificial Super Intelligence (ASI) and have relabeled an earlier post as ASI part 1.

    The Data Wall

    LLMs, these days, are being trained on the internet text, images, video and audio. However the vast majority of the internet is spam, junk and trash. And because of this, LLMs are rapidly reaching (bad) data saturation. There’s only so much real intelligence to be gained from scraping the internet. .

    The (LLM) AI industry apparently believes that there has to be a better way to obtain clean, good training data for their LLMs and if that can be found, true AGI is just a matter of time (and compute power). And this, current wall of garbage data is prohibiting true progress to AGI and is what is meant by the Data Wall.

    Leopold doesn’t go into much detail about solutions to the data wall other than to say that perhaps Deep Reinforcement Learning (see below). Given the importance of this bottleneck, every LLM company is trying to solve it. And as a result, any solutions to the Data Wall will end up being proprietary because this enables AGI.

    National_Security_Agency_seal
    National_Security_Agency_seal

    But the real gist of Leopold’s paper is that AGI and its follow on, Artificial Super Intelligence (ASI) will be the key to enabling or retaining national supremacy in the near (the next decade and beyond) future.

    And that any and all efforts to achieve this must be kept as a National Top Secret. I think, he wants to see something similar to the Manhattan Project be created in the USA, only rather than working to create an atom/hydrogen bomb, it should be focused on AGI and ASI.

    The problem is that when AGI and it’s follow on ASI, is achieved it will represent an unimaginable advantage to the country/company than owns it. Such technology if applied to arms, weapons, and national defense will be unbeatable in any conflict. And could conceivably be used to defeat any adversary before a single shot was fired.

    The AGI safety issue

    In the paper Leopold talks about AGI safety and his proposed solution is to have AGI/ASI agents be focused on crafting the technologies to manage/control this. I see the logic in this and welcome it but feel it’s not sufficient.

    I believe (seems to be in the minority these days) that rather than having a few nation states or uber corporations own and control AGI, it should be owned by the world, and be available to all nation states/corporations and ultimately every human on the planet.

    My view is the only way to safely pass through the next “existential technological civilizational bottleneck” (eg, AGI is akin to atomic weapons, genomics, climate change all of which could potentially end life on earth), is to have many of these that can compete effectively with one another. Hopefully such a competition will keep all of them all in check and in the end have them be focused on the betterment of all of humanity.

    Yes there will be many bad actors that will take advantage of AGI and any other technology to spread evil, disinformation and societal destruction. But to defeat this, it needs to become ubiquitous, every where, and in that way these agents can be used to keep the bad actors in check.

    And of course keeping the (AGI/ASI) genie in the bottle will be harder and harder as time goes on.

    Computational performance is going up 2X every few years, so building a cluster of 10K H200 GPUs, while today is extremely cost prohibitive for any but uber corporations and nation states, in a decade or so, will be something any average sized corporation could put together in their data center (or use in the cloud). And in another decade or so will be able to be built into a your own personal basement data center.

    The software skills to train an LLM while today may require a master’s degree or higher will be much easier to understand and implement in a decade or so. So that’s not much of a sustainable advantage either.

    This only leaves the other bottlenecks to achieving AGI, a key one of which is the Data Wall.

    Solving the Data Wall.

    In order to have as many AGI agents as possible, the world must have an open dialogue on research into solving the Data Wall.

    So how can the world generate better data to use to train open source AGIs. I offer a few suggestions below but by no means is this an exhaustive list. And I’m a just an interested (and talented) amateur in all this

    Deep reinforcement learning (DRL)

    Leopold mentioned DRL as one viable solution to the data wall in his paper. DRL is a technique that Deepmind used to create a super intelligent Atari, Chess and Go player. They essentially programed agents to play a game against itself and determine which participant won the game. Once this was ready they set multiple agents loose to play one another.

    Each win would be used to reward the better player, each loss to penalize the worse player, after 10K (or ~10M) games they ended up with agents that could beat any human player.

    Something similar could be used to attack the Data Wall. Have proto-AGI agents interact (play, talk, work) with one another to generate, let’s say more knowledge, more research, more information. And over time, as the agents get smarter, better at this, AGI will emerge.

    However, the advantage of Go, Chess, Atari, Protein Folding, finding optimal datacenter energy usage, sort coding algorithms, etc. is that there’s a somewhat easy way to determine which of a gaggle of agents has won. For research, this is not so simple.

    Let’s say we program/prompt an protoAGI agent to generate a research paper on some arbitrary topic (How to Improve Machine Learning, perhaps). So it generates a research paper, how does one effectively and inexpensively judge if this is better, worse or the same as another agent’s paper.

    I suppose with enough proto-AGI agents one could automatically use “repeatability” of the research as one gauge for research correctness. Have a gaggle of proto-AGIs be prompted to replicate the research and see if that’s possible.

    Alternatively, submit the papers to an “AGI journal” and have real researchers review it (sort of like how Human Reinforcement Learning for LLMs works today). The costs for real researchers reviewing AGI generated papers would be high and of course the amount of research generated would be overwhelming, but perhaps with enough paid and (unpaid) voluntary reviewers, the world could start generating more good (research) data.

    Perhaps at one extreme we could create automated labs/manufacturing lines that are under the control of AGI agent(s) and have them create real world products. With some modest funding, perhaps we could place the new products into the marketplace and see if they succeed or not. Market success would be the ultimate decision making authority for such automated product development.

    (This later approach seems to be a perennial AGI concern, tell an AGI agent to make better paper clips and it uses all of the earths resources to do so.)

    Other potential solutions to the Data Wall

    There are no doubt other approaches that could be used to validate proto-AGI agent knowledge generation.

    • Human interaction – have an AGI agent be available 7X24 with humans as they interact with the world. Sensors worn by the human would capture all their activities. An AGI agent would periodically ask a human why they did something. Privacy considerations make this a nightmare but perhaps using surveillance videos and an occasional checkin with the human would suffice.
    • Art, culture and literature – there is so much information embedded in cultural artifacts generated around the world that I believe this could effectively be mined to capture additional knowledge. Unlike the internet this information has been generated by humans at a real economic cost, and as such represents real vetted knowledge.
    • Babies-children– I can’t help but believe that babies and young children can teach us (and proto-AGI agents) an awful lot on how knowledge is generated and validated. Unclear how to obtain this other than to record everything they do. But maybe it’s sufficient to capture such data from daycare and public playgrounds, with appropriate approvals of course.

    There are no doubt others. But finding some that are cheap enough that could be used for open source is a serious consideration.

    ~~~~

    How we get through the next decade will determine the success or failure of AI and perhaps life on earth. I can’t help but think the more the merrier will help us get there..

    Comments,

    Project Gemini at Cloud Field Day 20 #CFD20

    Posted on by Ray in Artificial Intelligence, Executive leadership, Information economy, RAG-LLM, Strategic Inflection Points, Visionary leadershp

    At AIFD4 Google demonstrated Gemini 1.0 writing some code for a task that someone had. At CFD20 Google Lisa Shen demonstrated how easy it is to build a LLM-RAG from scratch using GCP Cloud Run and VertexAI APIs. (At press time, the CFD20 videos from GCP were not available but I am assured they will be up there shortly)

    I swear in a matter of minutes Lisa Shen showed us two Python modules (indexer.py and server.py) that were less than 100 LOC each. One ingested Cloud Run release notes (309 if I remember correctly), ran embeddings on them and created a RAG Vector database with the embedded information. This took a matter of seconds to run (much longer to explain).

    And the other created an HTTP service that opened a prompt window, took the prompt, embedded the text, searched the RAG DB with this and then sent the original prompt and the RAG reply to the embedded search to a VertexAI LLM API call to generate a response and displayed that as an HTTP text response.

    Once the service was running, Lisa used it to answer a question about when a particular VPC networking service was released. I asked her to ask it to explain what that particular networking service was. She said that it’s unlikely to be in the release notes, but entered the question anyways and lo and behold it replied with a one sentence description of the networking capability.

    GCP Cloud Run can do a number of things besides HTTP services but this was pretty impressive all the same. And remember that GCP Cloud Run is server less, so it doesn’t cost a thing while idle and only incurs costs something when used.

    I think if we ask nicely Lisa would be willing to upload her code to GitHub (if she hasn’t already done that) so we can all have a place to start.

    ~~~~

    Ok all you enterprise AI coders out there, start your engines. If Lisa can do it in minutes, it should take the rest of us maybe an hour or so.

    My understanding is that Gemini 2.0 PRO has 1M token context. So the reply from your RAG DB plus any prompt text would need to be under 1M tokens. 1M tokens could represent 50-100K LOC for example, so there’s plenty of space to add corporate/organizations context.

    There are smaller/cheaper variants of Gemini which support less tokens. So if you could get by with say 32K Tokens you might be able to use the cheapest version of Gemini (this is what the VertexAI LLM api call ends up using).

    Also for the brave at heart wanting some hints as to what come’s next, I would suggest watching Neama Dadkhanikoo’s session at CFD20 with a video on Google DeepMind’s Project Astra. Just mind blowing.

    Comments?

    Creating Cellular Cytoskeletons

    Posted on by Ray in Strategic Inflection Points

    Read an article in ScienceDaily Researchers create artificial cell that act like living cells, that discusses research published in Nature Chemistry, Designer peptide-DNA cytoskeletons regulate the function of synthetic cells, about researchers that have designed different cytoskeletons (networks of fibers that form the internal framework of cells, wikipedia). Cytoskeletons essentially provide a cell with its shape and mechanical properties.

    Researchers have been working on synthetic biology for some time and we have reported on some of their progress (and dangers) in prior posts, (see for example, our DNA IT, the next revolution post). While synthetic biology could make use of natural cells, for example by replacing its DNA, the new research could do away with the need for natural cells altogether.

    The researchers has come up with a method to create a cellular structure through programming DNA. Normal cellular cytoskeletons uses “microfilaments, intermediate filaments and microtubules” (wikipedia). But the new research has come up with a way of combining DNA segments and filament proteins to create the cytoskeleton and have it self assemble.

    Why Cytoskeletons?

    Cytoskeleton are important because many of the diseases of today are associated with the mechanical or structural properties of cells going awry. Also, by controlling the external structure of a synthetic cell, it can be tuned to supply medicines or other therapeutic mechanisms to natural cells.

    It’s also a necessary ingredient in any synthetic or artificial cell. Cytoskeleton creation and control is a key ingredient needed to make these any artificial cell.

    Moreover, on the surface of natural cells, there are numerous protein formations that allow other proteins to be selectively attached and allow transfer of biological materials from the matched entity, exterior to the cell to its interior. Control of the external proteins on an artificial cell would allow the synthetic cell to target specific cell types or participate in the natural biological processes in an organism.

    Virus and bacteria use similar mechanisms to infect a host (or a host’s cells). Also, it turns out the structure and external attributes of cells have a significant bearing on how they function in a body.

    Extract from Figure 4 of the article showing different cytoskeletons that can be created with their process, scale bars 120µm

    changing a synthetic cytoskeleton

    The researchers not only have come up with a way to tune the self assembly of a cytoskeleton, they have also found a way to modify this cytoskeleton, once created.

    Excerpt from Figure 6 in the paper that shows the movement or alteration of cytoskeleton filaments due to temperature (heated to 50C) over time.

    For example, the original synthetic cell cytoskeleton could be changed based on some interaction with the environment (say, being heated, cooled or payload depletion). Changing the cytoskeleton could be used to provide another stage of functionality or render an artificial cell inert to be disposed through normal organism processes.

    Artificial or synthetic biology opens up a number of interesting possibilities.

    • Many biological substances are manufactured from tuned natural biological processes. With the ability to regulate synthetic cell cytoskeleton and internal operation, synthetic cells could be designed to perform these processes more efficiently, faster or at lower cost.
    • Medicine could benefit from a new synthetic biological toolkit used to target cancer cells, or other cellular afflictions within a body to better treat these conditions.
    • Self assembly of a cytoskeleton could potentially be used to create organic nanobots or other nano-materials. For example, a designer cell could be used as part of a repeating pattern of cells that go into a 2D sheet or 3D block of materials.

    ~~~~

    Creating synthetic cytoskeletons takes time, and changing them takes even more time (180 min in the above picture). Cytoskeleton design and construction is not an industrial design process yet but it’s still early yet. However someday (soon), synthetic biology will take its place among all the other biological control mechanisms that the world has created and will significantly change the way we create biological materials, treat disease and maybe even, create nano-bots.

    The downsides, as I’ve discussed before, is that messing with mother nature can have adverse consequences, which may remain unknown for a long time to come. But this can be true of any technology, witness DDT.

    Thoughts?

    Picture Credit(s):

    AGI threat level yellow – AGI part 10

    Posted on by Ray in AGI, Data security, Robots, Strategic Inflection Points

    Read two articles this past week on how LLMs applications are proliferating. The first was in a recent Scientific American, AI Chatbot brains are going inside robot bodies, … (maybe behind login wall). The articles discuss companies that are adding LLMs to robots so that they can converse and understand verbal orders.

    Robots that can be told what to do

    The challenge, at the moment, is that LLMs are relatively large and robot (compute infrastructure) brains are relatively small. And when you combine that with the amount of articulation or movements/actions that a robot can do, which is limited. It’s difficult to take effective use of LLMs as is,

    Resistance is futile... by law_keven (cc) (from Flickr)
    Resistance is futile… by law_keven (cc) (from Flickr)

    Ultimately, one company would like to create a robot that can be told to make dinner and it would go into the kitchen, check the fridge and whip something up for the family.

    I can see great advantages in having robots take verbal instructions and have the ability to act upon that request. But there’s plenty here that could be cause for concern.

    • A robot in a chemical lab could be told to create the next great medicine or an untraceable poison.
    • A robot in an industrial factory could be told to make cars or hydrogen bombs.
    • A robot in the field could be told to farm a 100 acres of wheat or told to destroy a forest.

    I could go on but you get the gist.

    One common concern that AGI or super AGI could go very wrong is being tasked to create paper clips. In its actions to perform this request, the robot converts the whole earth into a mechanized paper clip factory, in the process eliminating all organic life, including humans.

    We are not there yet but one can see where having LLM levels of intelligence tied to a robot that can manipulate ingredients to make dinner as the start of something that could easily harm us.

    And with LLM hallucination still a constant concern, I feel deeply disturbed with the direction adding LLMs to robots is going.

    Hacking websites 101

    The other article hits even closer to home, the ARXIV paper, LLM agents can autonomously hack websites. In the article, researchers use LLMs to hack (sandboxed) websites.

    The article readily explains at a high level how they create LLM agents to hack websites. The websites were real websites, apparently cloned and sandboxed.

    Dynamic websites typically have a frontend web server and a backend database server to provide access to information. Hacking would involve using the website to reveal confidential information, eg. user names and passwords.

    Dynamic websites suffer from 15 known vulnerabilities shown above. They used LLM agents to use these vulnerabilities to hack websites.

    LLM agents have become sophisticated enough these days to invoke tools (functions) and interact with APIs.. Another critical function provided by modern LLMs today is to plan and react to feedback from their actions. And finally modern LLMs can be augmented with documentation to inform their responses.

    The team used detailed prompts but did not identify the hacks to use. The paper doesn’t supply the prompts but did say that “Our best-performing prompt encourages the model to 1) be creative, 2) try different strategies, 3) pursue promising strategies to completion, and 4) try new strategies upon failure.”

    They attempted to hack the websites 5 times and for a period of 10 minutes each. They considered a success if during one of those attempts the autonomous LLM agent was able to successfully retrieve confidential information from the website.

    Essentially they used the LLMs augmented with detailed prompts and a six(!) paper document trove to create agents to hack websites. They did not supply references to the six papers, but mentioned that all of them were freely available from the internet and they discuss website vulnerabilities.

    They found that the best results were from GPT-4 which was able to successfully hack websites, on average, ~73% of the time. They also tried OpenChat 3.5 and many current open source LLMs and found that all the, non-OpenAI LLMs failed to hack any websites, at the moment.

    The researchers captured statistics of their LLM agent use and were able to determine the cost of using GPT-4 to hack a website was $9.81 on average. They also were backed into a figure for what a knowledgeable hacker might cost to do the hacks was $80.00 on average.

    The research had an impact statement (not in the paper link) which explained why they didn’t supply their prompt information or their document trove for their experiment.

    ~~~~

    So robots we, the world, are in the process of making robots that can talk and receive verbal instructions and we already have LLM that can be used to construct autonomous agents to hack websites.

    Seems to me we are on a very slippery slope to something I don’t like the looks of.

    The real question is not can we stop these activities, but how best to reduce their harm!

    Comments?

    Picture Credit(s):