Quantum computing – part 2

Held a briefing recently with IBM, to discuss their recent Q Systems One announcements and their Quantum Compting Cloud. IBM Q Systems One currently offers 10 quantum computers usable by any IBM Q premiere partners. Not clear what in costs, but there aren’t many available for use anywhere else. .

On the call they announced the coming availability of their 53 qubit QC to be installed at their new IBM Quantum Computing Center in Poo’keepsie, New York. The previous IBM Q systems were all located in Yorktown Heights center and their prior largest systems were 20 qubits.

At this time in the industry running quantum computing algorithms generate probabilistic results and as such need to be run many times to converege onto a reliable answer. IBM calls these runs quantum shots. It’s not unusual to run 1000 quantum computing shots to generate a solution to a quantum algorithm. They call running any number of quantum shots to generate an answer a quantum experiment or computation

On the call they mentioned besides the 160K registered users they have for IBM Q Systems One, there’s also over 80 commercial partners. IBM is moving to a cloud version of their development kit Qiskit which should start slowing their download counts down.

Last week in our race for quantum supremacy post we discussed Google’s latest quantum computer with 54 qubits. In that system each qubit had 4 connections (one to each of its nearest neighbors). IBM decried the lack of a peer-reviewed research paper on their technology which made it difficult to understand.

NISQ era

The industry describes the current quantum computing environment as Noisy Intermediate-Scale Quantum (NISQ) technology (for more information see Quantum computing in the NISQ era and beyond paper). The hope is that someday we’ll figure out how to design a quantum computer that has sufficient error correction, noise suppression, and qubit connectivity to perform quantum computing without having to do 1000 quantum shots. Although, quantum mechanics being a probabilistic environment, I don’t foresee how this could ever be the case.

IBM appears to be experimenting with other arrangements for their qubit connections. They say the more qubit connections the noisier the system is, so having fewer but enough to work, seems to be what they are striving for. Their belief is that smart (quantum computing) compilers can work with any qubit connectivity topology and make it work for a problem. It may generate more gates, or a deeper quantum circuit but it should result in equivalent answers. Kind of like running a program on X86 vs. ARM, they both should generate the same answers, but one may be longer than the other.

At the moment IBM has 10 QCs that are going to be split between Poo’keepsie and Yorktown Heights (IBM research), in Yorktown 3-20qubit QC, 1-14qubit QC, as well as 1-5qubit QC and in Poo’keepsie 2-20qubit QC as well as 3-5qubit QC. As I understand it, all of these are available for any IBM Q registered users to run (not sure about the cost though).

The new IBM Q System

The new IBM Q System One 53 qubit system is arrayed with mostly 2 qubit connections and only a select few (11) having 3 qubit connections. This seems to be following down the path of their earlier Q Systems that had similar levels of connectivity between quits.

Quantum Supremacy vs. Quantum Volume

Rather than talking about Quantum Supremacy, IBM prefers to discuss Quantum Advantage and it’s all about Quantum Volume (QV). QV provides a measure of how well quantum computers perform real work. There’s many factors that go into QV and IBM offers a benchmark in their toolkit that can be run on any gate-based, quantum computer to measure quantum volume.

IBM plans to double QV every year and in a decade, they believe they will have achieved Quantum Advantage. Current IBM Q Systems One QC have a QV of 8 or 16. IBM claims that if they can double QV every year they should reach Quantum Advantage in a decade (2029)!

They have yet to measure QV on their new, as yet to be built, 53 qubit quantum computer. But they say, given all the factors, over time the same quantum computer can have different QV. And even though IBM’s 53 qubit quantum computer will be built shortly, some of the components will likely change over time (e.g. microwave drivers) as they find better technologies to use, which will impact QV.

Qiskit, IBM’s open source, quantum computing stack

There was lots of talk about Qiskit being opensource (Qiskit GitHub). This means anyone can take advantage of it to generate solutions for their own quantum computer.

It’s written in Python3 and consists of 4 components:

Ignis – which provides characterization tools and error mitigation tools

Aqua – which provides a framework for users to leverage QC to perform algorithms and provides current QC algorithms that have been proved out to work

Aer – which provides special purpose (classical computing) simulators for QC algorithms to run on

Terra – which seems to be the low level programming for IBM’s QC, presumably this would need to be customized for every quantum computer.

IBM seems to be trying to make Qiskit easily usable. They have created a Youtube video series on it, with Qiskit API and other educational resources, to make quantum computing on IBM Q Systems One more widely usable..

IBM also announced they will add another Quantum Computing center at an yet to be announced, IBM facility in Germany. It seems that IBM wants to make quantum computing something everyone, across the world, can use.

~~~

Some would say we are very early in the use of quantum computing technology. On the other hand, Google, IBM and others companies are using it to perform real (if not theoretical) work. The cloud seems like a great vehicle for IBM to introduce this sort of technology to a wider audience.

Luckily, I have 10 more years to change out all my data encryption to a “quantum proof ” encryption algorithm. Hopefully, one can be found by then.

Photo Credit(s):

Screen shots from IBM’s Q Systems One briefing...

The race for quantum supremacy

Last week, there were a number of reports on Google having achieved quantum supremacy (e.g, see Rumors hint of Google achieving quantum supremacy ScienceNews). The technical article behind the reports was taken down shortly after having appeared. Unclear why but some of their quantum computing rivals have declared this as a true breakthrough to just a publicity stunt.

Using BING, I was able to find an (almost readable) copy of the paper on FileBin and cached copy on SpaceRef.

What Google seems to have done is to implement in a new quantum processor, a (pseudo) random number generator, a random sampling algorithm and an algorithm that verifies randomness based on the sample. he pseudo-random number generator was a quantum circuit, and the sampling another quantum circuit. Randomness verification was done by computing the probability of a bit string (random number) appearing in a random number sequence sample and then verifying that that bit string did appear that often. This is a simple problem for classical computers to solve for a few random numbers but get’s increasingly hard the more random numbers you throw at it.

What is quantum supremacy?

Essentially, quantum supremacy states that a quantum computer can perform a function much faster than a standard digital computer. By much faster we are talking many orders of magnitude faster. The articles noted above state that the Google Quantum computer did the algorithm in minutes, that would take 1M cores, 10,000 years to perform on digital processors.

Quantum supremacy applies an algorithm at a time. There’s a class of problems that are considered NP (non-deterministic polynomial [time]) complete which can only be solved on classical, digital computers using brute force algorithms (e.g., algorithms that check every possible path) which can take forever for many path problem spaces (see NP-completeness, wikipedia)

Quantum computing, because of it’s inherent quantum ness characteristics, should be able to test every path of an NP complete problem in parallel. As a result a quantum processor should be able to solve any NP complete problem in a single pass.

Quantum supremacy simply states that such a quantum computer and algorithm have been implemented to solve an NP complete problem. The articles say that Google has created a quantum computer and algorithm to do just that.

Google’s Sycamore quantum computer

Google’s research was based on a new quantum computing chip, called the Sycamore processor, which supports a two dimensional array of 54 (transmon) qubits, where each qubit is “tunably” coupled to its nearest 4 neighbors. In the figure the “A” qubit can be coupled to it’s 4 nearest neighbor ‘a” qubits via tuning. 1 qubit failed, leaving 53 qubits for the computation.

The system is cooled to reduce noise and to enable super conductivity. The qubits are written by two mechanisms, a microwave drive to excite the qubit and a magnetic flux inducer to control its frequency. Qubits are read via a linear resonator which is connected to each qubit. All qubits can be read simultaneusly by a frequency multiplexing approach and are digitized to 8 bits @1GS/s (giga-samples/second, so 1GB/sec).

The Sycamore quantum architecture could generate 1113 single-qubit gate or 430 dual-qubit gate circuits. This inherently constrains the algorithmic complexity or problem state space that it could solve.

Qubit state, tunable couplers and quantum processor controls are written via 277 digital to analog converters in 14 bits [words] @1GS/s (or ~1.75GB/s). A single cycle of a Sycamore single-qubit circuit can be stimulated by a 25ns microwave pulse (with no qubit to qubit coupling active).

A key engineering advance of the Sycamore systems was achieving high-fidelity single- and dual-qubit operations, which means that they were completed correctly

What the Sycamore quantum computer accomplished

They generated 30M random numbers (with m=20 bits/random number) and then randomly sampled 1M of them, computed the probability of their occurrence and verified that that number was seen (within some error threshold). The random number generation, sampling and probability computation/verification took 200 seconds on the Sycamore system. They said that most of that activity was input and output and that the Sycamore quantum computer was only busy for 30 seconds working on the problem.

In comparison, the Google team benchmarked GCP processors and the Oak Ridge National Labs Summit supercomputer complex doing a portion of the algorithm using standard digital functionality and have concluded it would have taken 10K years on 1M processing cores to perform the same function using digital computers.

Although the problem to generate vast numbers of pseudo-random numbers, randomly sample them and verify they meet proper probability distributions may not be that useful, it does represent an NP-complete problem. Many NP-complete problems can be solved with the same algorithm, so there’s a good probability that similar NP complete problems may also be solvable with the Sycamore quantum computer.

1M cores for 10K years seems much too long

However, it doesn’t seem to me to take that long to generate pseudo random numbers, sample them and validate their probabilities were correct.

It took me ~1 sec to generate 10,000 (probably 32 bit but could be 64 bit) =RAND() numbers in Microsoft Excel, so 30M would take take about 15K 3K seconds on my desktop or about 4.2 hrs 50min.. Generating a random sample of 1M should take another 100 seconds (to generate another 1M random numbers), creating indexes out of these another 100 seconds or so, then accessing these 1M random numbers in the 30M random number list should take another 100 seconds (maybe) so add another 5 min to that. Verifying that the sampled numbers met all the random number probabilities it should ], is another question but here we are only dealing with 1M pseudo random numbers so it shouldn’t take that long to compute its frequency occurence and its probability to validate randomness. Yes, Excel =rand() function is probably not the best pseudo random number generator but it’s typical of what exists for digital computers and even if it took twice as long to generate better pseudo random numbers , it’s still not anywhere near 1M cores for 10K years.

But then again I’m no numerical specialist and even at 30 seconds vs 15.2K 3.3K + ? (for probability calculation + frequency verification), so multiply this by 5X or 16.5K seconds, makes the Sycamore quantum computer, ~1000X ~550Xfaster or ~2.5 orders of magnitude faster.

In my mind, Sycamore and its quantum algorithm is more of a proof of concept. The Sycamore processor proves that a) quantum computer with 53 qubits can solve some real world NP-complete problems and b) that the more qubits that quantum computing industry provides the more sophisticated NP complete problems it will be able to solve. Just not sure this is actually a 13 16 orders of magnitude speedup.

[My math was off but I tried to fix it in these last three paragraph, Ed.]

~~~~

Let’s see if a 20 bit NP complete problem can be solved in a quantum computer with 53 qubits, how many qubits should it take to solve a 256 bit NP complete problem (e.g. cracking an AES-256 bit encryption), maybe ~680 qubits. Phew, my AES encrypted data are safe, at least for the moment.

Photo Credit(s): From ScienceNews article about achieving quantum supremacy

From SpaceRef cached copy of report

Quantum computing NNs

As many who have been following our blog know, AI, Machine Learning (ML) and Deep Learning (DL) (e.g. see our Learning machine learning – part 3, & Industrial revolution deep learning & NVIDIA’s 3U supercomputer, AI reaches a crossroads posts), have become much more mainstream and AI has anointed DL as the best approach for pattern recognition, classification, and prediction, but has applicability beyond that.

One problem with DL has been it’s energy costs. There have been some approaches to address this, but none have been entirely successful (e.g. see Intel new DL Boost, New GraphCore GC2 chips, AI processing at the edge posts) just yet. At one time neuromorphic hardware was the answer but I’ve become disillusioned with that technology over time (see Are neuromorphic chips a dead end post).

This past week we learned of a whole new approach, something called a Quantum Convolutional NN or QCNN (see PhysOrg Introducing QCNN, pre-print of Quantum CNNs, presentation deck on QCNNs, Nature QCNN paper paywall).

Some of you may not know that convolutional neural networks (ConvNets) are the latest in a long line of DL architectures focused on recognizing patterns and classification of data in sequence. DL ConvNets can be used to recognize speech, classify photo segments, analyze ticker tapes, etc.

But why quantum computing

First off, quantum computing (QC) is a new leading edge technology targeted to solving very hard (NP Complete, wikipedia) problems, like cracking Public Key encryption keys, solving the traveling salesperson problem and assembling an optimum Bitcoin block problem (see List of NP complete problems, wikipedia).

QC utilizes quantum mechanical properties of the universe to solve these problems without resorting to brute force searches, such as, going down every path in the traveling salesmen problem (see our QC programming and QC at our doorsteps posts).

At the moment, IBM, Google, Intel and others are all working on the QC and trying to scale it up, by increasing the number of Qubits (quantum bits) their systems support. The more qubits, the more quantum storage you have, and the more sophisticated NP complete problems one can solve. Current qubit counts include: 72 qubits for Google, 42 for Intel, and 50 for IBM. Apparently not all qubits are alike, and they don’t last very long, ~100 microseconds (see Timeline of QC, wikipedia).

What’s a QCNN?

What’s new is the use of quantum computing circuits to create ConvNets. Essentially the researchers have created a way to apply AI DL (ConvNet) techniques to quantum computing data (qubits).

Apparently there are QC [qubit] phases that need to be recognized and what better way to do that than use DL ConvNets. The only problem is that performing DL on QC data with today’s tools, would require reading out the phase into a digital (a pattern recognition problem), converting to digital data, and then processing it via CPU/GPU DL ConvNets, a classic chicken or egg problem. But with QCNNs, one has a DL ConvNet entirely implemented in QC.

DL ConvNets are typically optimized for a specific problem, varying layer counts, nodes/layer, node connectivity, etc. QCNNs match this and also come in various sizes. Above is a QCNN circuit, optimized to recognize the phase (joining?) of two sets of symmetrically-protected topology numbers (SPT, see pre-print article).

I won’t go into the QC technology used in any detail (as I barely understand it), but the researchers have come up with a way to map DL ConvNets into QC circuitry. Assuming this all works, one can then use QC to perform DL pattern recognition on qubit data.

~~~~

Comments?

Photo Credits:

Your Rx on blockchain

We have been discussing blockchain technology for a while now (e.g., see our posts Etherium enters the enterprise, Blockchains go mainstream, and our podcast Discussing blockchains with Donna Dillenberger). And we were at VMworld 2019 where there was brief mention of Project Concord and VMware’s blockchain use for supply chain management.

But recently there was an article in Science Daily about the use of blockchain technology to improvedprescriptions. This was summary of a research paper on Cryptopharmacueticals (paper behind paywall).

How cryptopharmaceuticals could work

Essentially the intent is to use a medical blockchain, to fight counterfeit drugs. They have a proof of concept IOS & Android MedBlockChain app that would show how it could work, but it’s just a sample of some of its functionality, and doesn’t use an external blockchain.

The MedBlockChain app would create a platform and have at least two sides to it.

  • One side would be the phamaceutical manufacturers which would use the blockchain to add or checkin medications that they manufacture to it in an immutable fashion. So the block chain would essentially have an unfalsifiable record of each pill or batch of pills, that was ever manufactured by pharmaceutical companies around the world.
  • The other side would be used by a person taking a medication. Here they could check-out or use the app to see if the medication they are taking was manufactured by a certified supplier of the drug. Presumably there would be a QR code or something similar, that could be read off the medicine package or pill itself. The app would scan the QR code and then use MedBlockChain to look up the provenance of the medication to see if it’s valid or not (a fraudulent copy).

The example MedBlockChain app also has more medical information that could be made available on the block chain such as test results, body measurements, vitals, etc. These could all be stored immutably in the MedBlockChain and provided to medical practitioners. How such medical (HIPPA controlled) personal information would be properly secured and only supplied in plaintext to appropriate personnel is another matter..

~~~~
Cryptopharmaceuticals and the MedBlockChain reminds me of IBM’s blockchain providing diamond provenance and other supply chain services, only in this case applied to medications. Diamond provenance makes sense because of its high cost but drugs seem a harder market to make to me.

I was going to say that such a market may not exist in first world countries. But then I -saw a wikipedia article on counterfeit medicines (bad steroids and cancer medicines with no active ingredients). It appears that counterfeit/fraudulent medications are a problem wherever you may live.

Then of course, the price of medications seems to be going up. So maybe, it could start as a provenance tool for expensive medications and build a market from there.

How to convince manufacturers and the buying public to use the blockchain is another matter. It’s sort of a chicken and egg thing. You need the manufacturers to use it for medications, pills or batch that they manufacture. Doing so adds overhead, time and additional expense and they would need to add a QR code or something similar to every pill, pen or other drug delivery device.

Then maybe you could get consumers and medical practitioners administering drugs to start using it to validate expensive meds. Starting with expensive medications could potentially build the infrastructure, consumer/medical practitioner and pharmaceutical company buy in that would kick start the MedBlockChain. Once started there, it could work its way down to more widely used medications.

Comments?

Photo Credit(s):

Phonons, the next big technology underpinning integrated circuits

Often science and industry seems to advance from investigating phenomena that is a side effect of something else we want to try to accomplish. Optical fibers have been in use for over a decade now and have always had a problem called Brillouin scattering, where light photon’s interact with surrounding cladding and generate small vibrations or sound packets. This feedback causes light to disperse across the length of the fibre due to Brillouin scattering and create sound packets called phonons aka hyper sound.

As a recent article I read in Science Daily (Wired for sound a third wave emerges in integrated circuits) describes it, the first wave of ICs was based on electronics and was developed after WW II, the second wave was based on photons and came about largely at the start of this century, and now the third wave is emerging based on sound, phonons.

The research team at The University of Sydney, Nano Institute have published over 70 papers on Brillouin scattering and Prof. Benjamin J. Eggleton recently published a summary of their research in a Nature Photonics paper (Brillouin integrated photonics, behind paywall) but one can download the deck he presented as a summary of the paper, at an OSA Optoelectronics Technical Group webinar, last year..

It appears as if the Brillouin scattering technology is particularly useful for (microwave) photonics computing. In the Science Daily article, the professor says that the big advance here is in the control of light and sound over small distances. In the Sarticle, the Professor goes onto say that “Brillouin scattering of light helps us measure material properties, transform how light and sound move through materials, cool down small objects, measure space, time and inertia, and even transport optical information.”

I believe from a photonics IC perspective, transforming how light, other electromagnetic radiation, and sound move through materials is exciting. New technology for measuring material properties, cool down small objects, measure space, time and inertia are also of interest, but not as important in our view.

What’s a phonon

As discussed earlier, phonons are packets of sound vibration above 100mhz, that come about due to optical photons interaction with cladding. As photons bounce off the cladding they generate phonons within the material. Such bouncing creates optical and acoustical waves or phonons.

There’s been a lot of research on how to create “Stimulated Brillouin Scattering” (SBS) on silicon CMOS devices and still goes on, but lately they have found an effective hybrid (Silicon, SiO2, & As2S3) formula to generate SBS at will at chip scale.

What can you do with SBS phonons

Essentially SBS phonons can be used to measure, monitor, alter and increase the flow of electromagnetic (EM) waves in a substance or wave guide. I believe this can be light, microwaves, or just about anything on the EM spectrum. Nothing was mentioned about X-Rays, but it’s just another band of EM radiation.

With SBS, one can supply microwave filters, phase shifters and sources, recover carrier signal in coherent optical communications, store (or delay) light, create lasers and measure, at the sub-mm scale, optical material characteristics. Although the article discusses cooling down materials, I didn’t see anything in the deck describing this.

As SBS technologies are optical-acoustical devices, they are immune to EMI (electro- magnetic interference), EMPs (electro-magnetic pulses) and consume less energy than electronic circuits performing similar functions.

We’ve talked about photonic computing before (see our Photonic computing, seeing the light of day post). But to make photonics a real alternative to electronic computing they need a lot of optical management devices. We discussed a couple in the blog post mentioned above but SBS opens up another dimension of ways to control photonic data flow and processing.

Unclear why the research into SBS seems to be generated out of Australian Universities. However their research is being (at least partially) funded by a number of US DoD entities.

It’s unclear whether SBS will ultimately be one of those innovations in the long run, which enables a new generation of (photonic) IC technologies. But the team has shown that with SBS they can do a lot of useful work with optical/microwave transmission, storage and measurement.

It seems to me that to construct full photonic computing, we need an optical DRAM device. Storing light (with SBS) is a good first step, but any optical store/memory device needs to be randomly accessible, and store Kb, Mb or Gb of optical data, in chip size areas and persist (dynamic refreshing is ok).

The continued use of DRAM for this would make the devices susceptible to EMI, EMP and consume more energy. Maybe something could be done with an all optical 3DX that would suffice as a photonics memory device. Then it could be called Optical DC PM.

So, ICs with electronics, photonics and now phononics are in our future.

Comments?

Photo Credits:

Shedding light on all optical neural networks

Read a couple of articles in the past week or so on all optical neural networks (see All optical neural network (NN) closes performance gap with electronic NN and New design advances optical neural networks that compute at the speed of light using engineered matter).

All optical NN solutions operate faster and use less energy to inference than standard all electronic ones. However, in reality they aree more of a hybrid soulution as they depend on the use of standard ML DL to train a NN. They then use 3D printing and other lithographic processes to create a series diffraction layers of an all optical NN that matches the trained NN.

The latest paper (see: Class-specific Differential Detection in Diffractive Optical Neural Networks Improves Inference Accuracy) describes a significant advance beyond the original solution (see: All-Optical Machine Learning Using Diffractive Deep Neural Networks, Ozcan’s original paper).

How (all optical) Diffractive Deep NNs (DDNNs) work for inferencing

In the original Ozcan discussion, a DDNN consists of a coherent light source (laser), an image, a bunch of refractive and reflective diffraction layers and photo detectors. Each neural network node is represented by a point (pixel?) on a diffractive layer. Node to node connections are represented by lights path moving through the diffractive layer(s).

In Ozcan’s paper, the light flowing through the diffraction layer is modified and passed on to the next diffraction layer. This passing of the light through the diffraction layer is equivalent to the mathematical bias (neural network node FP multiplier) in the trained NN.

The previous challenge has been how to fabricate diffraction layers and took a lot of hand work. But with the advent of 3D printing and other lithographic techniques, nowadays, creating a diffraction layer is relatively easy to do.

In DDNN inferencing, one exposes (via a coherent beam of light) the first diffraction layer to the input image data, then that image is transformed into a different light pattern which is sent down to the next layer. At some point the last diffraction layer converts the light hitting it into classification patterns which is then be detected by photo detectors. Altenatively, the classification pattern can be sent down an all optical computational path (see our Photonic computing sees the light of day post and Photonic FPGAs on the horizon post) to perform some function.

In the original paper, they showed results of an DDNN for a completely connected, 5 layer NN, with 0.2M neurons and 8B connections in total. They also showed results from a sparsely connected, 5 layer NN ,with 0.45M neurons and <0.1B connections

Note, that there’s significant power advantages in exposing an image to a series of diffraction gratings and detecting the classification using a photo detector vs. an all electronic NN which takes an image, uses photo detectors to convert it into an electrical( pixel series) signal and then process it through NN layers performing FP arithmetic at layer node until one reaches the classification layer.

Furthermore, the DDNN operates at the speed of light. The all electronic network seems to operate at FP arithmetic speeds X number of layers. That is only if it could all done in parallel (with GPUs and 1000s of computational engines. If it can’t be done in parallel, one would need to add another factor X the number of nodes in each layer . Let’s just say this is much slower than the speed of light.

Improving DDNN accuracy

The team at UCLA and elsewhere took on the task to improve DDNN accuracy by using more of the optical technology and techniques available to them.

In the new approach they split the image optical data path to create a positive and negative classifier. And use a differential classifier engine as the last step to determine the image’s classification.

It turns out that the new DDNN performed much better than the original DDNN on standard MNIST, Fashion MNIST and another standard AI benchmark.

DDNN inferencing advantages, disadvantages and use cases

Besides the obvious power efficiencies and speed efficiencies of optical DDNN vs. electronic NNs for inferencing, there are a few other advantages:

  • All optical data paths are less noisy – In an electronic inferencing path, each transformation of an image to a pixel file will add some signal loss. In an all optical inferencing engine, this would be eliminated.
  • Smaller inferencing engine – In an electronic inferencing engine one needs CPUs, memory, GPUs, PCIe busses, networking and all the power and cooling to make it work. For an all optical DDNN, one needs a laser, diffraction layers and a set of photo detectors. Yes there’s some electronics involved but not nearly as much as an all electronic NN. And an all electronic NN with 0.5m nodes, and 5 layers with 0.1B connections would take a lot of memory and compute to support. Their DDNN to perform this task took up about 9 cm (3.6″) squared by ~3 to5 cm (1.2″-2.0″) deep.

But there’s some problems with the technology.

  • No re-training or training support – there’s almost no way to re-train the optical DDNN without re-fabricating the DDNN diffraction layers. I suppose additional layers could be added on top of or below the bottom layers, sort of like a corrective lens. Also, if perhaps there was some sort of way to (chemically) develop diffraction layers during training steps then it could provide an all optical DL data flow.
  • No support for non-optical classifications – there’s much more to ML DL NN functionality than optical classification. Perhaps if there were some way to transform non-optical data into optical images then DDNNs could have a broader applicability.

The technology could be very useful in any camera, lidar, sighting scope, telescope image and satellite image classification activities. It could also potentially be used in a heads up displays to identify items of interest in the optical field.

It would also seem easy to adapt DDNN technology to classify analog sensor data as well. It might also lend itself to be used in space, at depth and other extreme environments where an all electronic NN gear might not survive for very long.

Comments?

Photo Credit(s):

Figure 1 from All-Optical Machine Learning Using Diffractive Deep Neural Networks

Figure 2 from All-Optical Machine Learning Using Diffractive Deep Neural Networks

Figure 2 from Class-specific Differential Detection in Diffractive Optical Neural Networks Improves Inference Accuracy

Figure 3 from Class-specific Differential Detection in Diffractive Optical Neural Networks Improves Inference Accuracy

Where should IoT data be processed – part 1

I was at FlashMemorySummit 2019 (FMS2019) this week and there was a lot of talk about computational storage (see our GBoS podcast with Scott Shadley, NGD Systems). There was also a lot of discussion about IoT and the need for data processing done at the edge (or in near-edge computing centers/edge clouds).

At the show, I was talking with Tom Leyden of Excelero and he mentioned there was a real need for some insight on how to determine where IoT data should be processed.

For our discussion let’s assume a multi-layered IoT architecture, with 1000s of sensors at the edge, 100s of near-edge processing/multiplexing stations, and 1 to 3 core data center or cloud regions. Data comes in from the sensors, is sent to near-edge processing/multiplexing and then to the core data center/cloud.

Data size

Dans la nuit des images (Grand Palais) by dalbera (cc) (from flickr)
Dans la nuit des images (Grand Palais) by dalbera (cc) (from flickr)

When deciding where to process data one key aspect is the size of the data. Tin GB or TB but given today’s world, can be PB as well. This lone parameter has multiple impacts and can affect many other considerations, such as the cost and time to transfer the data, cost of data storage, amount of time to process the data, etc. All of these sub-factors include the size of the data to be processed.

Data size can be the largest single determinant of where to process the data. If we are talking about GB of data, it could probably be processed anywhere from the sensor edge, to near-edge station, to core. But if we are talking about TB the processing requirements and time go up substantially and are unlikely to be available at the sensor edge, and may not be available at the near-edge station. And PB take this up to a whole other level and may require processing only at the core due to the infrastructure requirements.

Processing criticality

Human or machine safety may depend on quick processing of sensor data, e. g. in a self-driving car or a factory floor, flood guages, etc.. In these cases, some amount of data (sufficient to insure human/machinge safety) needs to be done at the lowest point in the hierarchy, with the processing power to perform this activity.

This could be in the self-driving car or factory automation that controls a mechanism. Similar situations would probably apply for any robots and auto pilots. Anywhere some IoT sensor array was used to control an entity, that could jeopardize the life of human(s) or the safety of machines would need to do safety level processing at the lowest level in the hierarchy.

If processing doesn’t involve safety, then it could potentially be done at the near-edge stations or at the core. .

Processing time and infrastructure requirements

Although we talked about this in data size above, infrastructure requirements must also play a part in where data is processed. Yes sensors are getting more intelligent and the same goes for near-edge stations. But if you’re processing the data multiple times, say for deep learning, it’s probably better to do this where there’s a bunch of GPUs and some way of keeping the data pipeline running efficiently. The same applies to any data analytics that distributes workloads and data across a gaggle of CPU cores, storage devices, network nodes, etc.

There’s also an efficiency component to this. Computational storage is all about how some workloads can better be accomplished at the storage layer. But the concept applies throughout the hierarchy. Given the infrastructure requirements to process the data, there’s probably one place where it makes the most sense to do this. If it takes a 100 CPU cores to process the data in a timely fashion, it’s probably not going to be done at the sensor level.

Data information funnel

We make the assumption that raw data comes in through sensors, and more processed data is sent to higher layers. This would mean at a minimum, some sort of data compression/compaction would need to be done at each layer below the core.

We were at a conference a while back where they talked about updating deep learning neural networks. It’s possible that each near-edge station could perform a mini-deep learning training cycle and share their learning with the core periodicals, which could then send this information back down to the lowest level to be used, (see our Swarm Intelligence @ #HPEDiscover post).

All this means that there’s a minimal level of processing of the data that needs to go on throughout the hierarchy between access point connections.

Pipe availability

binary data flow

The availability of a networking access point may also have some bearing on where data is processed. For example, a self driving car could generate TB of data a day, but access to a high speed, inexpensive data pipe to send that data may be limited to a service bay and/or a garage connection.

So some processing may need to be done between access point connections. This will need to take place at lower levels. That way, there would be no need to send the data while the car is out on the road but rather it could be sent whenever it’s attached to an access point.

Compliance/archive requirements

Any sensor data probably needs to be stored for a long time and as such will need access to a long term archive. Depending on the extent of this data, it may help dictate where processing is done. That is, if all the raw data needs to be held, then maybe the processing of that data can be deferred until it’s already at the core and on it’s way to archive.

However, any safety oriented data processing needs to be done at the lowest level and may need to be reprocessed higher up in the hierachy. This would be done to insure proper safety decisions were made. And needless the say all this data would need to be held.

~~~~

I started this post with 40 or more factors but that was overkill. In the above, I tried to summarize the 6 critical factors which I would use to determine where IoT data should be processed.

My intent is in a part 2 to this post to work through some examples. If there’s anyone example that you feel may be instructive, please let me know.

Also, if there’s other factors that you would use to determine where to process IoT data let me know.

Smart proteins, the dawn of new intra-cell therapeutics

Saw an article the other day about Smart Cells (Incredible artificial proteins opens up potential for smart cells). There’s another paper that goes into a bit more depth than the original article (‘Limitless potential’ of artificial proteins ushers in new era of ‘smart’ cell therapies).

The two freely available articles talk about two papers in Nature (De novo design of bioactive protein switches & Modular and tunable biological feedback using a de novo protein switch, behind a paywall) that explain the technology in more detail.

Smart proteins act as switches

What the researchers have created is an artificial protein that builds a cage around some bioactive (protein based) mechanism that can be unlocked by another protein while residing in a cell. This new protein is called LOCKR (Latcheable Orthogonal Cage-Key pRotein). LOCKR proteins act as switch activated therapeutics within a cell.

In the picture above the blue coils are the cage proteins and the yellow coil is the bioactive device (protein). Bioactive devices can be designed that degrade other proteins, can modify biological processes within the cell, initiate the cells self-destruct mechanism, etc, just about anything a protein can do within a cell.

In the second Nature paper, they discuss one example of a LOCKR protein, called degron-LOCKR which once inside a cell is used to degrade (destroy) a specific protein. The degron-LOCKR protein only activates when the other protein is active (found) within the cell and it operates only as long as that protein is in sufficient concentration.

The nice thing about the degron-LOCKR protein is that is completely self-regulating. It only operates when the protein to be degraded exists in the cell. That protein acts as the switch in this LOCKR. Until then it remains benign, waiting for a time when the targeted protein starts to be present in the cell.

How LOCKR works

In the picture above the cage is in shown by the grey structure, the bio-active therapy is shown by the yellow structure, and the protein key is shown by the black structure. When the key is introduced into the LOCKR protein, the yellow structure is unfolded (enabled) and can then impact whatever intra-cellular process/protein, it’s been designed to impact.

One key attribute to LOCKR is that the bioactive device within the cage, can be just about anything that works inside the cell. It could be used to create more proteins, less proteins, disable proteins, and perhaps enhance the activity of other proteins.

And, both the LOCKR and the bioactive device can be designed from scratch and fabricated outside or inside the cell. Of course the protein key is the other aspect of the LOCKR mechanism that is fully determined by the designer of the LOCKR protein.

Sort of reminds me of the transistor. Both are essentially switches. For transistors, as long as voltage is applied, it will allow current to flow across the switch. LOCKR does something very similar, but uses a key protein and a bioactive protein that only allows the bioactive protein to activate when the key protein is present.

We’ve talked extensively in the past about using DNA/cells as rudimentary computers and storage, but this takes that technology to a whole other level, (please see our DNA computing series here & here as well as our posts on DNA as storage here & here ). And all that work was done without LOCKR. With LOCKR much of these systems would be even easier to construct and design.

The articles go on to say that LOCKR unleashes the dawn of a new age of intra-cell therapeutics with fine grained control over when and where a particular bio-active therapy is activated within the cell

Some questions

Some of these may be answered in the Nature papers (behind paywall), so sorry in advance, if you have access to those.

How the LOCKR protein(s) are introduced into cells, was not discussed in the freely available articles. We presume that DNA designed to create the LOCKR protein could be injected into cells via a virus, added to the cells DNA via CRISPR, or the LOCKR protein could just be injected into the cell.

Moreover, how LOCKR proteins are scaled up within the cell to be more or less active and scaled up throughout an organ to “fix” multiple cells is yet another question.

Adding artificial DNA or LOCKR proteins to cells may be easy in the lab, but putting such therapy into medical practice will take much time and effort. And any side effects of introducing artificial DNA or LOCKR proteins (not found in nature) to cells will need to be investigated. And finally how such protein technology impacts germ lines would need to be fully understood.

But the fact that the therapeutic process is only active when unlocked by another key protein makes for an intriguing possibility. You would need both the LOCKR protein and the key (unlock-er) protein to be present in a cell for the therapy to be active.

But they present one example, the degron-LOCKR, where the key seems to be a naturally active protein in a cell that needs to be degraded, not a different, artificial protein introduced into the cell. So the key doesn’t have to be an artificial protein and probably would not be for most LOCKR designed proteins.

~~~~

Not a bad start for a new therapy. It has much potential, especially if it can be scaled easily and targeted specifically. Both of which seem doable (given our limited understanding of biological processes).

Comments?

Picture Credit(s): From De novo design of bioactive protein switches article

From Limitless potential… article