Saw a recent article that discussed IBM’s research into new computing architectures that are inspired by brain computational techniques (see A new brain inspired architecture … ). The article reports on research done by IBM R&D into using Phase Change Memory (PCM) technology to implement various versions of computer architectures for AI (see Tutorial: Brain inspired computation using PCM, in the AIP Journal of Applied Physics).
As you may recall, we have been reporting on IBM Research into different computing architectures to support AI processing for quite awhile now, (see: Parts 1, 2, 3, 4, & 5). In our last post, More power efficient deep learning through IBM and PCM, we reported on a unique hybrid PCM-silicon solution to deep learning computation.
Readers should also be familiar with PCM as well as it’s been discussed at length in a number of our posts (see The end of NAND is near, maybe; The future of data storage is MRAM; and New chip architectures with CPU, storage & sensors …). MRAM, ReRAM and current 3D XPoint seem to be all different forms of PCM (I think).
In the current research, IBM discusses three different approaches to support AI utilizing PCM devices. All three approaches stem from the physical characteristics of PCM.
(Some) PCM physics
It turns out that PCM devices have many characteristics that lend themselves to be useful for specialized computation. PCM devices crystalize and melt in order to change state. The properties associated with melting and crystallization of the PCM media cell can be used to support unique forms of computation. Some of these PCM characteristics include::
- Analog, not digital memory – PCM devices are, at the core, an analog memory device. We mean that they don’t record just a 0 or 1 (actually resistant or conductive) state, but rather a continuum of values between those two.
- PCM devices have an accumulation capability – each PCM cell actually accumulates a level of activation. This means that one cell can be more or less likely to change state depending on prior activity.
- PCM devices are noisy – PCM cells arenot perfect recorders of state chang signals but rather have a well known, random noise which impacts the state level attained, that can be used to introduce randomness into processing.
The other major advantage of PCM devices is that they take a lot less power than a GPU-CPU to work.
Three ways to use PCM for AI learning
The Applied Physics article describes three ways to use PCM devices in AI learning. These three include:
- Computational storage – which uses the analog capabilities of PCM to perform arithmetic and learning computations. In a sort of combined compute and storage device.
- AI co-processor – which uses PCM devices, in an “all PCM nodes connected to all other PCM nodes” operation that could be used to perform neural network learning. In an AI co-processor there would be multiple all connected PCM modules, each emulating a neural network layer.
- Spiking neural networks – which uses PCM activation accumulation characteristics & inherent randomness to mimic, biological spiking neuron activation.
It’s the last approach that intrigues me.
Spiking neural nets (SNN)
Biological neurons accumulate charge from all input (connected) neurons and when they reach some input threshold, generate an output signal or spike. This spike is then used to start the process with another neuron up stream from it
Biological neurons also exhibit randomness in their threshold-spiking process.
Emulating spiking neurons, n today’s neural nets, takes computation. Also randomness takes more.
But with PCM SNN, both the spiking process and its randomness, comes from device physics. Using PCM to create SNN seems a logical progression.
PCM as storage, as memory, as compute or all the above
In the storage business, we look at Optane (see our 3D Xpoint post) SSDs as blazingly fast storage. Intel has also announced that they will use 3D Xpoint in a memory form factor which should provide sadly slower, but larger memory devices.
But using PCM for compute, is a radical departure from the von Neumann computer architectures we know and love today. HPE has been discussing another new computing architecture with their memristor technology, but only in prototype form.
It seems IBM, is also prototyping hardware done this path.
Welcome to the next computing revolution.
Photo & Caption Credit(s): Photo and caption from Figure 2 in AIP Journal of Applied Physics article