Low power spiking AI-Arm edge processing from voltage scaling on ETA TENSAI chip

Read an article in IEEE Spectrum (ETA Compute debuts spiking NN chip for edge AI) about a company producing a new AI IOT chip based on ARM microprocessors with DSPs for dedicated matrix computations.

The new ETA Compute TENSAI chip  technology supports spiking neural networks (NN) as well as more normal, convolutional NN  depending on  edge AI requirements. More  information can be found in an Embedded Computing article (Micropower intelligence for edge devices) and a EE News article (ETA adds spiking NN support to MCU)

We have discussed spiking NN  in prior posts  latest post: IBM using PCM for better AI -round 6). Any spiking NN more closely mimics real biological neurons present in the brains of humans and other life.

ETA also claims that spiking NN perform better unsupervised learning. They included some examples of this in videos. In one video, ETA trains a spiking NN to do the same job as a convolutional NN with 1/10th the pixel data.

In addition, spiking NN only use neural weight values of 0 or 1, whereas normal convolutional NN operations require 8 to 16 bit numbers. So spiking NN arithmetic really only uses addition while convolutional NN need 8-16 bit multiplicative arithmetic to determine resultant weights.

Voltage scaling saves power

The other claim is that the TENSAI chip can perform computations on the order of 10 microwatts of power per megahertz (~10 μW/MHz) using the ARM/DSP combination with voltage scaling

The new TENSAI chip is their 3rd generation with multiple ARM M3 cores and NXP digital signal processors. These cores are implemented in a sub-threshold, asynchronous mode process that allows them to operate at much lower voltage, ~0.2V and at varying clock frequencies. This is called voltage scaling electronics. Doing this took analog design, which ETA considers part of their special IP.

ETA claims the TENSAI chip can operate in listen mode (“Ok Google?”) and only consume 50 microwatts of power and once a key word is discovered, operate in full computational mode with 500 microwatts of power.

I couldn’t find a data sheet for the TENSAI chip, so was unable to see how many TOPS/W it could perform as discussed in prior posts (see: AI processing at the edge post). But it looks to be even more power efficient.

The TENSAI chip was announced at ARM Tech Con(terence) and won the Design Innovation of the Year award at the conference.

Comments?

Photo Credits: Photo and caption from Figure 12 in  A brain inspired architecture…,  AIP Journal of Applied Physics article

Photo from EE News ETA adds spiking NN support to MCU article

Chart from Embedded Computing  Micropower intelligence for edge devices article

Board photo from IEEE Spectrum ETA Compute debuts spiking NN chip for edge AI article

Vacuum tubes on silicon

Read an interesting article the other day about researchers at NASA having invented a vacuum tube on a chip (see ExtremeTech, Vacuum tube strikes back). Their report was based on an IEEE Spectrum article called Introducing the Vacuum Transistor.

Computers started out early in the last century being mechanical devices (card sorters), moved up to electronic sorters/calculators/computers with vacuum tubes and eventually transitioned to solid state devices with the silicon transistor. Since then the MOS and CMOS transister have pretty much ruled the world of electronic devices.

Vacuum tube?

Vacuum tubes had a number of problems not the least of which was power consumption, size and reliability. It was nothing for a vacuum tube to burn out every couple of times it was powered on and the ENIAC (panel pictured here) had over 17,000 of them, took over 200 sq meters of space, used a lot (150KW) of power and weighed (27 metric) tons.

Of course each vacuum tube was the equivalent of just one transistor and the latest generation Intel Quad Core processors have over 2B transistors in them. So to implement an Intel Quad Core processor with vacuum tubes this might take over 3,000 football fields of space and over 17GW for power/cooling.

There were plenty of niceties with vacuum tubes not the least of which was their nice ruler flat frequency response, ability to support much higher frequencies, significantly less prone to noise and had less problems with radiation than transistors.  This last item meant that vacuum tubes were less susceptible to electromagnetic pulses. Many modern musical/instrument amplifiers are still made today using vacuum tube technology due to their perceived better sound.

But the main problems was their size and power consumption. If you could only shrink a vacuum tube to the size of a MOS field effect transistor (FET) and correspondingly reduce its power consumption, then you would have something.

NASA shrinks the vacuum tube

NASA researchers have shrunk the vacuum tube to nanometer dimensions in a vacuum- channel transistor. They believe it can be fabricated on standard CMOS technology lines and that it can operate at 460GHz. 

This new vacuum-channel transistor marries the benefits of vacuum tubes to the fabrication advantages of MOSFET technology. Making them as small as MOSFET transistors eliminates all of the problems with vacuum tube technology and handily solves a serious problem or two with MOSFETs.

07OLVacuumtransistors-1403115198821

One problem with MOSFET technology today is that we can no longer speed it up any faster than a 4-5GHz.  This limit was reached in 2004 when Intel and others determined that clock speed couldn’t be sped up much more without serious problems resulting and as a result, they started using additional transistors to offer multi-core processor chips.  A lot of time and money is continuing to be spent on seeing how best to offer even more cores but in the end there’s only so much parallelism that can be achieved in most applications and this limits the speed ups that can be attained with multi-core architectures.

But a shrunken vacuum tube doesn’t seem to have the same issues with higher clock speeds.  Also, there is a serious reduction in power consumption that accrues along with reduction in size.

The vacuum in a vacuum tube was there to inhibit electrons from being interfered with by gases. With the vacuum-channel transistor they don’t think they need a vacuum anymore due to the reduction of size and power being used but there’s a little problem on how to creating a helium filled enclosure which they feel will work instead of a vacuum. NASA feels that with todays chip packaging this shouldn’t be a problem.

Also, their current prototypes use 10V but other researchers have reduced other vacuum-channel transistors to use only 1-2v. As of yet the NASA researchers haven’t fabricated their vacuum-channel transistors on a real CMOS line but that’s the next major hurdle.

Imagine a much faster IT

A 400GHz processor in your desktop and maybe a 200GHz processor in your phone/tablet could all be possible with vacuum-channel transistors. They would be so much faster than today’s multi-core systems, that it would be almost impossible to compare the two. Yes there are some apps where multi-core could speed things up considerably but something that’s 10X faster than todays processors would operate much faster than a 10 core CPU. And it still doesn’t mean you couldn’t have multi-core vacuum-channel systems as well.

SSD or NAND flash storage is essentially based on CMOS transistors and the speed of flash is a somewhat of a function of the speed of its transistors.  A 400GHz vacuum-channel transistor could speed up flash storage by an order of magnitude or more. Flash access times are already at the 7µsec level (see my posts on MCS and UltraDIMM storage here and here).  How much of that 7µsec access time is due to the memory channel aand how much is a function of the SanDisk SSD storage is an open question. But whatever portion is on the SSD side could be potentially reduced by a factor of 10 or more with the use of vacuum-channel transistors.

From a disk perspective there are myriad issues that effect how much data can be stored linearly on a disk platter. But one of them is the speed of switching of electromagnetic  (GMR) head and the electronics. Vacuum-channel transistors should be able to eliminate that issue at least in the electronics and maybe with some work in the head as well so disk densities would no longer have to worry about switching speeds. Similar issues apply to magnetic tape densities as well.

Unclear to me how faster switching time would impact network transmission speeds. But it seems apparent that optical transmission times have already reached some sort of limit based on light frequencies used for transmission. However, electronic networking transfer speeds may be able to be enhanced significantly with faster speed switching.

Naturally, WIFI and other forms of radio transmission are seriously impeded by the current frequency and power of electronic switching. That’s one of the reasons why radio stations still depend somewhat on vacuum tubes. However, with vacuum-channel transistors problems with switching speed go away.  Indeed, NASA researchers believe that their vacuum-channel transistors should be able to reach terahertz (1000GHz) transmission switching. Which might make WIFI almost faster than any direct connect networking today.

~~~~
Comments?

Photo Credit(s): ENIAC panel (rear) by Erik Pittit, The Vacuum Tube Transistor from IEEE Spectrum

DC is back

Power sub distribution board by Tom Raftery (cc) (from Flickr)
Power sub distribution board by Tom Raftery (cc) (from Flickr)

Read an article in this month’s IEEE Spectrum on providing direct current (DC) power distribution to consumers.  Apparently various groups around the world are preparing standards to provide 24-V DC and 380-V DC power distribution to home and office.

Why DC is good

It turns out that most things electronic today (with the possible exception of electro-magnetic motors) run off DC power.  In fact, the LED desklamp I just purchased has a converter card in the plug adapter that converts 120-V alternating current (AC ) to 24-V DC to power the lamp.

If you look at any PC, server, data storage, etc., you will find power supplies that convert AC to DC for internal electronic use.  Most data centers take in 480-V AC which is converted to DC to charge up uninterruptible power supply batteries that discharge DC power that is  converted back to AC which is then converted internally back to DC for server electronics.  I count 3 conversions there: AC to DC, DC to AC and AC to DC.

But the problem with all this AC-DC conversion going on, is that it takes energy.

The War of Currents or why we have AC power today

Edison was a major proponent of DC power distribution early in the history of electronic power distribution. But the issues with DC or even AC for that matter is that voltage is lost over any serious line distances which required that DC generation stations of the time had to be located within a mile of consumers.

In contrast, Tesla and Westinghouse first proposed distributing AC power because of the ability to convert high voltage AC to low voltage using transformers.  To see why this made a difference read on, …

It turns out the major problem with the amount of line loss depends on the current being transmitted.  But current is only one factor in the equation that determines electrical power, the other factor being voltage.  You see any electrical power level can be represented by high current-low voltage or low current-high voltage.

Because AC at the time could easily be converted from high to low voltage or vice versa, high voltage-low current AC power lines could easily be converted (locally) to low voltage-high current power lines.   A high voltage-low current line lost less power and as AC voltage could be converted more easily, AC won the War of Currents.

Move ahead a century or so, and electronics have advanced to the point that converting DC voltage is almost as easy as AC today.  But more to the point, with today’s AC distribution, changing from lot’s of small, individual AC to DC and DC to AC converters in each appliance, server, UPS, etc., can be better served by a few, larger AC to DC converters at the building or household level, improving energy efficiency.

Where DC does better today

Batteries, solar panels, solid state electronics (pretty much any electronic chip, anywhere today), and LED lighting all operate on DC power alone and in most cases convert AC to DC to use today’s AC power distribution.  But by having 24-V or 380-V DC power in the home or office, it would allow these devices to operate without converters and be more efficient.  The Spectrum article states that LED lighting infrastructure can save up to 15% of the energy required if it was just powered by DC rather than having to convert AC to DC.

However, with the industry standards coming out of Emerge and European Telecommunications Standards Institute we may have one other significant benefit.  We could have one worldwide plug/receptacle standard for DC power.  Whether this happens is anyone’s guess, and given today’s nationalism may not be feasible.  But we can always hope for sanity to prevail…

Comments?