Hewitt Crane was a practical minded kind of guy. To help the world get a better feel for just how much oil it used in a year, he came up with the unit he called the cubic mile of oil (CMO) to considerable acclaim. Crane was actually one of the pioneers of computing. He was an early developer of magnetic core RAM, eye-tracking devices, pen input devices, and invented the first all-magnetic computers still finding extensive use for fail-safe systems in the military. Today, another kind of device he presciently envisioned back in 1960 is starting to attract attention – the neuristor.
A neuristor is the simplest possible device that can capture the essential property of a neuron – that is, the ability to generate a spike or impulse of activity when some threshold is exceeded. A neuristor can be thought of as a slightly leaky balloon that receives inputs in the form of puffs of air. When its limit is reached, it pops. The only major difference is that more complex neuristors can repeat the process again and again, as long as spikes occur no faster than a certain recharge period known as the refractory period.
A neuristor uses a relatively simple electronic circuit to generate spikes. Incoming signals charge a capacitor that is placed in parallel with a device called a memristor. The memristor behaves like a resistor except that once the small currents passing through it start to heat it up, its resistance rapidly drops off. When that happens, the charge built up on the capacitor by incoming spikes discharges, and there you have it – a spiking neuron comprised of just two elementary circuit elements.
The details of trying to build neuristors have been largely unglamorous. In fact, most people trying to create artificial brains with neuristor-like elements have been content to do so by simulating them on regular old digital computers. The Spaun computer comprised of 2.5 million neurons is a recent example.
A group at HP Labs has come up with an improved method of fabricating a neuristor made from niobium dioxide (NbO2). For now, their prototype devices are too inefficient to be used in large numbers on a single chip, but the group is developing ways to make them more compatible with current fabrication methods. This new study does not deny the long-standing efforts of the neuromorphic community to build analogue neurons into chips, it is just the next step towards getting people on board with neuristor hardware instead of neuristor simulation.
When spikes beat bits
In a good electronics course, the first and simplest computer you usually build is an analogue device that uses just a handful of simple amplifier chips to map and solve a rudimentary equation. The fact that digital computers waste a lot of energy just to do a small amount of computation was not lost on the energy-conscious mind of Hewitt. Taking a cue from the brain, he formulated ideas for building devices that would exchange spikes rather than bits. In principle, for spiking networks, the only thing that absolutely requires energy is the spike. A spike doesn’t need any ordered reference structure like a bit – it can become whatever is desired.
A group of fireflies exchanging flashes, crows cawing, or locusts emitting seasonal calls, are all examples of spiking networks. The individual integrates these spikes of information, emitted in the local currency by its peers, and feeds spikes back to the system as the behaviour of the group evolves. What the spikes eventually come to represent is in many ways indefinably unique to the particular unit that generates them.
While for machines this would make them a bit more difficult to interpret or reproduce, it makes them infinitely easier to program – the hardware is the software. If it becomes apparent that the system needs to internalise more information about say, a visual scene, it just tosses in some more neuristors and lets them competitively integrate or die according to the properties of the network.
When a newborn first opens their eyes to the world, those first rays of light modify the founding circuitry of the retina and brain that has been roughed out according to the genes. Initially random, spiking activity begins to acquire meaning relative to the external world. In this way, spikes can be considered a baggage-free way to represent sensory input or motor output, among other things.
It is certainly possible to represent bits with spikes, and spikes with bits, but the conversion is usually imperfect. For example, if we put an obstacle in the way of a fly in an experimental setup, the fly can make an adjustment to its flight path in about 30ms. A single neuron in the fly brain detects the visual stimulus captured by photoreceptors contacting its input neurites, and signals the flight muscle contacted directly by its output neurites. In this context we might then ask, how much information in bits per second is transmitted by this neuron?
If for the sake of argument, we suppose that this particular neuron can spike at a maximum of 500Hz, it follows that in the 30ms behavioural window there would be 15 intervals, each 2ms long, where a spike may or may not occur. There are then 2 to the power of 15, or 32768 possible spike “words” that the neuron could in theory transmit. The bit rate of this neuron, at least in the given window would therefore be 15 bits per 30ms, or more conveniently, 500 bits per second. That number is just what the neuron is transmitting to the flight muscle in its termination zone. In fact each of the neuron’s 100 or more input neurites may be said to be integrating information at a roughly similar bit rate, we just cannot map a real world function to them as easily at that level of detail.
In principle the requirement that spikes fall within windows is artificial. Bats can reportedly detect Doppler shifts in echoes down to the nanosecond level using spikes. Spikes with that kind of temporal precision, particularly when considered in the context of a network, could have much higher corresponding bit rates.
Brains, or in particular neurons, use spikes in entirely different ways depending on what kind of job the neuron does and what kind of network it is in. Neurons controlling the eye muscles need to fire quickly and often to maintain eye position. On the other hand, neurons that control the movements of food through the gut need to spike rhythmically on cue and be able to put out a constant volley of spikes at each peak in the rhythm to drive the gut muscle properly – these neurons might be likened to performing the “wave” much like you would do at a football game.
Are neuristors the future of prosthesis and robotics?
If HP Labs and other research groups can get neuristor efficiency down to the level that neurons routinely operate, things may start to get a little more exciting. For one thing, when we get around to making devices safe enough to interface directly with the brain, neuristors already speak the local language, albeit one of rigid dialect. Smartphone sensors which represent data with spikes will be prosthesis-compatible out of the box, and sip juice from the battery at barely perceptible levels.
Robots perhaps stand to gain the most. The Disney robot which can play a fair game of catch with a human requires a good bit of processing power to perform. It must process encoder signals in its joints, likely at 10-bit or higher resolution, digitally solve an analogue servo loop, and then generate an analogue voltage or PWM (Pulse Width Modulation) control signal to drive its servomotors. If instead the entire job could be done with a handful of analogue spiking neurons, much like a spinal loop of its human partner, there would be incredible gains in efficiency and speed.
If the network is rich enough, all the trainer might do is arrange things so that the total energy needed by the system is driven towards the minimal condition within the context of the desired goal, and the robot might eventually teach itself the task. For now, that is a whole lot easier said than done. One thing that neuristors on neuromorphic chips can’t do today is grow. They can change their strengths of interconnection, but they cannot actually seek out new connections according to any underlying principle. Growth on these chips might one day be approximated if sufficient latent connections can be integrated on the chip and activated when needed, but those capabilities are just beginning to be imagined.
Some readers may have been slightly irked that while the neuristor processes an analogue input, its spike output sure looks like a digital event. That observation is certainly correct, but bear in mind that spikes are not bits. They are a different animal altogether, something in between, perhaps akin to an extreme analogue signal, compressed at its critical points. If chips like those envisioned by HP Labs can be put into the hands of many experimenters, the larger community will become more familiar with these new ways to sense, compute, and control – inviting an era of analogue computing to recommence.
Research paper: doi:10.1038/nmat3510 – “A scalable neuristor built with Mott memristors”
Published under license from Ziff Davis, Inc., New York, All rights reserved.
Copyright © 2012-2013 Ziff Davis, Inc