It resembles the strung-out entrails of a computer: five circuit boards, connected by an intestinal tangle of wires. A lens peers out at Kwabena Boahen, the creator of the device, from one of the boards, and on a screen nearby the blocky pixels of his face nod and gesture.
Though the chips mounted on these circuit boards are the same sort used to build everything from iPods to supercomputers, that’s where the similarity ends. This machine can’t run a word processor or let you email, but it can do things most desktop computers find virtually impossible to do in real time: it can see.
That’s not to say it is conscious, of course, but what it can do is organise raw optical stimuli into a useful representation of the thing it’s “looking” at, and identify the outlines of different objects in its field of view. Instead of mindlessly number-crunching like an ordinary computer, these chips are physically mimicking the electrical behaviour of the nerve cells found in a lemon-sized wedge of your brain called the primary visual cortex.
Advertisement
Kwabena Boahen, now at Stanford University in California, and his colleagues at the University of Pennsylvania in Philadephia, from where he recently moved, have built this device to investigate how brains work, and it lets them experiment in ways that neuroscientists working with the real thing can only dream of. For example, in real mammalian brains you can’t simultaneously trace the activity of many individual neurons in real time, or adjust the connections between thousands of neurons to see what happens. Boahen can do all this and more with his artificial neurons, and he’s using this ability to a remarkable end. “I want to figure out how the brain works in a very nuts-and-bolts way,” he says. “I want to figure it out such that I can build it.”
“I want to figure out how the brain works such that I can build it”
This approach in itself isn’t new. Boahen’s team is one of several that have already built an artificial retina. But the next stage in the evolution of these chips is nothing short of revolutionary: Boahen wants to generalise the chip’s function to build a silicon version of the brain’s cerebral cortex – the all-purpose tissue that has functions as diverse as recognising faces, processing language, and memory. Such a material could be our best shot at building neural prostheses for people whose brains have been damaged by injury or disease. If you could get a silicon implant to communicate with squishy, living neurons it might help, say, a stroke patient regain their sense of direction. One day, these “neuromorphic” implants might even boost the brainpower of healthy people.
The idea of mimicking networks of neurons in hardware grew out of decades-old research into how they manage to create the amazing abilities of the human brain. Researchers devised mathematical algorithms to simulate real neurons in software. Simple artificial neural networks have ended up in everyday gadgets, doing tasks such as controlling the autofocus on digital cameras, but they bear little resemblance to real neural networks. That’s because to ensure these simulations aren’t grindingly slow a lot of the detail is missed out, or coarsely approximated. To emulate real neurons faithfully would mean solving hundreds of millions of equations every second. Computers – even supercomputers – would get hopelessly bogged down.
The problem stems from the gulf between how digital computers and living brains process information. Conventional computers operate on bits, strings of 1s and 0s that encode the input, and the outputs are yet more strings of binary digits. Inside the computer, the building blocks of every microchip are transistors, electronic switches that operate in such a way that they are either on or off. It is these two states that are used to represent the 1s and 0s, with transistors switching states depending on the on/off status of those that feed into them.
Furthermore, to make sure these operations all occur in the right order and at the right time, computers contain a central clock whose pulses are used to synchronise the switching of the transistors. So for example, the 2-gigahertz Pentium computer on your desktop contains a clock that beats 2 billion times per second to synchronise its transistor-level operations. This is all great for doing clear-cut computing such as word processing or high-speed arithmetic, but almost useless when it comes to the complex tasks we humans take for granted, such as recognising objects in a scene or picking out a single voice from a jumble of voices in a crowd. That’s because it is just not how our brains work.
Neurons are electrical devices too, but not in the digital on/off manner of transistors; they are analogue, meaning they can produce output voltages not just faint or strong, but all points in between, and likewise can respond to a continuous range of input voltages. So instead of being either on or off, the voltages travelling in the brain span a continuum.
On top of this, there’s no central clock in our head to synchronise events. Instead it is the real-time simultaneity of inputs that causes voltages to spike through our brains. Each neuron can monitor up to several thousand inputs from other neurons and might only fire a pulse onwards if the sum total of all the input voltages at a particular moment in time passes a certain threshold. Emerging somehow out of this web of connections is our brain’s ability to process information in a radically different way to conventional computers, a way that no one yet fully understands.
In the late 1980s, Carver Mead, a microchip pioneer at the California Institute of Technology in Pasadena, came up with a radical idea: if we want to make computers behave like the brain in real time, why not forget trying to program software and build hardware equivalents of neurons.
Mead realised he could do this with the same transistors used to build digital computers. Transistors can operate in two modes, or phases. In the first, an operating voltage above a certain threshold switches the transistor on or off. In the second phase, at lower voltages, the transistor acts like an amplifier and produces a magnified output current proportional to the input voltage.
Analogue breakthrough
Digital processors use transistors in their on/off switching phase. Mead realised that by using transistors in their analogue amplifier phase instead, he could build circuits that accurately mimicked the electrical behaviour of real neurons. Using transistors in this way also meant Mead could dispense with the central clock altogether, dramatically cutting the power demand. As long as the input signals arrived within a few milliseconds of each other, the circuit of transistors imitating a given neuron would sum the input values, and if that tipped over a certain threshold, would produce an output spike. “It’s totally foreign to the way we’ve built computers for the last 40 years,” says Boahen.
“It’s a totally new way to build a computer”
Mead’s original silicon neurons were rather crude affairs that provided only a rough approximation to the electrical behaviour of real neurons. Things have come a long way since. Based on a detailed analysis of the visual centres in various animal brains, Boahen’s team has built silicon chips that mimic the mammalian retina and primary visual cortex in far greater detail than any previous model. Their newest artificial retina chip can see images with a resolution of 96 × 60 pixels. That’s still 200 times coarser than the human retina, but it’s enough to make out a person’s face, says Boahen, and sometimes even distinguish between one face and another.
Photosensitive transistors on the artificial retina’s surface convert incoming light into analogue voltages with a value that depends on the intensity of the light and which last for as long as light is shone on the transistors. These signals then get fed through to the artificial retinal neurons. Here motion and areas of contrast are identified, which indicate edges of objects in the image. To save valuable time and power, only these useful tidbits – a tiny fraction of the total information reaching the artificial retina – are then forwarded to the next stage of processing: the neuromorphic visual cortex chips. These chips consume several thousand times less power than a digital computer would running a software simulation of the same number of neurons. Modelled on the brain’s primary visual cortex, these chips process the information about edges and motion in the visual scene, and then assemble these signals into the outlines of objects. Boahen’s latest vision chips cram 8000 “neurons” onto 10 square millimetres of silicon. When the artificial retina sees a circle or some other shape, the neuromorphic chips pick out its edges just like the real visual cortex does.
Building a neuromorphic visual cortex chip is an impressive start, but the real promise – and challenge – lies in developing the technology in these chips into a general-purpose silicon version of the cerebral cortex. “[The] cortex has this computational engine which it can tune to particular tasks like vision, recognising faces, or deciding what to have for lunch,” says Rodney Douglas of the Institute of Neuroinformatics at the Swiss Federal Institute of Technology in Zurich. It has the same general blueprint, regardless of the function it carries out, he says. In contrast, today’s neuromorphic chips are task-specific: neuroscientists still haven’t cracked the underlying language of electrical pulses that neurons use. If we can decode it, in future we may see generalised cortical chips that can serve a range of functions.
However, it’s not enough to replicate the way individual neurons behave. Neuromorphic chips that truly emulate the cortex would need to constantly adapt the many connections between silicon neurons. One of the keys to the brain’s amazing adaptability is the plasticity of the myriad connections between neurons. A cortical implant designed to replace damaged tissue and work in harmony with the brain wouldn’t be preprogrammed like a normal computer. Instead, it would have to create and adapt its connections as it went along, just as a baby’s brain develops as it learns to interact with the world.
In its first few months inside the patient’s brain, the implant would adapt according to the signals coming from neighbouring real neurons. The connections between the silicon neurons would adjust themselves until the chip began to emulate the area of the brain it replaced. “It’s as if you were given an Intel processor with 80 per cent of its wires in place,” says Douglas, “and the last 20 per cent grow themselves depending on whether you put it into a Mac or a PC or whatever. Think of what a cool technology that would be.”
Cool, but practically impossible to do with existing hardware. The facts are sobering: each cubic millimetre of living cortex contains 4 kilometres of axons (the thread-like extensions that neurons use for long-range communication), and each neuron touches around 10,000 of its neighbours. So a chip with 10,000 neurons would need 100 million potential connections between them – perhaps several hundred metres of wires – and those connections would need to be able to adapt over time.
It’s a hard problem, and so far no one has found an answer. So for now, Boahen, Douglas, and others are relying on old digital computers to oversee the database of connections between their silicon neurons. When a silicon neuron spikes, a parallel digital system looks up its entry in the database to determine which neurons the spike should be routed on to, and this is then done using a grid of wires that connects to each neuron. Changing connections simply involves updating the database.
Boahen has used this arrangement to mimic the way our visual system rearranges connections during prenatal development. As the brain develops, neurons in an early visual area called the tectum emit a hormone called neurotropin, which helps create connections to the axons of neurons in the retina. This is how the connections involved in vision are formed in the right way to avoid scrambling the scene in your visual field as it is relayed deeper into the brain.
Instead of hormones, when silicon neurons on Boahen’s tectum chip fire, they leak a small amount of charge onto a grid of wires overlying the chip. The chip samples the leaked charge in several points across the grid and can thus pinpoint its source, revealing the silicon neuron that a connection should be established to. Boahen tested the idea by connecting his retina chip to his artificial tectum in such a way that the connections between the artificial neurons on the two chips were scrambled. He then stimulated the retina with flashes of light, which in turn stimulated the tectum chip. Sure enough, the two chips autonomously reorganised their connections.
It’s a proof of concept for the kind of flexibility that prosthetics will surely need. No surgeon can position an implant precisely enough to ensure that neurons in the patient’s brain connect to the right silicon neurons on a chip. A chip that reroutes its connections might be the only way to solve that problem.
Paul Hasler at the Georgia Institute of Technology in Atlanta is working on a separate solution to adaptability that could replace the need for the connectivity database. If a junction between living neurons – called a synapse – is used frequently, it strengthens; if it isn’t used, it weakens. To imitate this synaptic behaviour, Hasler has built transistors that strengthen (i.e. let more current through) or weaken (let less through) based on how often current flows through them.
Use it or lose it
Hasler’s idea was to harness a quirk that digital computer engineers normally seek to suppress: when current flows through a certain type of transistor known as a field-effect transistor, a few electrons unavoidably leap onto the “gate” electrode – the one that regulates the flow of current. As more electrons accumulate on the gate, they push more electricity through the transistor, so its signal strengthens with use, just like a synapse’s would. And if the transistor is left idle, those electrons leak away and the transistor’s signal weakens like a neglected synapse. Hasler says the process works beautifully in small silicon circuits of one or two synapses. He is now working on making sure that flexibility doesn’t unravel into chaos when it is put in a larger system.
If prosthetics for people with brain injuries are now on the horizon, then it’s tempting to speculate whether they might also spark another industry – neural enhancement for people with healthy brains. Imagine improving your navigation skills by bumping up the size of your hippocampus, or sharpening your eyesight by implanting a silicon retina with 10 times as many photoreceptors as the one you were born with. Then again, it’s not clear whether these implants on their own would be enough. Would the optic nerve or visual cortex be able to handle the added load? Some neuroscientists believe that the capacities of different brain regions have evolved to match one another. If so, then it might be difficult to expand our cognitive powers by augmenting just one or two parts of the brain.
Mariano Sigman, who studies cognitive bottlenecks at the University of Buenos Aires in Argentina, suspects that working memory might be one limit that we can expand. “There are things which are very easy, which any calculator can do, and we cannot do them, like multiplying large numbers together,” he says. That’s because most of us cannot hold enough numbers in our working memory to perform the calculation mentally. Augmenting working memory might help us get around this problem.
Don’t get too excited just yet though. Boahen already receives calls and emails from people with brain injuries or their family members, wondering when neuromorphic implants will be available as prosthetics. He always gives a very cautious reply. “I tell people that it’s not going to happen in their lifetime,” he says.
In any case, if you’re reading this you might already be too old to benefit from these technologies. Brain-boosting devices might produce optimal results only when they’re implanted in young children, says Robert Shannon of the House Ear Institute in Los Angeles. For example, deaf children who receive cochlear implants to stimulate the auditory nerve often show better results than adults – possibly because the wiring of their cortex remains flexible enough to adapt.
Boahen’s team are now building other types of nervous tissue in silicon, including super-sensitive cochlear chips with pitch discrimination 15 times finer than existing cochlear implants; artificial neurons that emulate the thalamus, which regulates our focus of attention; and others that mimic the hippocampus, which, among other things, provides the kind of navigational memory that helps us find our way around the neighbourhood. A new era of brain-enhancing chips and implants is probably still a long way off, but researchers are taking us closer with every step.
