Nico gazes into the mirror in front of him. Looking back is his reflected self, wearing a grey Yale University sweatshirt and a baseball cap cocked at a jaunty angle. When Nico raises an arm, he recognises the arm moving in the mirror as his own.
It may not sound like much of a feat, but Nico is a humanoid robot. He has just become the first of his kind to recognise his own reflection in a mirror.
The ability to recognise your reflection is considered an important milestone in infant development, and as a mark of self-awareness, sociability and intelligence in a non-human animal. Nico’s ability to perform the same feat could pave the way for more sophisticated robots that can recognise their own bodies even if they are damaged or reconfigured.
Advertisement
The achievement is one of a cluster of recent instances in which robots have begun to approach the major milestones in cognitive development. If robots can be taught to move from one developmental stage to the next, as infants do, they may eventually be capable of learning more complicated tasks and therefore become more useful to humans. “It’s less about recreating a human than making a human-compatible being,” says Matt Berlin, a robotics researcher at the Massachusetts Institute of Technology.
To endow Nico with the ability to recognise himself, Kevin Gold and his supervisor Brian Scassellati at Yale equipped Nico with a video camera behind one of his eyes, a jointed arm and an attached computer running some clever software. When Nico points his camera eye at the mirror, the software assigns sections of the image a probability of being “self”, “another”, or “neither”. The software does this by constantly comparing successive frames to identify pixels that change from one frame to the next. At the same time motion sensors in Nico’s arm tell the software when he is moving. Whenever a section of the image changes at the same time as his motion sensors detect movement in the arm, he assigns that section a high probability of being “self”. If a section of the image shifts and Nico detects no movement in his arm, he assigns that image section a high probability of being “another”, while static sections are likely to be “neither”. This allows him to recognise not only his own moving limbs, but those of other robots or people.
To test the self-recognition software, Gold programmed Nico to move his arm for 4 minutes while filming it with his camera, allowing him to learn when movement of his arm, detected by his arm sensors, corresponded to motion of the arm in the video. Nico was then positioned so that he could see both his own reflection in a mirror and Gold standing beside it. Gold carried out a range of different tasks, including juggling and drinking a bottle of water, while Nico moved his arm around. Nico’s software was able to correctly classify the pixel regions corresponding to his own reflection and those of Gold 95 per cent of the time.
The same system should also make it possible for robots to recognise their own limbs even if they are damaged, wearing different clothes or in different lighting conditions, by correlating movement detected by on-board cameras with those reported by sensors on their limbs, says Gold. This should help them carry out tasks such as manipulating objects, or let them adapt their gait to a changing terrain, when conventional vision software can be fooled by changes in appearance or environment.
The ability to tell self from other should also allow robots to carry out more sophisticated tasks, says Olaf Sporns, a cognitive scientist and roboticist at Indiana University in Bloomington. For instance, researchers are investigating imitation as a way of helping robots learn how to carry out tasks. To successfully and safely imitate someone, though, robots will need to distinguish between their own limbs and those of another person, as Nico can. “The distinction between self and other is a fundamental problem for humanoid robotics,” says Sporns.
Meanwhile, a furry robot called Leonardo, built at MIT, recently reached another developmental milestone, the ability to grasp that someone else might believe something you know to be untrue. You can test the capacity for “false belief” in children by showing them a scene in which a child puts chocolate in a drawer and goes away. While he is out of sight, his mother moves the chocolate somewhere else. Young children are incapable of seeing the world through the other child’s eyes, and so predict that he will look for the chocolate in the place his mother has left it. Only when they reach 4 or 5 can they predict that the other child will mistakenly look for the chocolate in the drawer.
“Leonardo can grasp that someone else might believe something you know to be untrue”
Leonardo, developed by Cynthia Breazeal together with Berlin and colleague Jesse Gray, uses face, image and voice recognition software running on an array of attached computers to build a “brain” for himself – basically a list of objects around him in the room and events that he has witnessed. Whenever he spots a new face, he builds and stores another “brain” which processes information in the same way as his own but sees the world from the new person’s point of view. Unlike his own brain, copies are not updated with objects or events that the other person would not be aware of, either because something is blocking their view or because they temporarily leave the room. “It’s built around the idea that you can use the self as a simulator for understanding others,” says Berlin. See a video of Leonardo.
When faced with the false-belief test, Leonardo knows that a toy (used in place of chocolate) has been moved and also that a person who left the room before this will not be aware of it. See a video of this experiment.It is more than just a cute trick, however. Gray and Berlin have found that the ability to model other people’s beliefs allows Leonardo to better understand their goals, even when these are unclear. In one demonstration, Leonardo watched as Gray and Berlin put potato chips in one box and cookies in another. Berlin then left the room, and Gray switched the boxes before also disappearing. When Berlin reappeared and attempted to open the first box, Leonardo anticipated that he wanted chips, as Berlin was unaware that they had been moved. Similarly, he understood that when Gray rattled the same box he wanted cookies, and so offered them to him.
As well as helping to build better robots, such research could ultimately enhance our understanding of cognitive development in infants. Developmental milestones such as self-recognition and modelling other people’s beliefs are believed to be associated with the development of other important capabilities, such as empathy and sociability. By performing feats associated with these milestones, such robots could help researchers understand what capabilities infants need to reach them, says Sporns. “It shows us that complex phenomena can sometimes be explained on the basis of simple mechanisms,” he says. “Ultimately, if we want to really understand how it is done in humans we need to go back to the lab. But this tells us where to look.”
Not that impressed by Nico’s mirror test
Nico the robot’s ability to recognise his own reflection is prompting his creators to doubt the significance of the mirror test as a sign of self-awareness.
Ever since chimps were first shown to recognise their reflection in 1970, the mirror test has been used as a mark of self-awareness in animals. Since then orangutans, some dolphins and elephants have passed the test, pioneered by Gordon Gallup who is now at the University of Albany in New York.
But Kevin Gold and his supervisor Brian Scassellati at Yale, who built Nico, argue that the test may not be as important as researchers believe because Nico shows no other signs of self-awareness other than recognising himself in a mirror. “When we see an animal or robot recognise itself in the mirror, it’s a powerful symbol,” Gold says. “But I am now thoroughly unimpressed by the feat.”
Cognitive scientists, however, say there is no evidence that the robot can truly recognise itself and therefore the experiment does not diminish the significance of the test. “This says nothing about mirror self-recognition,” says Daniel Povinelli, a cognitive scientist and chimp expert at the University of Louisiana at Lafayette. The robot’s ability to correlate the motion of its own body to the image in the mirror is “necessary but not sufficient” for self-recognition, he says. To be classified as truly self-aware, an animal must use the mirror as a tool to examine a mark painted on its forehead. Animals that pass the mirror test tend to exhibit other behaviours in front of a mirror, says Povinelli. For example, chimps will examine their eyes, open and peer into their mouths, and try to view their anus and genitals, indicators of self-recognition the robot does not display.
“They are using the mirror as a tool to view parts of their body that they can’t see without it,” says Diana Reiss, a cognitive scientist who specialises in dolphins and elephants at Hunter College in New York City. “They wouldn’t use it as a tool unless they understood it was a representation of themselves.”
