Innovation is our regular column that highlights the latest emerging technological ideas and where they may lead.
At the in Los Angeles, California, this week, Microsoft unveiled a new hands-free, full-body-control system for its console, . Using it, players can interact with games simply by talking and moving their body.
Microsoft also claims that it can recognise emotions, and Natal has impressed , developers and movie mogul alike.
Advertisement
It is undeniably impressive. And you have to wonder: how does Natal actually work? Microsoft is remaining tight-lipped. But we鈥檝e talked to industry insiders and pulled together some material from the 麻豆传媒 archive to suggest how it could live up to the high expectations generated by partially enhanced concept videos like .
Camera capture
Although techniques to capture a person鈥檚 movement using cameras have been in development since the 1970s, doing so without attaching markers to the person鈥檚 body as Natal does is a more recent possibility.
is one of the few commercial companies currently offering markerless motion capture technology. Andrew Tschesnok, the firm鈥檚 CEO, says his system uses as many as 14 cameras positioned around an actor to get a 3D picture of their movement in real time.
But Natal comes equipped with just two cameras. So how can it work with such a limited input?
In-depth view
A year ago, Israeli firm unveiled a system that uses an infrared depth-sensing camera. The is said to be able to pinpoint the depth of an object to within 1 to 2 centimetres, and capture the information at a rate of 60 frames per second for very smooth motion. A second, full colour camera in the device records textures and colours.
In March of this year, the company sold its assets to a third party, . Many people have concluded that Natal鈥檚 motion capture system owes a lot to the ZCam. When 麻豆传媒 asked 3DV System鈥檚 CEO Zvika Klier about the possibility this week, he had no comment.
Tschesnok says the results from ZCam-like systems fall short of what he鈥檇 call true markerless motion capture 鈥� Natal can鈥檛 see what鈥檚 going on behind a gamer鈥檚 back 鈥� but it is still an improvement over other games consoles鈥� interfaces and will work well for Natal鈥檚 needs. 鈥淚n a way, it鈥檚 the same effect as having a Wiimote in each hand, one on each foot and one on your head,鈥� he says.
But more is yet to come: Tschesnok predicts that the games consoles due out early in the next decade will use his multi-camera system, built into the widely used 鈥渟urround sound鈥� speaker systems to provide users with a truly immersive experience.
Voice in the crowd
People are also understandably excited about Natal appearing to . However, while speech recognition software may be increasingly common, it still suffers from the 鈥渃ocktail party鈥� effect; where background noise and multiple voices can cause errors.
Finding a way round this issue is vital for gaming. Microsoft is targeting Natal at social and casual gamers who are likely to play in the company of others, yet few, if any, voice recognition systems have been created capable of following a single voice in a noisy room.
IBM has shown that a system that can read lips provides one way around that. That could help Natal, but would need to be tuned separately to different languages.
Research that enabled one of Honda鈥檚 Asimo humanoid鈥檚 to understand three voices all speaking at once even suggests that multiplayer voice control could be possible one day.
Face reading
Speech software could also feasibly be used to recognise emotions from stresses and emphasis in a player鈥檚 voice. For example, software called Emotive Alert is able to classify voices into one of eight basic emotional states.
We are told, though, that Natal will use facial recognition, so it may read emotions that way too. Earlier this year, artist teamed up with neuroscientist at University College London, UK, to develop an art installation that responds to the emotion of visitors, using an algorithm Frith created to read faces.
Frith told 麻豆传媒 that getting such systems to perform reliably is difficult, but not impossible. However, it is unlikely that Natal will detect more than a handful of basic emotions.
Emotional connection
Even with that limitation, a console able to sense emotions and respond to them has the potential to make gaming a much richer experience.
In 2005, an 鈥渆motionally aware鈥� virtual fitness trainer, name Laura, was tested on groups of volunteers. Her friendly gestures and sympathetic body language were found to genuinely foster a better connection with users. This positive influence significantly increased the participants鈥� exercise levels compared to a control group interacting with a version of Laura that didn鈥檛 recognise emotions.
The Natal concept videos may seem on first viewing to show technology bordering on the fictional, but it鈥檚 clear that many of the basics are within our technological reach. How many of these features gamers will see when Natal finally hits shelves, though, is unknown.
Read previous Innovations: Winning the cybersecrets armsrace, Software to track our emotional outbursts, Mind-reading headsets will change your brain, Harnessing spammers to advance AI, 100-mpg car contest under starters orders, The mobile future of the keyboard, A licence to print gadgets, What next after the megapixel wars?, How social networking might change the world, The battle for the paperless book, The cellphone economy
