Video: Feeling blue? See how a digital shrink could help
“WHEN was the last time you felt really happy?”
The interviewee shifts uncomfortably in his seat before stumbling over his answer. The movement, hesitation and telltale gaze aversion are noted: this person may be depressed.
The probing questioner is SimSensei, a digital avatar that interviews humans to judge their state of mind. SimSensei is one of several new initiatives designed to partially automate one of the medical profession’s trickiest tasks: diagnosing depression.
Advertisement
SimSensei is more than an astute questioner. Behind the scenes, it uses face recognition technology and depth-sensing cameras built into Microsoft’s Kinect to record and interpret the interviewee’s body language. The animated psychologist can then respond appropriately.
In work to be presented at the conference in Shanghai, China, next month, of the University of Southern California and colleagues used the system to identify characteristic movements that indicate someone may be depressed. To extract the right features, his team interviewed a mixture of healthy volunteers and those who had previously been diagnosed with depression or post-traumatic stress disorder.
After filling out standard questionnaires used to screen people for such conditions, volunteers were interviewed with a high-definition webcam trained on their face and had their body movements logged using Kinect. Scherer found that interviewees who were depressed were more likely to fidget and let their gaze drop. They also smiled less than average.
He says a system that looks for these clues could be a more thorough way to screen people for depression. “Presently broad screening is done by using only a checklist of yes/no questions or point scales, but all the non-verbal behaviour is not taken into account,” Scherer says. “This is where we would like to put our technology to work.”
“Screening does not look at non-verbal behaviour – this is where our technology can be put to work”
Diagnosing depression correctly depends heavily on the experience of the doctor as well as the patient’s ability to express how they feel. As a result, the condition is easily overlooked. Between 15 and 20 per cent of people become clinically depressed at some point in their lives, but they often aren’t treated, with potentially serious consequences.
An automated system can help because it acts as an objective observer, says . Working with the in Sydney, which researches mental health, Joshi and colleagues have created a machine vision system that looks for distinctive facial expressions, slower-than-usual blinking and certain upper-body movements that are characteristic of depression. Like SimSensei, it also detects when the interviewee is looking away while answering or makes fewer gestures than normal.
The team interviewed 60 people, half of whom had been diagnosed with severe depression. As they answered questions about their feelings, their responses were filmed and then processed by the system. It proved to be 90 per cent accurate in its diagnoses, Joshi says.
If depression is diagnosed, automated methods may also be useful in monitoring the severity of symptoms. of the University of Pittsburgh in Pennsylvania and colleagues have looked at how people’s facial expressions change as they receive treatment for depression.
In work also to be presented at the Shanghai conference, Cohn’s team used four cameras to track the faces of 34 people diagnosed with depression as they answered questions about their condition. This was used to train a machine learning system which looked at 66 different parts of the face to spot movements that betrayed certain emotions. The system can therefore also reveal any subtle changes in those movements as the severity of a person’s depression changes.
The team discovered that even though people with depression might smile, they inadvertently use facial muscles as “smile controls” to restrain their expressions.
Perhaps surprisingly, they also showed fewer expressions associated with sadness, something Cohn puts down to the “social risk hypothesis” – the idea that people with depression try to minimise engaging with others.
There has always been interest in what the face can tell us about depression, but measuring people’s expressions has been a challenge, Cohn says. “What’s really changed is that the technology is getting sufficiently advanced that it can now begin to be used.” But we are just beginning on this road, he says.
Exactly how far along we are will soon be put to the test. In October, researchers from around the world will take part in a contest to find the most accurate system for diagnosing depression, to be held at the ACM Multimedia conference in Barcelona, Spain. Challengers will be given a video database of interviews, some involving clinically depressed people. Contestants will train their algorithms on these and will be scored on how good their system is at picking out depressed people from a group. Organiser Michel Valstar of the University of Nottingham, UK, says the challenge is a “unique opportunity for researchers to come together and try and contribute to the area of mental health in a possibly groundbreaking new way.”
Tweets reveal if we are feeling blue
People spill their guts on social media, revealings things that they wouldn’t necessarily share face to face. So Munmun De Choudhury and colleagues at Microsoft Research in Redmond, Washington, mined Twitter to see whether it can be used to gauge levels of depression in society.
The team analysed the language of 69,000 tweets by 489 people who had previously been diagnosed with depression. They also tracked how many Twitter followers each user had and how many they followed, and the timing of users’ tweets over a three-month period.
Users with fewer retweets and replies and those who tweeted more at night were slightly more likely to be depressed, the team found. The same was true of those whose tweets featured the word “I” more often than average.
An algorithm trained on this data was able to predict in 73 per cent of cases whether a Twitter user was depressed or not. The system was then applied to a large number of random Twitter posts to gauge depression levels in the “” – 20 cities picked out by an earlier study that looked at antidepressant use. The Twitter depression index closely correlated with the antidepressant figures. The team will present the results at the , France, in May.
This article appeared in print under the headline “Depressed? Computers know what it’s like”
