A computer model can diagnose depression based on an image in a social media user's post with an accuracy of 70 percent. While imperfect, this easily exceeds the 42 percent success rate achieved by general practitioners when assessing someone in person. The finding could lead to frightening privacy intrusions, but could also increase the chance of people getting the help they need.
Professor Chris Danforth of the University of Vermont had 166 volunteers provide access to their Instagram accounts, with a total of 43,950 photographs. They also provided records of their mental health, with 71 participants having been diagnosed with depression in the previous three years. Images were analyzed for features, such as whether they included faces, the filters applied, and responses received.
Danforth tried several algorithms, drawing on research showing people’s preferences regarding color and brightness change when depressed. In EPJ Data Science he reported that the most successful algorithm proves there is a reason we talk about “feeling blue”.
"Pixel analysis of the photos in our dataset revealed that depressed individuals in our sample tended to post photos that were, on average, bluer, darker, and grayer than those posted by healthy individuals," Danforth and his co-author, Harvard graduate student Andrew Reece, write in a blog post discussing their work. Those who were depressed steered clear of Instagram filters that make images look warmer or lighter, preferring Inkwell, which turns color shots to black and white. "In other words, people suffering from depression were more likely to favor a filter that literally drained all the color out the images they wanted to share," the authors write.
Color was not the only distinguishing feature. People with depression posted more often, but used photographs with fewer people in them, which the authors speculate may reflect the reduced amounts of socializing. However, the authors also note that since the computer did not distinguish between selfies and photos of others, there may be a tendency for people who are suffering to not post pictures of themselves.
All these may seem like easy patterns to learn, but when the authors had volunteers look at the same photographs, their capacity to identify who had depression, while better than chance, was not as good as the computer. Indeed, to the extent other people could recognize depression from the photographs, it seems they may be using different cues from the machine.
One aspect of the study that runs against intuition is that there were more comments on posts made by people with depression than those without. Danforth told IFLScience this made only a small contribution to the capacity to identify depression compared to other factors; “So I wouldn’t put much weight behind that finding.” Nevertheless, it raises interesting questions if verified, possibly suggesting friends and family who are aware of someone's depression use increased commentary as a way of showing support.
Danforth imagines a time when “You can go to doctor and push a button to let an algorithm read your social media history as part of the exam." Alternatively, “Imagine an app you can install on your phone that pings your doctor for a checkup when your behavior changes for the worse, potentially before you even realize there is a problem."
Such a process seems desirable, compared to the low reliability of general practitioners, possibly because their assessments are based on narrow windows of time. On the other hand, it’s not hard to imagine such assessments being done without people’s permission, particularly if the test was extended to other forms of mental illness – big brother really could be watching you.
Neither scenario is imminent. “This study is not yet a diagnostic test, not by a long shot,” Danforth said. “We acknowledge that depression describes a general clinical status, and is frequently combined with other conditions,” the paper notes. Moreover, the algorithm did a substantially better job of recognizing depression when it looked at photographs taken both before and after a diagnosis had been made than when restricted to those taken beforehand, suggesting some behavior may be reinforced by diagnosis.
