An AI-powered app developed by Dartmouth College researchers promises to detect signs of depression using facial recognition software on your smartphone—and may even be able to alert you before you even realize you’re depressed.
The app is called MoodCapture and it uses your phone’s front-facing camera to analyze your emotions. As you use the facial unlock feature throughout the day, the app provides “in-the-moment” analysis to “predict your depression,” Andrew Campbell, a computer scientist at Dartmouth who co-created MoodCapture, told The Daily Beast in an email.
“Mental illness has profound effects on individuals and their families, including my own,” Campbell said. “If we can navigate the challenging issues related to privacy and ethics, a future app like MoodCapture holds the potential to significantly help individuals with depression. When combined with other AI tools, it could provide personalized interventions to help people to function and stay healthy.”
ADVERTISEMENT
The team plans to present their findings at the Association of Computing Machiner’s CHI conference in May 2024, but have published an advanced copy of their paper on Arxiv. The study involved 177 participants diagnosed with major depressive disorder who used MoodCapture over three months.
The app collected more than 125,000 images with an average of six photos a day. By the end of the study, MoodCapture successfully identified early symptoms of depression in users with 75 percent accuracy. The team also created a more personalized model that improved accuracy by up to 80 percent.
“The model focuses on a myriad of facial features, including expressions stemming from muscle activations, eye gaze, head pose, and the 2D/3D locations of various face landmarks such as lips and eyes,” Campbell said. “AI is used to derive meaningful insights from each image, including factors like lighting conditions, the number of people in the image, dominant colors, photo location, and background objects.”
The authors added that the app also has the potential to detect signs of depression before users even realize that they have an issue. This could help them get treatment or help before their condition worsens—thereby improving mental health outcomes.
While the app offers a lot of promise, there are a lot of causes for concern. For one, AI technology being used in such a manner could lead to unintended outcomes that cause harm. The emerging technology has a long and sordid history of bias and discrimination. In the past, issues with facial recognition AI have led to issues like Google Photos misidentifying the faces of Black users as animals, or people of color being falsely identified as criminal suspects by criminal justice models.
From left, Guarini PhD candidates Arvind Pillai and Subigya Nepal are co-lead authors of the study, computer science professor Andrew Campbell is a corresponding author, and Geisel professor Nicholas Jacobson is a co-author.
Katie Lenhart / DartmouthThis could have potentially harmful consequences when it comes to an app to detect depression. For example, it might be unable to identify depression symptoms in people of color. Or it may tell someone they have depression when they don’t. Campbell acknowledges the issue of potential bias saying the model “exhibits higher representation of females and white participants, raising concerns about potential biases and limitations in generalizability.”
“It is crucial for the efficacy and validity of such a method to reach an accuracy level of around 90 percent and deal with bias before it would be widely available,” he explained, later adding that the “billions of images” that major tech companies like Google and Apple handle can help provide the scale to further refine their app.
However, there also remain big concerns concerning privacy and security. While all the participants of the study gave informed consent allowing their facial images to be recorded, third parties collecting personal data still pose a large concern for privacy advocates—even if the data collection is being used for noble purposes like detecting and helping treat depression.
“While acknowledging significant privacy concerns, I believe addressing these issues is feasible,” Campbell said. “For example, in a future MoodCapture app, all image processing and training will occur directly on AI processors within phones, ensuring that no images leave the device.”
For now, MoodCapture remains a proof of concept. AI isn’t all chatbots and “woke” image generators. It can also potentially let you learn a bit more about yourself and your mental health—and get you help when you need it.
“Many people have no access to mental health professionals, but most people have phones,” Campbell said. “Achieving this goal is akin to a moonshot in the field of mental health.”
Editor's note: A previous version of this story referred to the school as "The University of Dartmouth" instead of "Dartmouth College." We regret this error.
