A new company just launched that can detect depression based on the sound of your voice — here’s how it works

  • Cogito, a tech startup out of MIT, has raised $US73 million with backing from Goldman Sachs and Salesforce Ventures. It boosts the quality of call centres’ customer service by analysing customers’ voices and flagging when they’re unsatisfied.
  • This month, Cogito spun out a new company focused on detecting mental illness based on the sound of your voice.
  • A growing cluster of health-tech CEOs, clinicians, and entrepreneurs say voice is the future of healthcare.

For Sub Datta, the fight for better tools to identify severe bouts of depression is deeply personal.

Datta lives with clinical depression. He’s also the first leader of a new company that’s using data collected from smartphones – including the sound of a user’s voice – to detect acute episodes of mental illnesses like his.

Called CompanionMX, the company was recently spun out of MIT enterprise company Cogito, which has raised $US73 million from backers like Goldman Sachs and Salesforce Ventures. Cogito helps increase the quality of call centres’ customer service by analysing customers’ voices and flagging when they’re unsatisfied.

CompanionMX’s current focus is an app called Companion, which analyses the sound of your voice along with data on how much time you spend texting, calling, or using social media.

Every time you send Companion a short recording or voice note, the app analyses how you sound. Together with the data on how you’re using your phone, that information helps create a picture of your mental health. The information can also be shared with a clinician who can offer support.

Datta sees Companion’s continuous stream of data as a critical alternative to the current model, which consists largely of sporadic therapy visits and last-minute emergency room trips. Although depression is currently the leading cause of disability worldwide, as many as two-thirds of people with the disorder go without treatment, according to data from the National Alliance for Mental Illness. Beyond resulting in needless pain and suffering, that also costs US businesses billions in lost productivity every year.

The information that Companion gathers can be thought of as a new “vital sign” for mental health – one that could help clinicians intervene before symptoms worsen or even lead to thoughts of suicide – according to Datta.

“This way people like myself don’t have to rely on intermittent data but rather have continual data,” he told Business Insider.

How Companion works

Talking on the phone
When Companion prompts you to do so, you leave a short voice note on the app, similar to how you’d send a friend a voice note over iMessage or WhatsApp. MLazarevski/Flickr

The Companion tool is available as an app for iPhone and Android users after a healthcare professional recommends you use it.

Once you grant the app permission to collect data on how you use your phone, it looks at four main metrics: physical and social isolation (how much time you spend by yourself vs. with others), your mood, and your energy levels.

After a few days of running in the background, the app can give you a picture of what Datta calls your “overall behavioural stability,” or your baseline. That way when someone suddenly diverts from their regular behaviour – perhaps suddenly spending a lot of time alone, for example – the app can tell.

The app’s key metric, however, is voice.

When Companion prompts you to do so, you leave a short voice note on the app, similar to how you’d send a friend a voice note over iMessage or WhatsApp. You can say anything throughout the course of the voice note, but it’s generally regarded as a kind of short audio diary, Joshua Feast, the CEO of Cogito and a member of CompanionMX’s board of directors, told Business Insider.

Read more:
Your phone and smartwatch may soon tell doctors if you’re entering a depressive episode – and Olympian Michael Phelps is on board

“Think of it like a little moment of reflection where you get immediate feedback on how you’re sounding,” said Feast.

Just as the app looks at your overall phone usage to determine your baseline activity patterns, it also analyses your voice over time to give you an idea of your norm based on everything from your pitch and tone to your volume and enunciation. Then it can detect when you diverge from that pattern.

Before becoming available to the general public, researchers tested Companion in a series of clinical trials with medical institutions including Brigham and Women’s Hospital and the US Veterans Administration. Based on those results, the Companion team decided the app could provide patients with clinically meaningful information.

“With voice you get more precision – it’s quite unique and distinctive, like fingerprints,” David K. Ahern, the director of Brigham and Women’s Hospital’s digital behavioural health research program and a co-principal investigator of a recent clinical trial using Companion on a subset of patients, told Business Insider last year.

As a result of that research, the Companion team also discovered that leaving voice notes was something people genuinely enjoyed doing, Feast said.

Is 2019 the year of voice in healthcare?

There’s a growing movement in healthcare to home in on voice as the next big health tool. While some see voice as a way for clinicians to speed up patient visits, others see it more like Datta does, as a means of detecting illness in patients.

“The killer app is going to be voice” as an addition to electronic medical records, said Toby Cosgrove, the former CEO of the leading academic medical center Cleveland Clinic and a new executive advisor to Google Cloud, at a recent health conference in San Francisco called the Rock Health Summit.

Paul Markovich, the CEO of the health insurer Blue Shield of California, agreed with Cosgrove, he told Business Insider this month.

Voice is “going to be big in the future,” Markovich said.

Read more:
The CEO of one of the largest US health insurers has a warning for Amazon on its healthcare ambitions

He described a pilot program that Blue Shield is currently running with physicians that involves having them record themselves during patient visits. An algorithm translates the relevant information from the recordings into patients’ electronic medical records.

That’s similar to the way Suki, a new AI startup backed by Salesforce chief Marc Benioff, aims to help doctors record medical notes. Augmedix, another startup that helps doctors record notes from patient visits, does so using Google Glass.

On the patient side of healthcare, several companies are also looking at voice as a way to detect when a patient isn’t feeling well.

Amazon recently filed a patent aimed at enabling its voice assistant Alexa to detect when a user is sick with something like a cold or the flu, for example. And Cogito, CompanionMX’s parent company, has been using data collected from the sound of users’ voices to enhance companies’ customer service abilities for the past decade.

“Capturing voice and understanding and translating that meaning is helpful and can be useful clinically,” Markovich said.