Researchers are exploring the use of speech and voice analysis, assisted by artificial intelligence, to help diagnose depression and other mental illnesses. While not yet ready for use in practice, the research on the potential effectiveness and uses of artificial intelligence-driven speech and voice analysis as a biomarker for mental health conditions continues to accumulate.

Voice biomarkers are measurable features in a person’s speech, such as tone, pitch, cadence and speech rate, that can indicate aspects of their physical or mental health. Small, subtle changes in these features can indicate mental health conditions, including depression, anxiety and others. Recent technological advancements, including AI and machine learning, smartphones, and telehealth apps for mental health, have made this analysis possible and the process remotely accessible.

The voice analysis looks at two basic types of speech-related data: what the sound of the voice is like (acoustic features) and what the person says and how they use language (linguistic features). Measures of the sound can include pitch (how high or low the voice sounds), how loud or soft the voice is, how fast someone talks, how clear and regular the tone is, and the up and down pattern of the speech. Measures related to language include how varied the vocabulary is, how long the words are, and how complex the sentences are. The language measures relate to cognitive abilities, working memory and executive function.

In addition to helping with diagnosis, voice measures can also be used to measure depression severity and to help monitor depression symptoms over time. The voice recordings can be done in a clinical setting or via a smartphone app.

Irene Rodrigo and Jon Andoni Duñabeitia, Ph.D., summarized research identifying voice characteristics associated with specific conditions, for example

• Voice quality - zero-crossing rate (rate at which the signal changes from positive to negative, or vice versa) is associated with depression
• Pitch is associated with stress, depression, bipolar disorder and cognitive impairment
• Energy (intensity) is associated with stress
• Speech rate and speech pause duration are associated with depression, stress, dementia and cognitive impairment
• Vocabulary diversity and word length are associated with dementia.

One review study found voice measures had accuracy rates of 78% to 96% in identifying people with depression versus those without it. Another study assessed individuals with both depression and anxiety. Using data from a one-minute verbal fluency test, in which an individual names as many words as possible within a given category, it found approximately 70%–83% accuracy in detecting when a person has both anxiety and depression

This technology has also been able to identify voice characteristics consistent with depression symptoms from just 25 seconds of speech. In a study using samples of 25-second free-form speech from nearly 15,000 adults, machine-learning analysis correctly identified moderate-to-severe depression more than 70% of the time (based on comparison with self-reported Patient Health Questionnaire-9 (PHQ-9) scores of 10 or more).

Researchers note that while this technique offers significant promise, it faces several issues with technical aspects, standardizations, and how it will work in the real world and across diverse population groups. While still being explored, developed and refined, voice biomarkers could be used to detect, monitor and manage mental health conditions.