Zero-shot personalization of speech foundation models for depressed mood monitoring

Gerczuk M, Triantafyllopoulos A, Amiriparian S, Kathan A, Bauer JF, Berking M, Schuller BW (2023)

Publication Type: Journal article

Publication year: 2023

Journal

Patterns Cell Press

Book Volume: 4

Article Number: 100873

Journal Issue: 11

DOI: 10.1016/j.patter.2023.100873

Abstract

The monitoring of depressed mood plays an important role as a diagnostic tool in psychotherapy. An automated analysis of speech can provide a non-invasive measurement of a patient's affective state. While speech has been shown to be a useful biomarker for depression, existing approaches mostly build population-level models that aim to predict each individual's diagnosis as a (mostly) static property. Because of inter-individual differences in symptomatology and mood regulation behaviors, these approaches are ill-suited to detect smaller temporal variations in depressed mood. We address this issue by introducing a zero-shot personalization of large speech foundation models. Compared with other personalization strategies, our work does not require labeled speech samples for enrollment. Instead, the approach makes use of adapters conditioned on subject-specific metadata. On a longitudinal dataset, we show that the method improves performance compared with a set of suitable baselines. Finally, applying our personalization strategy improves individual-level fairness.

Authors with CRIS profile

Jonathan Felix Bauer Lehrstuhl für Klinische Psychologie und Psychotherapie Matthias Berking Lehrstuhl für Klinische Psychologie und Psychotherapie

Involved external institutions

Universität Augsburg DE Germany (DE)

How to cite

APA:

Gerczuk, M., Triantafyllopoulos, A., Amiriparian, S., Kathan, A., Bauer, J.F., Berking, M., & Schuller, B.W. (2023). Zero-shot personalization of speech foundation models for depressed mood monitoring. Patterns, 4(11). https://dx.doi.org/10.1016/j.patter.2023.100873

MLA:

Gerczuk, Maurice, et al. "Zero-shot personalization of speech foundation models for depressed mood monitoring." Patterns 4.11 (2023).

BibTeX: Download