Zero-shot personalization of speech foundation models for depressed mood monitoring

Gerczuk M, Triantafyllopoulos A, Amiriparian S, Kathan A, Bauer JF, Berking M, Schuller BW (2023)

Publication Type: Journal article

Publication year: 2023


Book Volume: 4

Article Number: 100873

Journal Issue: 11

DOI: 10.1016/j.patter.2023.100873


The monitoring of depressed mood plays an important role as a diagnostic tool in psychotherapy. An automated analysis of speech can provide a non-invasive measurement of a patient's affective state. While speech has been shown to be a useful biomarker for depression, existing approaches mostly build population-level models that aim to predict each individual's diagnosis as a (mostly) static property. Because of inter-individual differences in symptomatology and mood regulation behaviors, these approaches are ill-suited to detect smaller temporal variations in depressed mood. We address this issue by introducing a zero-shot personalization of large speech foundation models. Compared with other personalization strategies, our work does not require labeled speech samples for enrollment. Instead, the approach makes use of adapters conditioned on subject-specific metadata. On a longitudinal dataset, we show that the method improves performance compared with a set of suitable baselines. Finally, applying our personalization strategy improves individual-level fairness.

Authors with CRIS profile

Involved external institutions

How to cite


Gerczuk, M., Triantafyllopoulos, A., Amiriparian, S., Kathan, A., Bauer, J.F., Berking, M., & Schuller, B.W. (2023). Zero-shot personalization of speech foundation models for depressed mood monitoring. Patterns, 4(11).


Gerczuk, Maurice, et al. "Zero-shot personalization of speech foundation models for depressed mood monitoring." Patterns 4.11 (2023).

BibTeX: Download