Female, white, 27? Bias Evaluation on Data and Algorithms for Affect Recognition in Faces
Pahl J, Rieger I, Möller A, Wittenberg T, Schmid U (2022)
Publication Type: Conference contribution
Publication year: 2022
Publisher: Association for Computing Machinery
Pages Range: 973-987
Conference Proceedings Title: ACM International Conference Proceeding Series
Event location: Online
ISBN: 9781450393522
Abstract
Nowadays, Artificial Intelligence (AI) algorithms show a strong performance for many use cases, making them desirable for real-world scenarios where the algorithms provide high-impact decisions. However, one major drawback of AI algorithms is their susceptibility to bias and resulting unfairness. This has a huge influence for their application, as they have a higher failure rate for certain subgroups. In this paper, we focus on the field of affective computing and particularly on the detection of bias for facial expressions. Depending on the deployment scenario, bias in facial expression models can have a disadvantageous impact and it is therefore essential to evaluate the bias and limitations of the model. In order to analyze the metadata distribution in affective computing datasets, we annotate several benchmark training datasets, containing both Action Units and categorical emotions, with age, gender, ethnicity, glasses, and beards. We show that there is a significantly skewed distribution, particularly for ethnicity and age. Based on this metadata annotation, we evaluate two trained state-of-the-art affective computing algorithms. Our evaluation shows that the strongest bias is in age, with the best performance for persons under 34 and a sharp decrease for older persons. Furthermore, we see an ethnicity bias with varying direction depending on the algorithm, a slight gender bias and worse performance for facial parts occluded by glasses.
Authors with CRIS profile
Involved external institutions
Germany (DE)How to cite
APA:
Pahl, J., Rieger, I., Möller, A., Wittenberg, T., & Schmid, U. (2022). Female, white, 27? Bias Evaluation on Data and Algorithms for Affect Recognition in Faces. In ACM International Conference Proceeding Series (pp. 973-987). Online, KR: Association for Computing Machinery.
MLA:
Pahl, Jaspar, et al. "Female, white, 27? Bias Evaluation on Data and Algorithms for Affect Recognition in Faces." Proceedings of the 5th ACM Conference on Fairness, Accountability, and Transparency, FAccT 2022, Online Association for Computing Machinery, 2022. 973-987.
BibTeX: Download