Active Learning of Ordinal Embeddings: A User Study on Football Data

Löffler C, Fallah K, Fenu S, Zanca D, Eskofier B, Rozell CJ, Mutschler C (2023)

Publication Language: English

Publication Status: Submitted

Publication Type: Journal article, Online publication

Future Publication Type: Journal article

Publication year: 2023

Journal

Transactions on Machine Learning Research

Pages Range: 1-26

URI: https://openreview.net/forum?id=oq3tx5kinu

Open Access Link: https://openreview.net/pdf?id=oq3tx5kinu

Abstract

Humans innately measure distance between instances in an unlabeled dataset using an unknown similarity function. Distance metrics can only serve as proxy for similarity in information retrieval of similar instances. Learning a good similarity function from human annotations improves the quality of retrievals. This work uses deep metric learning to learn these user-defined similarity functions from few annotations for a large football trajectory dataset. We adapt an entropy-based active learning method with recent work from triplet mining to collect easy-to-answer but still informative annotations from human participants and use them to train a deep convolutional network that generalizes to unseen samples. Our user study shows that our approach improves the quality of the information retrieval compared to a previous deep metric learning approach that relies on a Siamese network. Specifically, we shed light on the strengths and weaknesses of passive sampling heuristics and active learners alike by analyzing the participants' response efficacy. To this end, we collect accuracy, algorithmic time complexity, the participants' fatigue and time-to-response, qualitative self-assessment and statements, as well as the effects of mixed-expertise annotators and their consistency on model performance and transfer-learning.

Authors with CRIS profile

Christoffer Löffler Machine Learning and Data Analytics Lab Dario Zanca Machine Learning and Data Analytics Lab Björn Eskofier Machine Learning and Data Analytics Lab

Involved external institutions

Fraunhofer-Institut für Integrierte Schaltungen (IIS) DE Germany (DE) Georgia Institute of Technology US United States (USA) (US)

How to cite

APA:

Löffler, C., Fallah, K., Fenu, S., Zanca, D., Eskofier, B., Rozell, C.J., & Mutschler, C. (2023). Active Learning of Ordinal Embeddings: A User Study on Football Data. Transactions on Machine Learning Research, 1-26.

MLA:

Löffler, Christoffer, et al. "Active Learning of Ordinal Embeddings: A User Study on Football Data." Transactions on Machine Learning Research (2023): 1-26.

BibTeX: Download