Home Publications Research Grants Inventions & Patents Awards Additional Research Activities Faculties & Institutions Research Areas

Are fast labeling methods reliable? a case study of computer-aided expert annotations on microscopy slides

Marzahl C, Bertram CA, Aubreville M, Petrick A, Weiler K, Gläsel AC, Fragoso M, Merz S, Bartenschlager F, Hoppe J, Langenhagen A, Jasensky AK, Voigt J, Klopfleisch R, Maier A (2020)

---

Publication Type: Conference contribution

Publication year: 2020

Journal

Publisher: Springer Science and Business Media Deutschland GmbH

Book Volume: 12261 LNCS

Pages Range: 24-32

Conference Proceedings Title: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Event location: Lima

ISBN: 9783030597092

DOI: 10.1007/978-3-030-59710-8_3

Abstract

Deep-learning-based pipelines have shown the potential to revolutionalize microscopy image diagnostics by providing visual augmentations and evaluations to a trained pathology expert. However, to match human performance, the methods rely on the availability of vast amounts of high-quality labeled data, which poses a significant challenge. To circumvent this, augmented labeling methods, also known as expert-algorithm-collaboration, have recently become popular. However, potential biases introduced by this operation mode and their effects for training deep neuronal networks are not entirely understood. This work aims to shed light on some of the effects by providing a case study for three pathologically relevant diagnostic settings. Ten trained pathology experts performed a labeling tasks first without and later with computer-generated augmentation. To investigate different biasing effects, we intentionally introduced errors to the augmentation. In total, the pathology experts annotated 26,015 cells on 1,200 images in this novel annotation study. Backed by this extensive data set, we found that the concordance of multiple experts was significantly increased in the computer-aided setting, versus the unaided annotation. However, a significant percentage of the deliberately introduced false labels was not identified by the experts.

Authors with CRIS profile

Involved external institutions

How to cite

APA:

Marzahl, C., Bertram, C.A., Aubreville, M., Petrick, A., Weiler, K., Gläsel, A.C.,... Maier, A. (2020). Are fast labeling methods reliable? a case study of computer-aided expert annotations on microscopy slides. In Anne L. Martel, Purang Abolmaesumi, Danail Stoyanov, Diana Mateus, Maria A. Zuluaga, S. Kevin Zhou, Daniel Racoceanu, Leo Joskowicz (Eds.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 24-32). Lima, PE: Springer Science and Business Media Deutschland GmbH.

MLA:

Marzahl, Christian, et al. "Are fast labeling methods reliable? a case study of computer-aided expert annotations on microscopy slides." Proceedings of the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020, Lima Ed. Anne L. Martel, Purang Abolmaesumi, Danail Stoyanov, Diana Mateus, Maria A. Zuluaga, S. Kevin Zhou, Daniel Racoceanu, Leo Joskowicz, Springer Science and Business Media Deutschland GmbH, 2020. 24-32.

BibTeX: Download