Exploring the relationship between segmentation uncertainty, segmentation performance and inter-observer variability with probabilistic networks
Chotzoglou E, Kainz B (2019)
Publication Type: Conference contribution
Publication year: 2019
Journal
Publisher: Springer
Book Volume: 11851 LNCS
Pages Range: 51-60
Conference Proceedings Title: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Event location: Shenzhen, CHN
ISBN: 9783030336417
DOI: 10.1007/978-3-030-33642-4_6
Abstract
Medical image segmentation is an essential tool for clinical decision making and treatment planning. Automation of this process led to significant improvements in diagnostics and patient care, especially after recent breakthroughs that have been triggered by deep learning. However, when integrating automatic tools into patient care, it is crucial to understand their limitations and to have means to assess their confidence for individual cases. Aleatoric and epistemic uncertainties have been subject of recent research. Methods have been developed to calculate these quantities automatically during segmentation inference. However, it is still unclear how much human factors affect these metrics. Varying image quality and different levels of human annotator expertise are an integral part of aleatoric uncertainty. It is unknown how much this variability affects uncertainty in the final segmentation. Thus, in this work we explore potential links between deep network segmentation uncertainties with inter-observer variance and segmentation performance. We show how the area of disagreement between different ground-truth annotators can be developed into model confidence metrics and evaluate them on the LIDC-IDRI dataset, which contains multiple expert annotations for each subject. Our results indicate that a probabilistic 3D U-Net and a 3D U-Net using Monte-Carlo dropout during inference both show a similar correlation between our segmentation uncertainty metrics, segmentation performance and human expert variability.
Authors with CRIS profile
Involved external institutions
United Kingdom (GB)How to cite
APA:
Chotzoglou, E., & Kainz, B. (2019). Exploring the relationship between segmentation uncertainty, segmentation performance and inter-observer variability with probabilistic networks. In Luping Zhou, Nicholas Heller, Yiyu Shi, Danny Chen, X. Sharon Hu, Yiming Xiao, Raphael Sznitman, Veronika Cheplygina, Diana Mateus, Emanuele Trucco, Matthieu Chabanas, Hassan Rivaz, Ingerid Reinertsen (Eds.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 51-60). Shenzhen, CHN: Springer.
MLA:
Chotzoglou, Elisa, and Bernhard Kainz. "Exploring the relationship between segmentation uncertainty, segmentation performance and inter-observer variability with probabilistic networks." Proceedings of the 4th International Workshop on Large-Scale Annotation of Biomedical Data and Expert Label Synthesis, LABELS 2019, the 1st International Workshop on Hardware Aware Learning for Medical Imaging and Computer Assisted Intervention, HAL-MICCAI 2019, and the 2nd International Workshop on Correction of Brainshift with Intra-Operative Ultrasound, CuRIOUS 2019, held in conjunction with the 22nd International Conference on Medical Imaging and Computer-Assisted Intervention, MICCAI 2019, Shenzhen, CHN Ed. Luping Zhou, Nicholas Heller, Yiyu Shi, Danny Chen, X. Sharon Hu, Yiming Xiao, Raphael Sznitman, Veronika Cheplygina, Diana Mateus, Emanuele Trucco, Matthieu Chabanas, Hassan Rivaz, Ingerid Reinertsen, Springer, 2019. 51-60.
BibTeX: Download