Cytologic scoring of equine exercise-induced pulmonary hemorrhage: Performance of human experts and a deep learning-based algorithm

Bertram CA, Marzahl C, Bartel A, Stayt J, Bonsembiante F, Beeler-Marfisi J, Barton AK, Brocca G, Gelain ME, Glasel A, Du Preez K, Weiler K, Weissenbacher-Lang C, Breininger K, Aubreville M, Maier A, Klopfleisch R, Hill J (2022)

Publication Type: Journal article

Publication year: 2022

Journal

Veterinary Pathology American College of Veterinary Pathologists

DOI: 10.1177/03009858221137582

Abstract

Exercise-induced pulmonary hemorrhage (EIPH) is a relevant respiratory disease in sport horses, which can be diagnosed by examination of bronchoalveolar lavage fluid (BALF) cells using the total hemosiderin score (THS). The aim of this study was to evaluate the diagnostic accuracy and reproducibility of annotators and to validate a deep learning-based algorithm for the THS. Digitized cytological specimens stained for iron were prepared from 52 equine BALF samples. Ten annotators produced a THS for each slide according to published methods. The reference methods for comparing annotator’s and algorithmic performance included a ground truth dataset, the mean annotators’ THSs, and chemical iron measurements. Results of the study showed that annotators had marked interobserver variability of the THS, which was mostly due to a systematic error between annotators in grading the intracytoplasmatic hemosiderin content of individual macrophages. Regarding overall measurement error between the annotators, 87.7% of the variance could be reduced by using standardized grades based on the ground truth. The algorithm was highly consistent with the ground truth in assigning hemosiderin grades. Compared with the ground truth THS, annotators had an accuracy of diagnosing EIPH (THS of < or ≥ 75) of 75.7%, whereas, the algorithm had an accuracy of 92.3% with no relevant differences in correlation with chemical iron measurements. The results show that deep learning-based algorithms are useful for improving reproducibility and routine applicability of the THS. For THS by experts, a diagnostic uncertainty interval of 40 to 110 is proposed. THSs within this interval have insufficient reproducibility regarding the EIPH diagnosis.

Authors with CRIS profile

Christian Marzahl Lehrstuhl für Informatik 5 (Mustererkennung) Katharina Breininger Juniorprofessur für Artificial Intelligence in Medical Imaging Andreas Maier Lehrstuhl für Informatik 5 (Mustererkennung)

Involved external institutions

Veterinärmedizinische Universität Wien (Vetmeduni Vienna) / University of Veterinary Medicine, Vienna AT Austria (AT) Justus-Liebig-Universität Gießen DE Germany (DE) Freie Universität Berlin DE Germany (DE) University of Padua / Universita degli Studi di Padova IT Italy (IT) University of Pretoria ZA South Africa (ZA) NovaVet Diagnostics Pty Ltd AU Australia (AU) University of Guelph (U of G) CA Canada (CA) Technische Hochschule Ingolstadt DE Germany (DE)

How to cite

APA:

Bertram, C.A., Marzahl, C., Bartel, A., Stayt, J., Bonsembiante, F., Beeler-Marfisi, J.,... Hill, J. (2022). Cytologic scoring of equine exercise-induced pulmonary hemorrhage: Performance of human experts and a deep learning-based algorithm. Veterinary Pathology. https://doi.org/10.1177/03009858221137582

MLA:

Bertram, Christof A., et al. "Cytologic scoring of equine exercise-induced pulmonary hemorrhage: Performance of human experts and a deep learning-based algorithm." Veterinary Pathology (2022).

BibTeX: Download