Discordance, accuracy and reproducibility study of pathologists' diagnosis of melanoma and melanocytic tumors

Haggenmüller S, Wies C, Abels J, Winterstein JT, Heinlein L, Nogueira Garcia C, Utikal JS, Wohlfeil SA, Meier F, Hobelsberger S, Gellrich FF, Sergon M, Hauschild A, French LE, Heinzerling L, Schlager JG, Ghoreschi K, Schlaak M, Hilke FJ, Poch G, Korsing S, Sarfert C, Berking C, Heppt M, Erdmann M, Haferkamp S, Drexler K, Schadendorf D, Sondermann W, Goebeler M, Schilling B, Kather JN, Fröhling S, Llamas-Velasco M, Requena LC, Ferrara G, Fernandez-Figueras M, Fraitag S, Müller CS, Starz H, Kutzner H, Barnhill R, Carr R, Resnik KS, Braun SA, Holland-Letz T, Brinker TJ (2025)

Publication Type: Journal article

Publication year: 2025

Journal

Book Volume: 16

Pages Range: 789-

Journal Issue: 1

DOI: 10.1038/s41467-025-56160-x

Abstract

Accurate melanoma diagnosis is crucial for patient outcomes and reliability of AI diagnostic tools. We assess interrater variability among eight expert pathologists reviewing histopathological images and clinical metadata of 792 melanoma-suspicious lesions prospectively collected at eight German hospitals. Moreover, we provide access to the largest panel-validated dataset featuring dermoscopic and histopathological images with metadata. Complete agreement is achieved in 53.5% of cases (424/792), and a majority vote ( ≥ five pathologists) in 90.9% (720/792). Considerable discordance is observed for non-invasive melanomas (complete agreement in only 10/73 cases). The expert panel disagrees with the local pathologists' and dermatologists' diagnoses in 14.9% and 33.5% of cases, respectively. This variability highlights the diagnostic challenges of early-stage melanomas and the need to reconsider how ground truth is established in routine care and AI research. Including at least two pathologists or virtual panels may contribute to more consistent diagnostic results.

Authors with CRIS profile

Involved external institutions

Deutsches Krebsforschungszentrum (DKFZ) Germany (DE) Universitätsklinikum Mannheim Germany (DE) Nationales Centrum für Tumorerkrankungen (NCT) Heidelberg Germany (DE) Universitätsklinikum Carl Gustav Carus Dresden Germany (DE) Universitätsklinikum Schleswig-Holstein (UKSH) Germany (DE) Klinikum der Universität München Germany (DE) Charité - Universitätsmedizin Berlin Germany (DE) Hospital Universitario de La Princesa Spain (ES) Universitätsklinikum Regensburg Germany (DE) Universitätsklinikum Essen Germany (DE) MVZ für Histologie, Zytologie und Molekulare Diagnostik Germany (DE) Dermpath München Germany (DE) Universitätsklinikum Würzburg Germany (DE) Universitätsklinikum Frankfurt am Main (KGU) Germany (DE) Institut Curie France (FR) Technische Universität Dresden Germany (DE) Warwick Hospital United Kingdom (GB) Westfälische Wilhelms-Universität (WWU) Münster Germany (DE) Universidad Autónoma de Madrid (UAM) Spain (ES) Istituto Nazionale Tumori IRCCS "Fondazione G. Pascale" Italy (IT) Universitat Internacional de Catalunya Spain (ES) Université Sorbonne Paris Cité France (FR)

How to cite

APA:

Haggenmüller, S., Wies, C., Abels, J., Winterstein, J.T., Heinlein, L., Nogueira Garcia, C.,... Brinker, T.J. (2025). Discordance, accuracy and reproducibility study of pathologists' diagnosis of melanoma and melanocytic tumors. Nature Communications, 16(1), 789-. https://doi.org/10.1038/s41467-025-56160-x

MLA:

Haggenmüller, Sarah, et al. "Discordance, accuracy and reproducibility study of pathologists' diagnosis of melanoma and melanocytic tumors." Nature Communications 16.1 (2025): 789-.

BibTeX: Download