Structural-based uncertainty in deep learning across anatomical scales: Analysis in white matter lesion segmentation

Nataliia Molchanova; Vatsal Raina; Andrey Malinin; Francesco La Rosa; Adrien Depeursinge; Mark Gales; Cristina Granziera; Henning Müller; Mara Graziani; Meritxell Bach Cuadra

doi:10.1016/j.compbiomed.2024.109336

Structural-based uncertainty in deep learning across anatomical scales: Analysis in white matter lesion segmentation

Comput Biol Med. 2024 Nov 14:184:109336. doi: 10.1016/j.compbiomed.2024.109336. Online ahead of print.

Authors

Affiliations

¹ Radiology Department, University of Lausanne and Lausanne University Hospital, Lausanne, Switzerland; MedGIFT, Institute of Informatics, School of Management, HES-SO Valais-Wallis University of Applied Sciences and Arts Western Switzerland, Sierre, Switzerland; CIBM Center for Biomedical Imaging, Lausanne, Switzerland. Electronic address: [email protected].
² ALTA Institute, University of Cambridge, Cambridge, United Kingdom.
³ Isomorphic Labs, London, United Kingdom.
⁴ Icahn School of Medicine at Mount Sinai, New York City, United States of America.
⁵ Radiology Department, University of Lausanne and Lausanne University Hospital, Lausanne, Switzerland; MedGIFT, Institute of Informatics, School of Management, HES-SO Valais-Wallis University of Applied Sciences and Arts Western Switzerland, Sierre, Switzerland.
⁶ Translational Imaging in Neurology (ThINK) Basel, Department of Medicine and Biomedical Engineering, University Hospital Basel and University of Basel, Basel, Switzerland; Department of Neurology, University Hospital Basel, Basel, Switzerland; Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB), University Hospital Basel and University of Basel, Basel, Switzerland.
⁷ MedGIFT, Institute of Informatics, School of Management, HES-SO Valais-Wallis University of Applied Sciences and Arts Western Switzerland, Sierre, Switzerland; Department of Radiology and Medical Informatics, University of Geneva, Geneva, Switzerland.
⁸ MedGIFT, Institute of Informatics, School of Management, HES-SO Valais-Wallis University of Applied Sciences and Arts Western Switzerland, Sierre, Switzerland.
⁹ Radiology Department, University of Lausanne and Lausanne University Hospital, Lausanne, Switzerland; CIBM Center for Biomedical Imaging, Lausanne, Switzerland.

PMID: 39546878
DOI: 10.1016/j.compbiomed.2024.109336

Abstract

This paper explores uncertainty quantification (UQ) as an indicator of the trustworthiness of automated deep-learning (DL) tools in the context of white matter lesion (WML) segmentation from magnetic resonance imaging (MRI) scans of multiple sclerosis (MS) patients. Our study focuses on two principal aspects of uncertainty in structured output segmentation tasks. First, we postulate that a reliable uncertainty measure should indicate predictions likely to be incorrect with high uncertainty values. Second, we investigate the merit of quantifying uncertainty at different anatomical scales (voxel, lesion, or patient). We hypothesize that uncertainty at each scale is related to specific types of errors. Our study aims to confirm this relationship by conducting separate analyses for in-domain and out-of-domain settings. Our primary methodological contributions are (i) the development of novel measures for quantifying uncertainty at lesion and patient scales, derived from structural prediction discrepancies, and (ii) the extension of an error retention curve analysis framework to facilitate the evaluation of UQ performance at both lesion and patient scales. The results from a multi-centric MRI dataset of 444 patients demonstrate that our proposed measures more effectively capture model errors at the lesion and patient scales compared to measures that average voxel-scale uncertainty values. We provide the UQ protocols code at https://github.com/Medical-Image-Analysis-Laboratory/MS_WML_uncs.

Keywords: Deep learning; Instancescale uncertainty; Magnetic resonance imaging; Multiple sclerosis; Patientscale uncertainty; Uncertainty quantification; White matter lesion segmentation.