-
Data-Centric Label Smoothing for Explainable Glaucoma Screening from Eye Fundus Images
Authors:
Adrian Galdran,
Miguel A. González Ballester
Abstract:
As current computing capabilities increase, modern machine learning and computer vision system tend to increase in complexity, mostly by means of larger models and advanced optimization strategies. Although often neglected, in many problems there is also much to be gained by considering potential improvements in understanding and better leveraging already-available training data, including annotat…
▽ More
As current computing capabilities increase, modern machine learning and computer vision system tend to increase in complexity, mostly by means of larger models and advanced optimization strategies. Although often neglected, in many problems there is also much to be gained by considering potential improvements in understanding and better leveraging already-available training data, including annotations. This so-called data-centric approach can lead to substantial performance increases, sometimes beyond what can be achieved by larger models. In this paper we adopt such an approach for the task of justifiable glaucoma screening from retinal images. In particular, we focus on how to combine information from multiple annotators of different skills into a tailored label smoothing scheme that allows us to better employ a large collection of fundus images, instead of discarding samples suffering from inter-rater variability. Internal validation results indicate that our bespoke label smoothing approach surpasses the performance of a standard resnet50 model and also the same model trained with conventional label smoothing techniques, in particular for the multi-label scenario of predicting clinical reasons of glaucoma likelihood in a highly imbalanced screening context. Our code is made available at github.com/agaldran/justraigs .
△ Less
Submitted 6 June, 2024;
originally announced June 2024.
-
Hardware Density Reduction To Avoid Proximal Junction Failure In Adult Spine Surgery: In Silico Case Studies and Virtual Cohort
Authors:
Morteza Rasouligandomani,
Alex del Arco,
Tomaso Villa,
Luigi La Barbera,
Ferran Pellise,
Miguel Angel Gonzalez Ballester,
Fabio Galbusera,
Jerome Noailly
Abstract:
Background: Proximal Junctional Failure (PJF) is a post-operative complication in adult spine surgery, often requiring reoperation. Osteotomy is often used in revision surgeries, leading to 34.8% complications. Hence, suboptimal decisions might be extending hardware without osteotomy, which yields to severe Global Alignment and Proportion (GAP) scores. High GAPs increase PJF risk, but Hardware Den…
▽ More
Background: Proximal Junctional Failure (PJF) is a post-operative complication in adult spine surgery, often requiring reoperation. Osteotomy is often used in revision surgeries, leading to 34.8% complications. Hence, suboptimal decisions might be extending hardware without osteotomy, which yields to severe Global Alignment and Proportion (GAP) scores. High GAPs increase PJF risk, but Hardware Density Reduction (HDR) might limit it.
Methods: Two clinical cases were evaluated: 1) Initially operated with hardware extended to T10, GAP 10; 2) PJF at T11 and hardware extended to T3, GAP 11. Two patient-personalized spine FE models were constructed through Statistical Shape Modelling (SSM) and mesh morphing. Intervertebral Disk (IVD) fiber strain, screw pull-out force, and rod stress were evaluated for the cases 1) and 2), also for 91 virtual HDR scenarios with different GAP scores, using Finite Element (FE) simulations. Different rod and bone material properties were also assessed.
Results: HDR could decrease IVD fiber strain (-70% at most) and increase screw pull-out forces (+142% at most) for cases with Ti rod and normal bone. Cr-Co rod and osteopenia, and osteoporotic bones had high PJF risk. Trade-off analyses could determine the best configurations avoiding PJF. Virtual cohort study showed that GAP 12 and 13 could not avoid PJF in any HDR scenarios either with Ti or Cr-Co rods. HDR in a UIV T10 virtual patient with GAP 11 could not de-risk in case of Cr-Co rods. UIV T3 with GAP 13 could not benefit any HDR strategy, independently of rod properties. In contrast, Ti rods might allow HDR to de-risk GAP 12 patients with UIV T3.
Conclusions: HDR could avoid PJF in the patients with medium high GAP scores, depending on the screw reduction pattern, and bone and rod material properties. Remarkably, HDR technique might avoid excessive spine surgeries and minimize the surgery cost.
△ Less
Submitted 20 February, 2024;
originally announced February 2024.
-
Data Repository of Finite Element Models of Normal and Deformed Thoracolumbar Spine
Authors:
Morteza Rasouligandomani,
Alex del Arco,
Francis Kiptengwer Chemorion,
Marc-Antonio Bisotti,
Fabio Galbusera,
Jerome Noailly,
Miguel Angel Gonzalez Ballester
Abstract:
Adult spine deformity (ASD) is prevalent and leads to a sagittal misalignment in the vertebral column. Computational methods, including Finite Element (FE) Models, have emerged as valuable tools for investigating the causes and treatment of ASD through biomechanical simulations. However, the process of generating personalized FE models is often complex and time-consuming. To address this challenge…
▽ More
Adult spine deformity (ASD) is prevalent and leads to a sagittal misalignment in the vertebral column. Computational methods, including Finite Element (FE) Models, have emerged as valuable tools for investigating the causes and treatment of ASD through biomechanical simulations. However, the process of generating personalized FE models is often complex and time-consuming. To address this challenge, we present a repository of FE models with diverse spine morphologies that statistically represent real geometries from a cohort of patients. These models are generated using EOS images, which are utilized to reconstruct 3D surface spine models. Subsequently, a Statistical Shape Model (SSM) is constructed, enabling the adaptation of a FE hexahedral mesh template for both the bone and soft tissues of the spine through mesh morphing. The SSM deformation fields facilitate the personalization of the mean hexahedral FE model based on sagittal balance measurements. Ultimately, this new hexahedral SSM tool offers a means to generate a virtual cohort of 16807 thoracolumbar FE spine models, which are openly shared in a public repository.
△ Less
Submitted 20 February, 2024;
originally announced February 2024.
-
Unsupervised Segmentation of Fetal Brain MRI using Deep Learning Cascaded Registration
Authors:
Valentin Comte,
Mireia Alenya,
Andrea Urru,
Judith Recober,
Ayako Nakaki,
Francesca Crovetto,
Oscar Camara,
Eduard Gratacós,
Elisenda Eixarch,
Fàtima Crispi,
Gemma Piella,
Mario Ceresa,
Miguel A. González Ballester
Abstract:
Accurate segmentation of fetal brain magnetic resonance images is crucial for analyzing fetal brain development and detecting potential neurodevelopmental abnormalities. Traditional deep learning-based automatic segmentation, although effective, requires extensive training data with ground-truth labels, typically produced by clinicians through a time-consuming annotation process. To overcome this…
▽ More
Accurate segmentation of fetal brain magnetic resonance images is crucial for analyzing fetal brain development and detecting potential neurodevelopmental abnormalities. Traditional deep learning-based automatic segmentation, although effective, requires extensive training data with ground-truth labels, typically produced by clinicians through a time-consuming annotation process. To overcome this challenge, we propose a novel unsupervised segmentation method based on multi-atlas segmentation, that accurately segments multiple tissues without relying on labeled data for training. Our method employs a cascaded deep learning network for 3D image registration, which computes small, incremental deformations to the moving image to align it precisely with the fixed image. This cascaded network can then be used to register multiple annotated images with the image to be segmented, and combine the propagated labels to form a refined segmentation. Our experiments demonstrate that the proposed cascaded architecture outperforms the state-of-the-art registration methods that were tested. Furthermore, the derived segmentation method achieves similar performance and inference time to nnU-Net while only using a small subset of annotated data for the multi-atlas segmentation task and none for training the network. Our pipeline for registration and multi-atlas segmentation is publicly available at https://github.com/ValBcn/CasReg.
△ Less
Submitted 7 July, 2023;
originally announced July 2023.
-
Multi-Head Multi-Loss Model Calibration
Authors:
Adrian Galdran,
Johan Verjans,
Gustavo Carneiro,
Miguel A. González Ballester
Abstract:
Delivering meaningful uncertainty estimates is essential for a successful deployment of machine learning models in the clinical practice. A central aspect of uncertainty quantification is the ability of a model to return predictions that are well-aligned with the actual probability of the model being correct, also known as model calibration. Although many methods have been proposed to improve cali…
▽ More
Delivering meaningful uncertainty estimates is essential for a successful deployment of machine learning models in the clinical practice. A central aspect of uncertainty quantification is the ability of a model to return predictions that are well-aligned with the actual probability of the model being correct, also known as model calibration. Although many methods have been proposed to improve calibration, no technique can match the simple, but expensive approach of training an ensemble of deep neural networks. In this paper we introduce a form of simplified ensembling that bypasses the costly training and inference of deep ensembles, yet it keeps its calibration capabilities. The idea is to replace the common linear classifier at the end of a network by a set of heads that are supervised with different loss functions to enforce diversity on their predictions. Specifically, each head is trained to minimize a weighted Cross-Entropy loss, but the weights are different among the different branches. We show that the resulting averaged predictions can achieve excellent calibration without sacrificing accuracy in two challenging datasets for histopathological and endoscopic image classification. Our experiments indicate that Multi-Head Multi-Loss classifiers are inherently well-calibrated, outperforming other recent calibration techniques and even challenging Deep Ensembles' performance. Code to reproduce our experiments can be found at \url{https://github.com/agaldran/mhml_calibration} .
△ Less
Submitted 2 March, 2023;
originally announced March 2023.
-
AIROGS: Artificial Intelligence for RObust Glaucoma Screening Challenge
Authors:
Coen de Vente,
Koenraad A. Vermeer,
Nicolas Jaccard,
He Wang,
Hongyi Sun,
Firas Khader,
Daniel Truhn,
Temirgali Aimyshev,
Yerkebulan Zhanibekuly,
Tien-Dung Le,
Adrian Galdran,
Miguel Ángel González Ballester,
Gustavo Carneiro,
Devika R G,
Hrishikesh P S,
Densen Puthussery,
Hong Liu,
Zekang Yang,
Satoshi Kondo,
Satoshi Kasai,
Edward Wang,
Ashritha Durvasula,
Jónathan Heras,
Miguel Ángel Zapata,
Teresa Araújo
, et al. (11 additional authors not shown)
Abstract:
The early detection of glaucoma is essential in preventing visual impairment. Artificial intelligence (AI) can be used to analyze color fundus photographs (CFPs) in a cost-effective manner, making glaucoma screening more accessible. While AI models for glaucoma screening from CFPs have shown promising results in laboratory settings, their performance decreases significantly in real-world scenarios…
▽ More
The early detection of glaucoma is essential in preventing visual impairment. Artificial intelligence (AI) can be used to analyze color fundus photographs (CFPs) in a cost-effective manner, making glaucoma screening more accessible. While AI models for glaucoma screening from CFPs have shown promising results in laboratory settings, their performance decreases significantly in real-world scenarios due to the presence of out-of-distribution and low-quality images. To address this issue, we propose the Artificial Intelligence for Robust Glaucoma Screening (AIROGS) challenge. This challenge includes a large dataset of around 113,000 images from about 60,000 patients and 500 different screening centers, and encourages the development of algorithms that are robust to ungradable and unexpected input data. We evaluated solutions from 14 teams in this paper, and found that the best teams performed similarly to a set of 20 expert ophthalmologists and optometrists. The highest-scoring team achieved an area under the receiver operating characteristic curve of 0.99 (95% CI: 0.98-0.99) for detecting ungradable images on-the-fly. Additionally, many of the algorithms showed robust performance when tested on three other publicly available datasets. These results demonstrate the feasibility of robust AI-enabled glaucoma screening.
△ Less
Submitted 10 February, 2023; v1 submitted 3 February, 2023;
originally announced February 2023.
-
On the Optimal Combination of Cross-Entropy and Soft Dice Losses for Lesion Segmentation with Out-of-Distribution Robustness
Authors:
Adrian Galdran,
Gustavo Carneiro,
Miguel Ángel González Ballester
Abstract:
We study the impact of different loss functions on lesion segmentation from medical images. Although the Cross-Entropy (CE) loss is the most popular option when dealing with natural images, for biomedical image segmentation the soft Dice loss is often preferred due to its ability to handle imbalanced scenarios. On the other hand, the combination of both functions has also been successfully applied…
▽ More
We study the impact of different loss functions on lesion segmentation from medical images. Although the Cross-Entropy (CE) loss is the most popular option when dealing with natural images, for biomedical image segmentation the soft Dice loss is often preferred due to its ability to handle imbalanced scenarios. On the other hand, the combination of both functions has also been successfully applied in this kind of tasks. A much less studied problem is the generalization ability of all these losses in the presence of Out-of-Distribution (OoD) data. This refers to samples appearing in test time that are drawn from a different distribution than training images. In our case, we train our models on images that always contain lesions, but in test time we also have lesion-free samples. We analyze the impact of the minimization of different loss functions on in-distribution performance, but also its ability to generalize to OoD data, via comprehensive experiments on polyp segmentation from endoscopic images and ulcer segmentation from diabetic feet images. Our findings are surprising: CE-Dice loss combinations that excel in segmenting in-distribution images have a poor performance when dealing with OoD data, which leads us to recommend the adoption of the CE loss for this kind of problems, due to its robustness and ability to generalize to OoD samples. Code associated to our experiments can be found at https://github.com/agaldran/lesion_losses_ood .
△ Less
Submitted 14 September, 2022; v1 submitted 13 September, 2022;
originally announced September 2022.
-
Test Time Transform Prediction for Open Set Histopathological Image Recognition
Authors:
Adrian Galdran,
Katherine J. Hewitt,
Narmin L. Ghaffari,
Jakob N. Kather,
Gustavo Carneiro,
Miguel A. González Ballester
Abstract:
Tissue typology annotation in Whole Slide histological images is a complex and tedious, yet necessary task for the development of computational pathology models. We propose to address this problem by applying Open Set Recognition techniques to the task of jointly classifying tissue that belongs to a set of annotated classes, e.g. clinically relevant tissue categories, while rejecting in test time…
▽ More
Tissue typology annotation in Whole Slide histological images is a complex and tedious, yet necessary task for the development of computational pathology models. We propose to address this problem by applying Open Set Recognition techniques to the task of jointly classifying tissue that belongs to a set of annotated classes, e.g. clinically relevant tissue categories, while rejecting in test time Open Set samples, i.e. images that belong to categories not present in the training set. To this end, we introduce a new approach for Open Set histopathological image recognition based on training a model to accurately identify image categories and simultaneously predict which data augmentation transform has been applied. In test time, we measure model confidence in predicting this transform, which we expect to be lower for images in the Open Set. We carry out comprehensive experiments in the context of colorectal cancer assessment from histological images, which provide evidence on the strengths of our approach to automatically identify samples from unknown categories. Code is released at https://github.com/agaldran/t3po .
△ Less
Submitted 27 June, 2022; v1 submitted 20 June, 2022;
originally announced June 2022.
-
Attri-VAE: attribute-based interpretable representations of medical images with variational autoencoders
Authors:
Irem Cetin,
Maialen Stephens,
Oscar Camara,
Miguel Angel Gonzalez Ballester
Abstract:
Deep learning (DL) methods where interpretability is intrinsically considered as part of the model are required to better understand the relationship of clinical and imaging-based attributes with DL outcomes, thus facilitating their use in the reasoning behind medical decisions. Latent space representations built with variational autoencoders (VAE) do not ensure individual control of data attribut…
▽ More
Deep learning (DL) methods where interpretability is intrinsically considered as part of the model are required to better understand the relationship of clinical and imaging-based attributes with DL outcomes, thus facilitating their use in the reasoning behind medical decisions. Latent space representations built with variational autoencoders (VAE) do not ensure individual control of data attributes. Attribute-based methods enforcing attribute disentanglement have been proposed in the literature for classical computer vision tasks in benchmark data. In this paper, we propose a VAE approach, the Attri-VAE, that includes an attribute regularization term to associate clinical and medical imaging attributes with different regularized dimensions in the generated latent space, enabling a better-disentangled interpretation of the attributes. Furthermore, the generated attention maps explained the attribute encoding in the regularized latent space dimensions. Using the Attri-VAE approach we analyzed healthy and myocardial infarction patients with clinical, cardiac morphology, and radiomics attributes. The proposed model provided an excellent trade-off between reconstruction fidelity, disentanglement, and interpretability, outperforming state-of-the-art VAE approaches according to several quantitative metrics. The resulting latent space allowed the generation of realistic synthetic data in the trajectory between two distinct input samples or along a specific attribute dimension to better interpret changes between different cardiac conditions.
△ Less
Submitted 12 December, 2022; v1 submitted 19 March, 2022;
originally announced March 2022.
-
A Hierarchical Multi-Task Approach to Gastrointestinal Image Analysis
Authors:
Adrian Galdran,
Gustavo Carneiro,
Miguel A. González Ballester
Abstract:
A large number of different lesions and pathologies can affect the human digestive system, resulting in life-threatening situations. Early detection plays a relevant role in the successful treatment and the increase of current survival rates to, e.g., colorectal cancer. The standard procedure enabling detection, endoscopic video analysis, generates large quantities of visual data that need to be c…
▽ More
A large number of different lesions and pathologies can affect the human digestive system, resulting in life-threatening situations. Early detection plays a relevant role in the successful treatment and the increase of current survival rates to, e.g., colorectal cancer. The standard procedure enabling detection, endoscopic video analysis, generates large quantities of visual data that need to be carefully analyzed by an specialist. Due to the wide range of color, shape, and general visual appearance of pathologies, as well as highly varying image quality, such process is greatly dependent on the human operator experience and skill. In this work, we detail our solution to the task of multi-category classification of images from the gastrointestinal (GI) human tract within the 2020 Endotect Challenge. Our approach is based on a Convolutional Neural Network minimizing a hierarchical error function that takes into account not only the finding category, but also its location within the GI tract (lower/upper tract), and the type of finding (pathological finding/therapeutic intervention/anatomical landmark/mucosal views' quality). We also describe in this paper our solution for the challenge task of polyp segmentation in colonoscopies, which was addressed with a pretrained double encoder-decoder network. Our internal cross-validation results show an average performance of 91.25 Mathews Correlation Coefficient (MCC) and 91.82 Micro-F1 score for the classification task, and a 92.30 F1 score for the polyp segmentation task. The organization provided feedback on the performance in a hidden test set for both tasks, which resulted in 85.61 MCC and 86.96 F1 score for classification, and 91.97 F1 score for polyp segmentation. At the time of writing no public ranking for this challenge had been released.
△ Less
Submitted 16 November, 2021;
originally announced November 2021.
-
Convolutional Nets Versus Vision Transformers for Diabetic Foot Ulcer Classification
Authors:
Adrian Galdran,
Gustavo Carneiro,
Miguel A. González Ballester
Abstract:
This paper compares well-established Convolutional Neural Networks (CNNs) to recently introduced Vision Transformers for the task of Diabetic Foot Ulcer Classification, in the context of the DFUC 2021 Grand-Challenge, in which this work attained the first position. Comprehensive experiments demonstrate that modern CNNs are still capable of outperforming Transformers in a low-data regime, likely ow…
▽ More
This paper compares well-established Convolutional Neural Networks (CNNs) to recently introduced Vision Transformers for the task of Diabetic Foot Ulcer Classification, in the context of the DFUC 2021 Grand-Challenge, in which this work attained the first position. Comprehensive experiments demonstrate that modern CNNs are still capable of outperforming Transformers in a low-data regime, likely owing to their ability for better exploiting spatial correlations. In addition, we empirically demonstrate that the recent Sharpness-Aware Minimization (SAM) optimization algorithm considerably improves the generalization capability of both kinds of models. Our results demonstrate that for this task, the combination of CNNs and the SAM optimization process results in superior performance than any other of the considered approaches.
△ Less
Submitted 12 November, 2021;
originally announced November 2021.
-
Double Encoder-Decoder Networks for Gastrointestinal Polyp Segmentation
Authors:
Adrian Galdran,
Gustavo Carneiro,
Miguel A. González Ballester
Abstract:
Polyps represent an early sign of the development of Colorectal Cancer. The standard procedure for their detection consists of colonoscopic examination of the gastrointestinal tract. However, the wide range of polyp shapes and visual appearances, as well as the reduced quality of this image modality, turn their automatic identification and segmentation with computational tools into a challenging c…
▽ More
Polyps represent an early sign of the development of Colorectal Cancer. The standard procedure for their detection consists of colonoscopic examination of the gastrointestinal tract. However, the wide range of polyp shapes and visual appearances, as well as the reduced quality of this image modality, turn their automatic identification and segmentation with computational tools into a challenging computer vision task. In this work, we present a new strategy for the delineation of gastrointestinal polyps from endoscopic images based on a direct extension of common encoder-decoder networks for semantic segmentation. In our approach, two pretrained encoder-decoder networks are sequentially stacked: the second network takes as input the concatenation of the original frame and the initial prediction generated by the first network, which acts as an attention mechanism enabling the second network to focus on interesting areas within the image, thereby improving the quality of its predictions. Quantitative evaluation carried out on several polyp segmentation databases shows that double encoder-decoder networks clearly outperform their single encoder-decoder counterparts in all cases. In addition, our best double encoder-decoder combination attains excellent segmentation accuracy and reaches state-of-the-art performance results in all the considered datasets, with a remarkable boost of accuracy on images extracted from datasets not used for training.
△ Less
Submitted 5 October, 2021;
originally announced October 2021.
-
Balanced-MixUp for Highly Imbalanced Medical Image Classification
Authors:
Adrian Galdran,
Gustavo Carneiro,
Miguel A. González Ballester
Abstract:
Highly imbalanced datasets are ubiquitous in medical image classification problems. In such problems, it is often the case that rare classes associated to less prevalent diseases are severely under-represented in labeled databases, typically resulting in poor performance of machine learning algorithms due to overfitting in the learning process. In this paper, we propose a novel mechanism for sampl…
▽ More
Highly imbalanced datasets are ubiquitous in medical image classification problems. In such problems, it is often the case that rare classes associated to less prevalent diseases are severely under-represented in labeled databases, typically resulting in poor performance of machine learning algorithms due to overfitting in the learning process. In this paper, we propose a novel mechanism for sampling training data based on the popular MixUp regularization technique, which we refer to as Balanced-MixUp. In short, Balanced-MixUp simultaneously performs regular (i.e., instance-based) and balanced (i.e., class-based) sampling of the training data. The resulting two sets of samples are then mixed-up to create a more balanced training distribution from which a neural network can effectively learn without incurring in heavily under-fitting the minority classes. We experiment with a highly imbalanced dataset of retinal images (55K samples, 5 classes) and a long-tail dataset of gastro-intestinal video frames (10K images, 23 classes), using two CNNs of varying representation capabilities. Experimental results demonstrate that applying Balanced-MixUp outperforms other conventional sampling schemes and loss functions specifically designed to deal with imbalanced data. Code is released at https://github.com/agaldran/balanced_mixup .
△ Less
Submitted 20 September, 2021;
originally announced September 2021.
-
Memory-aware curriculum federated learning for breast cancer classification
Authors:
Amelia Jiménez-Sánchez,
Mickael Tardy,
Miguel A. González Ballester,
Diana Mateus,
Gemma Piella
Abstract:
For early breast cancer detection, regular screening with mammography imaging is recommended. Routinary examinations result in datasets with a predominant amount of negative samples. A potential solution to such class-imbalance is joining forces across multiple institutions. Developing a collaborative computer-aided diagnosis system is challenging in different ways. Patient privacy and regulations…
▽ More
For early breast cancer detection, regular screening with mammography imaging is recommended. Routinary examinations result in datasets with a predominant amount of negative samples. A potential solution to such class-imbalance is joining forces across multiple institutions. Developing a collaborative computer-aided diagnosis system is challenging in different ways. Patient privacy and regulations need to be carefully respected. Data across institutions may be acquired from different devices or imaging protocols, leading to heterogeneous non-IID data. Also, for learning-based methods, new optimization strategies working on distributed data are required. Recently, federated learning has emerged as an effective tool for collaborative learning. In this setting, local models perform computation on their private data to update the global model. The order and the frequency of local updates influence the final global model. Hence, the order in which samples are locally presented to the optimizers plays an important role. In this work, we define a memory-aware curriculum learning method for the federated setting. Our curriculum controls the order of the training samples paying special attention to those that are forgotten after the deployment of the global model. Our approach is combined with unsupervised domain adaptation to deal with domain shift while preserving data privacy. We evaluate our method with three clinical datasets from different vendors. Our results verify the effectiveness of federated adversarial learning for the multi-site breast cancer classification. Moreover, we show that our proposed memory-aware curriculum method is beneficial to further improve classification performance. Our code is publicly available at: https://github.com/ameliajimenez/curriculum-federated-learning.
△ Less
Submitted 6 January, 2023; v1 submitted 6 July, 2021;
originally announced July 2021.
-
An Uncertainty-aware Hierarchical Probabilistic Network for Early Prediction, Quantification and Segmentation of Pulmonary Tumour Growth
Authors:
Xavier Rafael-Palou,
Anton Aubanell,
Mario Ceresa,
Vicent Ribas,
Gemma Piella,
Miguel A. González Ballester
Abstract:
Early detection and quantification of tumour growth would help clinicians to prescribe more accurate treatments and provide better surgical planning. However, the multifactorial and heterogeneous nature of lung tumour progression hampers identification of growth patterns. In this study, we present a novel method based on a deep hierarchical generative and probabilistic framework that, according to…
▽ More
Early detection and quantification of tumour growth would help clinicians to prescribe more accurate treatments and provide better surgical planning. However, the multifactorial and heterogeneous nature of lung tumour progression hampers identification of growth patterns. In this study, we present a novel method based on a deep hierarchical generative and probabilistic framework that, according to radiological guidelines, predicts tumour growth, quantifies its size and provides a semantic appearance of the future nodule. Unlike previous deterministic solutions, the generative characteristic of our approach also allows us to estimate the uncertainty in the predictions, especially important for complex and doubtful cases. Results of evaluating this method on an independent test set reported a tumour growth balanced accuracy of 74%, a tumour growth size MAE of 1.77 mm and a tumour segmentation Dice score of 78%. These surpassed the performances of equivalent deterministic and alternative generative solutions (i.e. probabilistic U-Net, Bayesian test dropout and Pix2Pix GAN) confirming the suitability of our approach.
△ Less
Submitted 18 April, 2021;
originally announced April 2021.
-
Detection, growth quantification and malignancy prediction of pulmonary nodules using deep convolutional networks in follow-up CT scans
Authors:
Xavier Rafael-Palou,
Anton Aubanell,
Mario Ceresa,
Vicent Ribas,
Gemma Piella,
Miguel A. González Ballester
Abstract:
We address the problem of supporting radiologists in the longitudinal management of lung cancer. Therefore, we proposed a deep learning pipeline, composed of four stages that completely automatized from the detection of nodules to the classification of cancer, through the detection of growth in the nodules. In addition, the pipeline integrated a novel approach for nodule growth detection, which re…
▽ More
We address the problem of supporting radiologists in the longitudinal management of lung cancer. Therefore, we proposed a deep learning pipeline, composed of four stages that completely automatized from the detection of nodules to the classification of cancer, through the detection of growth in the nodules. In addition, the pipeline integrated a novel approach for nodule growth detection, which relied on a recent hierarchical probabilistic U-Net adapted to report uncertainty estimates. Also, a second novel method was introduced for lung cancer nodule classification, integrating into a two stream 3D-CNN network the estimated nodule malignancy probabilities derived from a pretrained nodule malignancy network. The pipeline was evaluated in a longitudinal cohort and reported comparable performances to the state of art.
△ Less
Submitted 26 March, 2021;
originally announced March 2021.
-
Curriculum learning for improved femur fracture classification: scheduling data with prior knowledge and uncertainty
Authors:
Amelia Jiménez-Sánchez,
Diana Mateus,
Sonja Kirchhoff,
Chlodwig Kirchhoff,
Peter Biberthaler,
Nassir Navab,
Miguel A. González Ballester,
Gemma Piella
Abstract:
An adequate classification of proximal femur fractures from X-ray images is crucial for the treatment choice and the patients' clinical outcome. We rely on the commonly used AO system, which describes a hierarchical knowledge tree classifying the images into types and subtypes according to the fracture's location and complexity. In this paper, we propose a method for the automatic classification o…
▽ More
An adequate classification of proximal femur fractures from X-ray images is crucial for the treatment choice and the patients' clinical outcome. We rely on the commonly used AO system, which describes a hierarchical knowledge tree classifying the images into types and subtypes according to the fracture's location and complexity. In this paper, we propose a method for the automatic classification of proximal femur fractures into 3 and 7 AO classes based on a Convolutional Neural Network (CNN). As it is known, CNNs need large and representative datasets with reliable labels, which are hard to collect for the application at hand. In this paper, we design a curriculum learning (CL) approach that improves over the basic CNNs performance under such conditions. Our novel formulation reunites three curriculum strategies: individually weighting training samples, reordering the training set, and sampling subsets of data. The core of these strategies is a scoring function ranking the training samples. We define two novel scoring functions: one from domain-specific prior knowledge and an original self-paced uncertainty score. We perform experiments on a clinical dataset of proximal femur radiographs. The curriculum improves proximal femur fracture classification up to the performance of experienced trauma surgeons. The best curriculum method reorders the training set based on prior knowledge resulting into a classification improvement of 15%. Using the publicly available MNIST dataset, we further discuss and demonstrate the benefits of our unified CL formulation for three controlled and challenging digit recognition scenarios: with limited amounts of data, under class-imbalance, and in the presence of label noise. The code of our work is available at: https://github.com/ameliajimenez/curriculum-learning-prior-uncertainty.
△ Less
Submitted 9 November, 2021; v1 submitted 31 July, 2020;
originally announced July 2020.
-
Handling confounding variables in statistical shape analysis -- application to cardiac remodelling
Authors:
Gabriel Bernardino,
Oualid Benkarim,
María Sanz-de la Garza,
Susanna Prat-Gonzàlez,
Álvaro Sepulveda-Martinez,
Fàtima Crispi,
Marta Sitges,
Mathieu De Craene,
Bart Bijnens,
Miguel Ángel González Ballester
Abstract:
Statistical shape analysis is a powerful tool to assess organ morphologies and find shape changes associated to a particular disease. However, imbalance in confounding factors, such as demographics might invalidate the analysis if not taken into consideration. Despite the methodological advances in the field, providing new methods that are able to capture complex and regional shape differences, th…
▽ More
Statistical shape analysis is a powerful tool to assess organ morphologies and find shape changes associated to a particular disease. However, imbalance in confounding factors, such as demographics might invalidate the analysis if not taken into consideration. Despite the methodological advances in the field, providing new methods that are able to capture complex and regional shape differences, the relationship between non-imaging information and shape variability has been overlooked. We present a linear statistical shape analysis framework that finds shape differences unassociated to a controlled set of confounding variables. It includes two confounding correction methods: confounding deflation and adjustment. We applied our framework to a cardiac magnetic resonance imaging dataset, consisting of the cardiac ventricles of 89 triathletes and 77 controls, to identify cardiac remodelling due to the practice of endurance exercise. To test robustness to confounders, subsets of this dataset were generated by randomly removing controls with low body mass index, thus introducing imbalance. The analysis of the whole dataset indicates an increase of ventricular volumes and myocardial mass in athletes, which is consistent with the clinical literature. However, when confounders are not taken into consideration no increase of myocardial mass is found. Using the downsampled datasets, we find that confounder adjustment methods are needed to find the real remodelling patterns in imbalanced datasets.
△ Less
Submitted 28 July, 2020;
originally announced July 2020.
-
A radiomics approach to analyze cardiac alterations in hypertension
Authors:
Irem Cetin,
Steffen E. Petersen,
Sandy Napel,
Oscar Camara,
Miguel Angel Gonzalez Ballester,
Karim Lekadir
Abstract:
Hypertension is a medical condition that is well-established as a risk factor for many major diseases. For example, it can cause alterations in the cardiac structure and function over time that can lead to heart related morbidity and mortality. However, at the subclinical stage, these changes are subtle and cannot be easily captured using conventional cardiovascular indices calculated from clinica…
▽ More
Hypertension is a medical condition that is well-established as a risk factor for many major diseases. For example, it can cause alterations in the cardiac structure and function over time that can lead to heart related morbidity and mortality. However, at the subclinical stage, these changes are subtle and cannot be easily captured using conventional cardiovascular indices calculated from clinical cardiac imaging. In this paper, we describe a radiomics approach for identifying intermediate imaging phenotypes associated with hypertension. The method combines feature selection and machine learning techniques to identify the most subtle as well as complex structural and tissue changes in hypertensive subgroups as compared to healthy individuals. Validation based on a sample of asymptomatic hearts that include both hypertensive and non-hypertensive cases demonstrate that the proposed radiomics model is capable of detecting intensity and textural changes well beyond the capabilities of conventional imaging phenotypes, indicating its potential for improved understanding of the longitudinal effects of hypertension on cardiovascular health and disease.
△ Less
Submitted 21 July, 2020;
originally announced July 2020.
-
Pulmonary Nodule Malignancy Classification Using its Temporal Evolution with Two-Stream 3D Convolutional Neural Networks
Authors:
Xavier Rafael-Palou,
Anton Aubanell,
Ilaria Bonavita,
Mario Ceresa,
Gemma Piella,
Vicent Ribas,
Miguel A. González Ballester
Abstract:
Nodule malignancy assessment is a complex, time-consuming and error-prone task. Current clinical practice requires measuring changes in size and density of the nodule at different time-points. State of the art solutions rely on 3D convolutional neural networks built on pulmonary nodules obtained from single CT scan per patient. In this work, we propose a two-stream 3D convolutional neural network…
▽ More
Nodule malignancy assessment is a complex, time-consuming and error-prone task. Current clinical practice requires measuring changes in size and density of the nodule at different time-points. State of the art solutions rely on 3D convolutional neural networks built on pulmonary nodules obtained from single CT scan per patient. In this work, we propose a two-stream 3D convolutional neural network that predicts malignancy by jointly analyzing two pulmonary nodule volumes from the same patient taken at different time-points. Best results achieve 77% of F1-score in test with an increment of 9% and 12% of F1-score with respect to the same network trained with images from a single time-point.
△ Less
Submitted 22 May, 2020;
originally announced May 2020.
-
Medical-based Deep Curriculum Learning for Improved Fracture Classification
Authors:
Amelia Jiménez-Sánchez,
Diana Mateus,
Sonja Kirchhoff,
Chlodwig Kirchhoff,
Peter Biberthaler,
Nassir Navab,
Miguel A. González Ballester,
Gemma Piella
Abstract:
Current deep-learning based methods do not easily integrate to clinical protocols, neither take full advantage of medical knowledge. In this work, we propose and compare several strategies relying on curriculum learning, to support the classification of proximal femur fracture from X-ray images, a challenging problem as reflected by existing intra- and inter-expert disagreement. Our strategies are…
▽ More
Current deep-learning based methods do not easily integrate to clinical protocols, neither take full advantage of medical knowledge. In this work, we propose and compare several strategies relying on curriculum learning, to support the classification of proximal femur fracture from X-ray images, a challenging problem as reflected by existing intra- and inter-expert disagreement. Our strategies are derived from knowledge such as medical decision trees and inconsistencies in the annotations of multiple experts, which allows us to assign a degree of difficulty to each training sample. We demonstrate that if we start learning "easy" examples and move towards "hard", the model can reach a better performance, even with fewer data. The evaluation is performed on the classification of a clinical dataset of about 1000 X-ray images. Our results show that, compared to class-uniform and random strategies, the proposed medical knowledge-based curriculum, performs up to 15% better in terms of accuracy, achieving the performance of experienced trauma surgeons.
△ Less
Submitted 1 April, 2020;
originally announced April 2020.
-
Volumetric parcellation of the right ventricle for regional geometric and functional assessment
Authors:
Gabriel Bernardino,
Amir Hodzic,
Helene Langet,
Damien LeGallois,
Mathieu De Craene,
Miguel Angel González Ballester,
Eric Saloux,
Bart Bijnens
Abstract:
3D echocardiography is an increasingly popular tool for assessing cardiac remodelling in the right ventricle (RV). It allows quantification of the cardiac chambers without any geometric assumptions, which is the main weakness of 2D echocardiography. However, regional quantification of geometry and function is limited by the lower spatial and temporal resolution and the scarcity of identifiable ana…
▽ More
3D echocardiography is an increasingly popular tool for assessing cardiac remodelling in the right ventricle (RV). It allows quantification of the cardiac chambers without any geometric assumptions, which is the main weakness of 2D echocardiography. However, regional quantification of geometry and function is limited by the lower spatial and temporal resolution and the scarcity of identifiable anatomical landmarks. We developed a technique for regionally assessing the 3 relevant RV regions: apical, inlet and outflow. The method's inputs are end-diastolic (ED) and end-systolic (ES) segmented 3D surface models. The method first defines a partition of the ED endocardium using the geodesic distances from each surface point to apex, tricuspid valve and pulmonary valve: the landmarks that define the 3 regions. The ED surface mesh is then tetrahedralised, and the endocardial-defined partition is interpolated in the blood cavity via the Laplace equation. For obtaining an ES partition, the endocardial partition is transported from ED to ES using a commercial image-based tracking, and then interpolated towards the endocardium, similarly to ED, for computing volumes and ejection fraction (EF). We present a full assessment of the method's validity and reproducibility. First, we assess reproducibility under segmentation variability, obtaining intra- and inter- observer errors (4-10% and 10-23% resp.). Finally, we use a synthetic remodelling dataset to identify the situations in which our method is able to correctly determine the region that has remodelled. This dataset is generated by a novel mesh reconstruction method that deforms a reference mesh, locally imposing a given strain, expressed in anatomical coordinates. We show that the parcellation method is adequate for capturing local circumferential and global circumferential and longitudinal RV remodelling.
△ Less
Submitted 6 April, 2021; v1 submitted 18 March, 2020;
originally announced March 2020.
-
Re-Identification and Growth Detection of Pulmonary Nodules without Image Registration Using 3D Siamese Neural Networks
Authors:
Xavier Rafael-Palou,
Anton Aubanell,
Ilaria Bonavita,
Mario Ceresa,
Gemma Piella,
Vicent Ribas,
Miguel Ángel González Ballester
Abstract:
Lung cancer follow-up is a complex, error prone, and time consuming task for clinical radiologists. Several lung CT scan images taken at different time points of a given patient need to be individually inspected, looking for possible cancerogenous nodules. Radiologists mainly focus their attention in nodule size, density, and growth to assess the existence of malignancy. In this study, we present…
▽ More
Lung cancer follow-up is a complex, error prone, and time consuming task for clinical radiologists. Several lung CT scan images taken at different time points of a given patient need to be individually inspected, looking for possible cancerogenous nodules. Radiologists mainly focus their attention in nodule size, density, and growth to assess the existence of malignancy. In this study, we present a novel method based on a 3D siamese neural network, for the re-identification of nodules in a pair of CT scans of the same patient without the need for image registration. The network was integrated into a two-stage automatic pipeline to detect, match, and predict nodule growth given pairs of CT scans. Results on an independent test set reported a nodule detection sensitivity of 94.7%, an accuracy for temporal nodule matching of 88.8%, and a sensitivity of 92.0% with a precision of 88.4% for nodule growth detection.
△ Less
Submitted 22 December, 2019;
originally announced December 2019.
-
Integration of Convolutional Neural Networks for Pulmonary Nodule Malignancy Assessment in a Lung Cancer Classification Pipeline
Authors:
Ilaria Bonavita,
Xavier Rafael-Palou,
Mario Ceresa,
Gemma Piella,
Vicent Ribas,
Miguel A. González Ballester
Abstract:
The early identification of malignant pulmonary nodules is critical for better lung cancer prognosis and less invasive chemo or radio therapies. Nodule malignancy assessment done by radiologists is extremely useful for planning a preventive intervention but is, unfortunately, a complex, time-consuming and error-prone task. This explains the lack of large datasets containing radiologists malignancy…
▽ More
The early identification of malignant pulmonary nodules is critical for better lung cancer prognosis and less invasive chemo or radio therapies. Nodule malignancy assessment done by radiologists is extremely useful for planning a preventive intervention but is, unfortunately, a complex, time-consuming and error-prone task. This explains the lack of large datasets containing radiologists malignancy characterization of nodules. In this article, we propose to assess nodule malignancy through 3D convolutional neural networks and to integrate it in an automated end-to-end existing pipeline of lung cancer detection. For training and testing purposes we used independent subsets of the LIDC dataset. Adding the probabilities of nodules malignity in a baseline lung cancer pipeline improved its F1-weighted score by 14.7%, whereas integrating the malignancy model itself using transfer learning outperformed the baseline prediction by 11.8% of F1-weighted score. Despite the limited size of the lung cancer datasets, integrating predictive models of nodule malignancy improves prediction of lung cancer.
△ Less
Submitted 18 December, 2019;
originally announced December 2019.
-
Combining Multi-Sequence and Synthetic Images for Improved Segmentation of Late Gadolinium Enhancement Cardiac MRI
Authors:
Víctor M. Campello,
Carlos Martín-Isla,
Cristian Izquierdo,
Steffen E. Petersen,
Miguel A. González Ballester,
Karim Lekadir
Abstract:
Accurate segmentation of the cardiac boundaries in late gadolinium enhancement magnetic resonance images (LGE-MRI) is a fundamental step for accurate quantification of scar tissue. However, while there are many solutions for automatic cardiac segmentation of cine images, the presence of scar tissue can make the correct delineation of the myocardium in LGE-MRI challenging even for human experts. As…
▽ More
Accurate segmentation of the cardiac boundaries in late gadolinium enhancement magnetic resonance images (LGE-MRI) is a fundamental step for accurate quantification of scar tissue. However, while there are many solutions for automatic cardiac segmentation of cine images, the presence of scar tissue can make the correct delineation of the myocardium in LGE-MRI challenging even for human experts. As part of the Multi-Sequence Cardiac MR Segmentation Challenge, we propose a solution for LGE-MRI segmentation based on two components. First, a generative adversarial network is trained for the task of modality-to-modality translation between cine and LGE-MRI sequences to obtain extra synthetic images for both modalities. Second, a deep learning model is trained for segmentation with different combinations of original, augmented and synthetic sequences. Our results based on three magnetic resonance sequences (LGE, bSSFP and T2) from 45 different patients show that the multi-sequence model training integrating synthetic images and data augmentation improves in the segmentation over conventional training with real datasets. In conclusion, the accuracy of the segmentation of LGE-MRI images can be improved by using complementary information provided by non-contrast MRI sequences.
△ Less
Submitted 13 January, 2020; v1 submitted 3 September, 2019;
originally announced September 2019.
-
3D convolutional neural network for abdominal aortic aneurysm segmentation
Authors:
Karen López-Linares,
Inmaculada García,
Ainhoa García-Familiar,
Iván Macía,
Miguel A. González Ballester
Abstract:
An abdominal aortic aneurysm (AAA) is a focal dilation of the aorta that, if not treated, tends to grow and may rupture. A significant unmet need in the assessment of AAA disease, for the diagnosis, prognosis and follow-up, is the determination of rupture risk, which is currently based on the manual measurement of the aneurysm diameter in a selected Computed Tomography Angiography (CTA) scan. Howe…
▽ More
An abdominal aortic aneurysm (AAA) is a focal dilation of the aorta that, if not treated, tends to grow and may rupture. A significant unmet need in the assessment of AAA disease, for the diagnosis, prognosis and follow-up, is the determination of rupture risk, which is currently based on the manual measurement of the aneurysm diameter in a selected Computed Tomography Angiography (CTA) scan. However, there is a lack of standardization determining the degree and rate of disease progression, due to the lack of robust, automated aneurysm segmentation tools that allow quantitatively analyzing the AAA. In this work, we aim at proposing the first 3D convolutional neural network for the segmentation of aneurysms both from preoperative and postoperative CTA scans. We extensively validate its performance in terms of diameter measurements, to test its applicability in the clinical practice, as well as regarding the relative volume difference, and Dice and Jaccard scores. The proposed method yields a mean diameter measurement error of 3.3 mm, a relative volume difference of 8.58 %, and Dice and Jaccard scores of 87 % and 77 %, respectively. At a clinical level, an aneurysm enlargement of 10 mm is considered relevant, thus, our method is suitable to automatically determine the AAA diameter and opens up the opportunity for more complex aneurysm analysis.
△ Less
Submitted 3 March, 2019;
originally announced March 2019.
-
Computational Anatomy for Multi-Organ Analysis in Medical Imaging: A Review
Authors:
Juan J. Cerrolaza,
Mirella Lopez-Picazo,
Ludovic Humbert,
Yoshinobu Sato,
Daniel Rueckert,
Miguel Angel Gonzalez Ballester,
Marius George Linguraru
Abstract:
The medical image analysis field has traditionally been focused on the development of organ-, and disease-specific methods. Recently, the interest in the development of more 20 comprehensive computational anatomical models has grown, leading to the creation of multi-organ models. Multi-organ approaches, unlike traditional organ-specific strategies, incorporate inter-organ relations into the model,…
▽ More
The medical image analysis field has traditionally been focused on the development of organ-, and disease-specific methods. Recently, the interest in the development of more 20 comprehensive computational anatomical models has grown, leading to the creation of multi-organ models. Multi-organ approaches, unlike traditional organ-specific strategies, incorporate inter-organ relations into the model, thus leading to a more accurate representation of the complex human anatomy. Inter-organ relations are not only spatial, but also functional and physiological. Over the years, the strategies 25 proposed to efficiently model multi-organ structures have evolved from the simple global modeling, to more sophisticated approaches such as sequential, hierarchical, or machine learning-based models. In this paper, we present a review of the state of the art on multi-organ analysis and associated computation anatomy methodology. The manuscript follows a methodology-based classification of the different techniques 30 available for the analysis of multi-organs and multi-anatomical structures, from techniques using point distribution models to the most recent deep learning-based approaches. With more than 300 papers included in this review, we reflect on the trends and challenges of the field of computational anatomy, the particularities of each anatomical region, and the potential of multi-organ analysis to increase the impact of 35 medical imaging applications on the future of healthcare.
△ Less
Submitted 20 December, 2018;
originally announced December 2018.
-
Fully automatic detection and segmentation of abdominal aortic thrombus in post-operative CTA images using deep convolutional neural networks
Authors:
Karen López-Linares,
Nerea Aranjuelo,
Luis Kabongo,
Gregory Maclair,
Nerea Lete,
Mario Ceresa,
Ainhoa García-Familiar,
Iván Macía,
Miguel A. González Ballester
Abstract:
Computerized Tomography Angiography (CTA) based follow-up of Abdominal Aortic Aneurysms (AAA) treated with Endovascular Aneurysm Repair (EVAR) is essential to evaluate the progress of the patient and detect complications. In this context, accurate quantification of post-operative thrombus volume is required. However, a proper evaluation is hindered by the lack of automatic, robust and reproducible…
▽ More
Computerized Tomography Angiography (CTA) based follow-up of Abdominal Aortic Aneurysms (AAA) treated with Endovascular Aneurysm Repair (EVAR) is essential to evaluate the progress of the patient and detect complications. In this context, accurate quantification of post-operative thrombus volume is required. However, a proper evaluation is hindered by the lack of automatic, robust and reproducible thrombus segmentation algorithms. We propose a new fully automatic approach based on Deep Convolutional Neural Networks (DCNN) for robust and reproducible thrombus region of interest detection and subsequent fine thrombus segmentation. The DetecNet detection network is adapted to perform region of interest extraction from a complete CTA and a new segmentation network architecture, based on Fully Convolutional Networks and a Holistically-Nested Edge Detection Network, is presented. These networks are trained, validated and tested in 13 post-operative CTA volumes of different patients using a 4-fold cross-validation approach to provide more robustness to the results. Our pipeline achieves a Dice score of more than 82% for post-operative thrombus segmentation and provides a mean relative volume difference between ground truth and automatic segmentation that lays within the experienced human observer variance without the need of human intervention in most common cases.
△ Less
Submitted 1 April, 2018;
originally announced April 2018.