-
M&M: Tackling False Positives in Mammography with a Multi-view and Multi-instance Learning Sparse Detector
Authors:
Yen Nhi Truong Vu,
Dan Guo,
Ahmed Taha,
Jason Su,
Thomas Paul Matthews
Abstract:
Deep-learning-based object detection methods show promise for improving screening mammography, but high rates of false positives can hinder their effectiveness in clinical practice. To reduce false positives, we identify three challenges: (1) unlike natural images, a malignant mammogram typically contains only one malignant finding; (2) mammography exams contain two views of each breast, and both…
▽ More
Deep-learning-based object detection methods show promise for improving screening mammography, but high rates of false positives can hinder their effectiveness in clinical practice. To reduce false positives, we identify three challenges: (1) unlike natural images, a malignant mammogram typically contains only one malignant finding; (2) mammography exams contain two views of each breast, and both views ought to be considered to make a correct assessment; (3) most mammograms are negative and do not contain any findings. In this work, we tackle the three aforementioned challenges by: (1) leveraging Sparse R-CNN and showing that sparse detectors are more appropriate than dense detectors for mammography; (2) including a multi-view cross-attention module to synthesize information from different views; (3) incorporating multi-instance learning (MIL) to train with unannotated images and perform breast-level classification. The resulting model, M&M, is a Multi-view and Multi-instance learning system that can both localize malignant findings and provide breast-level predictions. We validate M&M's detection and classification performance using five mammography datasets. In addition, we demonstrate the effectiveness of each proposed component through comprehensive ablation studies.
△ Less
Submitted 11 August, 2023;
originally announced August 2023.
-
Problems and shortcuts in deep learning for screening mammography
Authors:
Trevor Tsue,
Brent Mombourquette,
Ahmed Taha,
Thomas Paul Matthews,
Yen Nhi Truong Vu,
Jason Su
Abstract:
This work reveals undiscovered challenges in the performance and generalizability of deep learning models. We (1) identify spurious shortcuts and evaluation issues that can inflate performance and (2) propose training and analysis methods to address them.
We trained an AI model to classify cancer on a retrospective dataset of 120,112 US exams (3,467 cancers) acquired from 2008 to 2017 and 16,693…
▽ More
This work reveals undiscovered challenges in the performance and generalizability of deep learning models. We (1) identify spurious shortcuts and evaluation issues that can inflate performance and (2) propose training and analysis methods to address them.
We trained an AI model to classify cancer on a retrospective dataset of 120,112 US exams (3,467 cancers) acquired from 2008 to 2017 and 16,693 UK exams (5,655 cancers) acquired from 2011 to 2015.
We evaluated on a screening mammography test set of 11,593 US exams (102 cancers; 7,594 women; age 57.1 \pm 11.0) and 1,880 UK exams (590 cancers; 1,745 women; age 63.3 \pm 7.2). A model trained on images of only view markers (no breast) achieved a 0.691 AUC. The original model trained on both datasets achieved a 0.945 AUC on the combined US+UK dataset but paradoxically only 0.838 and 0.892 on the US and UK datasets, respectively. Sampling cancers equally from both datasets during training mitigated this shortcut. A similar AUC paradox (0.903) occurred when evaluating diagnostic exams vs screening exams (0.862 vs 0.861, respectively). Removing diagnostic exams during training alleviated this bias. Finally, the model did not exhibit the AUC paradox over scanner models but still exhibited a bias toward Selenia Dimension (SD) over Hologic Selenia (HS) exams. Analysis showed that this AUC paradox occurred when a dataset attribute had values with a higher cancer prevalence (dataset bias) and the model consequently assigned a higher probability to these attribute values (model bias). Stratification and balancing cancer prevalence can mitigate shortcuts during evaluation.
Dataset and model bias can introduce shortcuts and the AUC paradox, potentially pervasive issues within the healthcare AI space. Our methods can verify and mitigate shortcuts while providing a clear understanding of performance.
△ Less
Submitted 28 March, 2023;
originally announced March 2023.
-
Deep is a Luxury We Don't Have
Authors:
Ahmed Taha,
Yen Nhi Truong Vu,
Brent Mombourquette,
Thomas Paul Matthews,
Jason Su,
Sadanand Singh
Abstract:
Medical images come in high resolutions. A high resolution is vital for finding malignant tissues at an early stage. Yet, this resolution presents a challenge in terms of modeling long range dependencies. Shallow transformers eliminate this problem, but they suffer from quadratic complexity. In this paper, we tackle this complexity by leveraging a linear self-attention approximation. Through this…
▽ More
Medical images come in high resolutions. A high resolution is vital for finding malignant tissues at an early stage. Yet, this resolution presents a challenge in terms of modeling long range dependencies. Shallow transformers eliminate this problem, but they suffer from quadratic complexity. In this paper, we tackle this complexity by leveraging a linear self-attention approximation. Through this approximation, we propose an efficient vision model called HCT that stands for High resolution Convolutional Transformer. HCT brings transformers' merits to high resolution images at a significantly lower cost. We evaluate HCT using a high resolution mammography dataset. HCT is significantly superior to its CNN counterpart. Furthermore, we demonstrate HCT's fitness for medical images by evaluating its effective receptive field.Code available at https://bit.ly/3ykBhhf
△ Less
Submitted 11 August, 2022;
originally announced August 2022.
-
A Multi-site Study of a Breast Density Deep Learning Model for Full-field Digital Mammography Images and Synthetic Mammography Images
Authors:
Thomas P. Matthews,
Sadanand Singh,
Brent Mombourquette,
Jason Su,
Meet P. Shah,
Stefano Pedemonte,
Aaron Long,
David Maffit,
Jenny Gurney,
Rodrigo Morales Hoil,
Nikita Ghare,
Douglas Smith,
Stephen M. Moore,
Susan C. Marks,
Richard L. Wahl
Abstract:
Purpose: To develop a Breast Imaging Reporting and Data System (BI-RADS) breast density deep learning (DL) model in a multi-site setting for synthetic two-dimensional mammography (SM) images derived from digital breast tomosynthesis exams using full-field digital mammography (FFDM) images and limited SM data.
Materials and Methods: A DL model was trained to predict BI-RADS breast density using F…
▽ More
Purpose: To develop a Breast Imaging Reporting and Data System (BI-RADS) breast density deep learning (DL) model in a multi-site setting for synthetic two-dimensional mammography (SM) images derived from digital breast tomosynthesis exams using full-field digital mammography (FFDM) images and limited SM data.
Materials and Methods: A DL model was trained to predict BI-RADS breast density using FFDM images acquired from 2008 to 2017 (Site 1: 57492 patients, 187627 exams, 750752 images) for this retrospective study. The FFDM model was evaluated using SM datasets from two institutions (Site 1: 3842 patients, 3866 exams, 14472 images, acquired from 2016 to 2017; Site 2: 7557 patients, 16283 exams, 63973 images, 2015 to 2019). Each of the three datasets were then split into training, validation, and test datasets. Adaptation methods were investigated to improve performance on the SM datasets and the effect of dataset size on each adaptation method is considered. Statistical significance was assessed using confidence intervals (CI), estimated by bootstrapping.
Results: Without adaptation, the model demonstrated substantial agreement with the original reporting radiologists for all three datasets (Site 1 FFDM: linearly-weighted $κ_w$ = 0.75 [95% CI: 0.74, 0.76]; Site 1 SM: $κ_w$ = 0.71 [95% CI: 0.64, 0.78]; Site 2 SM: $κ_w$ = 0.72 [95% CI: 0.70, 0.75]). With adaptation, performance improved for Site 2 (Site 1: $κ_w$ = 0.72 [95% CI: 0.66, 0.79], 0.71 vs 0.72, P = .80; Site 2: $κ_w$ = 0.79 [95% CI: 0.76, 0.81], 0.72 vs 0.79, P $<$ .001) using only 500 SM images from that site.
Conclusion: A BI-RADS breast density DL model demonstrated strong performance on FFDM and SM images from two institutions without training on SM images and improved using few SM images.
△ Less
Submitted 2 October, 2020; v1 submitted 23 January, 2020;
originally announced January 2020.
-
A Hypersensitive Breast Cancer Detector
Authors:
Stefano Pedemonte,
Brent Mombourquette,
Alexis Goh,
Trevor Tsue,
Aaron Long,
Sadanand Singh,
Thomas Paul Matthews,
Meet Shah,
Jason Su
Abstract:
Early detection of breast cancer through screening mammography yields a 20-35% increase in survival rate; however, there are not enough radiologists to serve the growing population of women seeking screening mammography. Although commercial computer aided detection (CADe) software has been available to radiologists for decades, it has failed to improve the interpretation of full-field digital mamm…
▽ More
Early detection of breast cancer through screening mammography yields a 20-35% increase in survival rate; however, there are not enough radiologists to serve the growing population of women seeking screening mammography. Although commercial computer aided detection (CADe) software has been available to radiologists for decades, it has failed to improve the interpretation of full-field digital mammography (FFDM) images due to its low sensitivity over the spectrum of findings. In this work, we leverage a large set of FFDM images with loose bounding boxes of mammographically significant findings to train a deep learning detector with extreme sensitivity. Building upon work from the Hourglass architecture, we train a model that produces segmentation-like images with high spatial resolution, with the aim of producing 2D Gaussian blobs centered on ground-truth boxes. We replace the pixel-wise $L_2$ norm with a weak-supervision loss designed to achieve high sensitivity, asymmetrically penalizing false positives and false negatives while softening the noise of the loose bounding boxes by permitting a tolerance in misaligned predictions. The resulting system achieves a sensitivity for malignant findings of 0.99 with only 4.8 false positive markers per image. When utilized in a CADe system, this model could enable a novel workflow where radiologists can focus their attention with trust on only the locations proposed by the model, expediting the interpretation process and bringing attention to potential findings that could otherwise have been missed. Due to its nearly perfect sensitivity, the proposed detector can also be used as a high-performance proposal generator in two-stage detection systems.
△ Less
Submitted 23 January, 2020;
originally announced January 2020.
-
Adaptation of a deep learning malignancy model from full-field digital mammography to digital breast tomosynthesis
Authors:
Sadanand Singh,
Thomas Paul Matthews,
Meet Shah,
Brent Mombourquette,
Trevor Tsue,
Aaron Long,
Ranya Almohsen,
Stefano Pedemonte,
Jason Su
Abstract:
Mammography-based screening has helped reduce the breast cancer mortality rate, but has also been associated with potential harms due to low specificity, leading to unnecessary exams or procedures, and low sensitivity. Digital breast tomosynthesis (DBT) improves on conventional mammography by increasing both sensitivity and specificity and is becoming common in clinical settings. However, deep lea…
▽ More
Mammography-based screening has helped reduce the breast cancer mortality rate, but has also been associated with potential harms due to low specificity, leading to unnecessary exams or procedures, and low sensitivity. Digital breast tomosynthesis (DBT) improves on conventional mammography by increasing both sensitivity and specificity and is becoming common in clinical settings. However, deep learning (DL) models have been developed mainly on conventional 2D full-field digital mammography (FFDM) or scanned film images. Due to a lack of large annotated DBT datasets, it is difficult to train a model on DBT from scratch. In this work, we present methods to generalize a model trained on FFDM images to DBT images. In particular, we use average histogram matching (HM) and DL fine-tuning methods to generalize a FFDM model to the 2D maximum intensity projection (MIP) of DBT images. In the proposed approach, the differences between the FFDM and DBT domains are reduced via HM and then the base model, which was trained on abundant FFDM images, is fine-tuned. When evaluating on image patches extracted around identified findings, we are able to achieve similar areas under the receiver operating characteristic curve (ROC AUC) of $\sim 0.9$ for FFDM and $\sim 0.85$ for MIP images, as compared to a ROC AUC of $\sim 0.75$ when tested directly on MIP images.
△ Less
Submitted 23 January, 2020;
originally announced January 2020.
-
Deep Learning-Guided Image Reconstruction from Incomplete Data
Authors:
Brendan Kelly,
Thomas P. Matthews,
Mark A. Anastasio
Abstract:
An approach to incorporate deep learning within an iterative image reconstruction framework to reconstruct images from severely incomplete measurement data is presented. Specifically, we utilize a convolutional neural network (CNN) as a quasi-projection operator within a least squares minimization procedure. The CNN is trained to encode high level information about the class of images being imaged…
▽ More
An approach to incorporate deep learning within an iterative image reconstruction framework to reconstruct images from severely incomplete measurement data is presented. Specifically, we utilize a convolutional neural network (CNN) as a quasi-projection operator within a least squares minimization procedure. The CNN is trained to encode high level information about the class of images being imaged; this information is utilized to mitigate artifacts in intermediate images produced by use of an iterative method. The structure of the method was inspired by the proximal gradient descent method, where the proximal operator is replaced by a deep CNN and the gradient descent step is generalized by use of a linear reconstruction operator. It is demonstrated that this approach improves image quality for several cases of limited-view image reconstruction and that using a CNN in an iterative method increases performance compared to conventional image reconstruction approaches. We test our method on several limited-view image reconstruction problems. Qualitative and quantitative results demonstrate state-of-the-art performance.
△ Less
Submitted 2 September, 2017;
originally announced September 2017.