-
Augmenting medical image classifiers with synthetic data from latent diffusion models
Authors:
Luke W. Sagers,
James A. Diao,
Luke Melas-Kyriazi,
Matthew Groh,
Pranav Rajpurkar,
Adewole S. Adamson,
Veronica Rotemberg,
Roxana Daneshjou,
Arjun K. Manrai
Abstract:
While hundreds of artificial intelligence (AI) algorithms are now approved or cleared by the US Food and Drugs Administration (FDA), many studies have shown inconsistent generalization or latent bias, particularly for underrepresented populations. Some have proposed that generative AI could reduce the need for real data, but its utility in model development remains unclear. Skin disease serves as…
▽ More
While hundreds of artificial intelligence (AI) algorithms are now approved or cleared by the US Food and Drugs Administration (FDA), many studies have shown inconsistent generalization or latent bias, particularly for underrepresented populations. Some have proposed that generative AI could reduce the need for real data, but its utility in model development remains unclear. Skin disease serves as a useful case study in synthetic image generation due to the diversity of disease appearance, particularly across the protected attribute of skin tone. Here we show that latent diffusion models can scalably generate images of skin disease and that augmenting model training with these data improves performance in data-limited settings. These performance gains saturate at synthetic-to-real image ratios above 10:1 and are substantially smaller than the gains obtained from adding real images. As part of our analysis, we generate and analyze a new dataset of 458,920 synthetic images produced using several generation strategies. Our results suggest that synthetic data could serve as a force-multiplier for model development, but the collection of diverse real-world data remains the most important step to improve medical AI algorithms.
△ Less
Submitted 23 August, 2023;
originally announced August 2023.
-
BenchMD: A Benchmark for Unified Learning on Medical Images and Sensors
Authors:
Kathryn Wantlin,
Chenwei Wu,
Shih-Cheng Huang,
Oishi Banerjee,
Farah Dadabhoy,
Veeral Vipin Mehta,
Ryan Wonhee Han,
Fang Cao,
Raja R. Narayan,
Errol Colak,
Adewole Adamson,
Laura Heacock,
Geoffrey H. Tison,
Alex Tamkin,
Pranav Rajpurkar
Abstract:
Medical data poses a daunting challenge for AI algorithms: it exists in many different modalities, experiences frequent distribution shifts, and suffers from a scarcity of examples and labels. Recent advances, including transformers and self-supervised learning, promise a more universal approach that can be applied flexibly across these diverse conditions. To measure and drive progress in this dir…
▽ More
Medical data poses a daunting challenge for AI algorithms: it exists in many different modalities, experiences frequent distribution shifts, and suffers from a scarcity of examples and labels. Recent advances, including transformers and self-supervised learning, promise a more universal approach that can be applied flexibly across these diverse conditions. To measure and drive progress in this direction, we present BenchMD: a benchmark that tests how well unified, modality-agnostic methods, including architectures and training techniques (e.g. self-supervised learning, ImageNet pretraining),perform on a diverse array of clinically-relevant medical tasks. BenchMD combines 19 publicly available datasets for 7 medical modalities, including 1D sensor data, 2D images, and 3D volumetric scans. Our benchmark reflects real-world data constraints by evaluating methods across a range of dataset sizes, including challenging few-shot settings that incentivize the use of pretraining. Finally, we evaluate performance on out-of-distribution data collected at different hospitals than the training data, representing naturally-occurring distribution shifts that frequently degrade the performance of medical AI models. Our baseline results demonstrate that no unified learning technique achieves strong performance across all modalities, leaving ample room for improvement on the benchmark. Code is released at https://github.com/rajpurkarlab/BenchMD.
△ Less
Submitted 26 June, 2023; v1 submitted 17 April, 2023;
originally announced April 2023.
-
Parallel remote state preparation for fully device-independent verifiable blind quantum computation
Authors:
Sean A. Adamson
Abstract:
We introduce a device-independent two-prover scheme in which a classical verifier is able to use a simple untrusted quantum measurement device (the client device) to securely delegate a quantum computation to an untrusted quantum server. To do this, we construct a parallel self-testing protocol to perform device-independent remote state preparation of $n$ qubits and compose this with the unconditi…
▽ More
We introduce a device-independent two-prover scheme in which a classical verifier is able to use a simple untrusted quantum measurement device (the client device) to securely delegate a quantum computation to an untrusted quantum server. To do this, we construct a parallel self-testing protocol to perform device-independent remote state preparation of $n$ qubits and compose this with the unconditionally secure universal verifiable blind quantum computation (VBQC) scheme of J. F. Fitzsimons and E. Kashefi [Phys. Rev. A 96, 012303 (2017)]. Our self-test achieves a multitude of desirable properties for the application we consider, giving rise to practical and fully device-independent VBQC. It certifies parallel measurements of all cardinal and intercardinal directions in the $XY$-plane as well as the computational basis, uses few input questions (of size logarithmic in $n$ for the client and a constant number communicated to the server), and requires only single-qubit measurements to be performed by the client device.
△ Less
Submitted 11 December, 2022;
originally announced December 2022.
-
Improving dermatology classifiers across populations using images generated by large diffusion models
Authors:
Luke W. Sagers,
James A. Diao,
Matthew Groh,
Pranav Rajpurkar,
Adewole S. Adamson,
Arjun K. Manrai
Abstract:
Dermatological classification algorithms developed without sufficiently diverse training data may generalize poorly across populations. While intentional data collection and annotation offer the best means for improving representation, new computational approaches for generating training data may also aid in mitigating the effects of sampling bias. In this paper, we show that DALL$\cdot$E 2, a lar…
▽ More
Dermatological classification algorithms developed without sufficiently diverse training data may generalize poorly across populations. While intentional data collection and annotation offer the best means for improving representation, new computational approaches for generating training data may also aid in mitigating the effects of sampling bias. In this paper, we show that DALL$\cdot$E 2, a large-scale text-to-image diffusion model, can produce photorealistic images of skin disease across skin types. Using the Fitzpatrick 17k dataset as a benchmark, we demonstrate that augmenting training data with DALL$\cdot$E 2-generated synthetic images improves classification of skin disease overall and especially for underrepresented groups.
△ Less
Submitted 23 November, 2022;
originally announced November 2022.
-
Practical parallel self-testing of Bell states via magic rectangles
Authors:
Sean A. Adamson,
Petros Wallden
Abstract:
Self-testing is a method to verify that one has a particular quantum state from purely classical statistics. For practical applications, such as device-independent delegated verifiable quantum computation, it is crucial that one self-tests multiple Bell states in parallel while keeping the quantum capabilities required of one side to a minimum. In this work, we use the $3 \times n$ magic rectangle…
▽ More
Self-testing is a method to verify that one has a particular quantum state from purely classical statistics. For practical applications, such as device-independent delegated verifiable quantum computation, it is crucial that one self-tests multiple Bell states in parallel while keeping the quantum capabilities required of one side to a minimum. In this work, we use the $3 \times n$ magic rectangle games (generalizations of the magic square game) to obtain a self-test for $n$ Bell states where the one side needs only to measure single-qubit Pauli observables. The protocol requires small input sizes [constant for Alice and $O(\log n)$ bits for Bob] and is robust with robustness $O(n^{5/2} \sqrt{\varepsilon})$, where $\varepsilon$ is the closeness of the ideal (perfect) correlations to those observed. To achieve the desired self-test, we introduce a one-side-local quantum strategy for the magic square game that wins with certainty, we generalize this strategy to the family of $3 \times n$ magic rectangle games, and we supplement these nonlocal games with extra check rounds (of single and pairs of observables).
△ Less
Submitted 31 March, 2022; v1 submitted 9 May, 2021;
originally announced May 2021.
-
Quantum Magic Rectangles: Characterization and Application to Certified Randomness Expansion
Authors:
Sean A. Adamson,
Petros Wallden
Abstract:
We study a generalization of the Mermin-Peres magic square game to arbitrary rectangular dimensions. After exhibiting some general properties, these rectangular games are fully characterized in terms of their optimal win probabilities for quantum strategies. We find that for $m \times n$ rectangular games of dimensions $m,n \geq 3$ there are quantum strategies that win with certainty, while for di…
▽ More
We study a generalization of the Mermin-Peres magic square game to arbitrary rectangular dimensions. After exhibiting some general properties, these rectangular games are fully characterized in terms of their optimal win probabilities for quantum strategies. We find that for $m \times n$ rectangular games of dimensions $m,n \geq 3$ there are quantum strategies that win with certainty, while for dimensions $1 \times n$ quantum strategies do not outperform classical strategies. The final case of dimensions $2 \times n$ is richer, and we give upper and lower bounds that both outperform the classical strategies. Finally, we apply our findings to quantum certified randomness expansion to find the noise tolerance and rates for all magic rectangle games. To do this, we use our previous results to obtain the winning probability of games with a distinguished input for which the devices give a deterministic outcome, and follow the analysis of C. A. Miller and Y. Shi [SIAM J. Comput. 46, 1304 (2017)].
△ Less
Submitted 9 December, 2020; v1 submitted 5 August, 2020;
originally announced August 2020.
