-
The Group Robustness is in the Details: Revisiting Finetuning under Spurious Correlations
Authors:
Tyler LaBonte,
John C. Hill,
Xinchen Zhang,
Vidya Muthukumar,
Abhishek Kumar
Abstract:
Modern machine learning models are prone to over-reliance on spurious correlations, which can often lead to poor performance on minority groups. In this paper, we identify surprising and nuanced behavior of finetuned models on worst-group accuracy via comprehensive experiments on four well-established benchmarks across vision and language tasks. We first show that the commonly used class-balancing…
▽ More
Modern machine learning models are prone to over-reliance on spurious correlations, which can often lead to poor performance on minority groups. In this paper, we identify surprising and nuanced behavior of finetuned models on worst-group accuracy via comprehensive experiments on four well-established benchmarks across vision and language tasks. We first show that the commonly used class-balancing techniques of mini-batch upsampling and loss upweighting can induce a decrease in worst-group accuracy (WGA) with training epochs, leading to performance no better than without class-balancing. While in some scenarios, removing data to create a class-balanced subset is more effective, we show this depends on group structure and propose a mixture method which can outperform both techniques. Next, we show that scaling pretrained models is generally beneficial for worst-group accuracy, but only in conjuction with appropriate class-balancing. Finally, we identify spectral imbalance in finetuning features as a potential source of group disparities -- minority group covariance matrices incur a larger spectral norm than majority groups once conditioned on the classes. Our results show more nuanced interactions of modern finetuned models with group robustness than was previously known. Our code is available at https://github.com/tmlabonte/revisiting-finetuning.
△ Less
Submitted 18 July, 2024;
originally announced July 2024.
-
Guiding the Last Centimeter: Novel Anatomy-Aware Probe Servoing for Standardized Imaging Plane Navigation in Robotic Lung Ultrasound
Authors:
Xihan Ma,
Mingjie Zeng,
Jeffrey C. Hill,
Beatrice Hoffmann,
Ziming Zhang,
Haichong K. Zhang
Abstract:
Navigating the ultrasound (US) probe to the standardized imaging plane (SIP) for image acquisition is a critical but operator-dependent task in conventional freehand diagnostic US. Robotic US systems (RUSS) offer the potential to enhance imaging consistency by leveraging real-time US image feedback to optimize the probe pose, thereby reducing reliance on operator expertise. However, determining th…
▽ More
Navigating the ultrasound (US) probe to the standardized imaging plane (SIP) for image acquisition is a critical but operator-dependent task in conventional freehand diagnostic US. Robotic US systems (RUSS) offer the potential to enhance imaging consistency by leveraging real-time US image feedback to optimize the probe pose, thereby reducing reliance on operator expertise. However, determining the proper approach to extracting generalizable features from the US images for probe pose adjustment remain challenging. In this work, we propose a SIP navigation framework for RUSS, exemplified in the context of robotic lung ultrasound (LUS). This framework facilitates automatic probe adjustment when in proximity to the SIP. This is achieved by explicitly extracting multiple anatomical features presented in real-time LUS images and performing non-patient-specific template matching to generate probe motion towards the SIP using image-based visual servoing (IBVS). This framework is further integrated with the active-sensing end-effector (A-SEE), a customized robot end-effector that leverages patient external body geometry to maintain optimal probe alignment with the contact surface, thus preserving US signal quality throughout the navigation. The proposed approach ensures procedural interpretability and inter-patient adaptability. Validation is conducted through anatomy-mimicking phantom and in-vivo evaluations involving five human subjects. The results show the framework's high navigation precision with the probe correctly located at the SIP for all cases, exhibiting positioning error of under 2 mm in translation and under 2 degree in rotation. These results demonstrate the navigation process's capability to accomondate anatomical variations among patients.
△ Less
Submitted 17 June, 2024;
originally announced June 2024.
-
The SZ flux-mass ($Y$-$M$) relation at low halo masses: improvements with symbolic regression and strong constraints on baryonic feedback
Authors:
Digvijay Wadekar,
Leander Thiele,
J. Colin Hill,
Shivam Pandey,
Francisco Villaescusa-Navarro,
David N. Spergel,
Miles Cranmer,
Daisuke Nagai,
Daniel Anglés-Alcázar,
Shirley Ho,
Lars Hernquist
Abstract:
Feedback from active galactic nuclei (AGN) and supernovae can affect measurements of integrated SZ flux of halos ($Y_\mathrm{SZ}$) from CMB surveys, and cause its relation with the halo mass ($Y_\mathrm{SZ}-M$) to deviate from the self-similar power-law prediction of the virial theorem. We perform a comprehensive study of such deviations using CAMELS, a suite of hydrodynamic simulations with exten…
▽ More
Feedback from active galactic nuclei (AGN) and supernovae can affect measurements of integrated SZ flux of halos ($Y_\mathrm{SZ}$) from CMB surveys, and cause its relation with the halo mass ($Y_\mathrm{SZ}-M$) to deviate from the self-similar power-law prediction of the virial theorem. We perform a comprehensive study of such deviations using CAMELS, a suite of hydrodynamic simulations with extensive variations in feedback prescriptions. We use a combination of two machine learning tools (random forest and symbolic regression) to search for analogues of the $Y-M$ relation which are more robust to feedback processes for low masses ($M\lesssim 10^{14}\, h^{-1} \, M_\odot$); we find that simply replacing $Y\rightarrow Y(1+M_*/M_\mathrm{gas})$ in the relation makes it remarkably self-similar. This could serve as a robust multiwavelength mass proxy for low-mass clusters and galaxy groups. Our methodology can also be generally useful to improve the domain of validity of other astrophysical scaling relations.
We also forecast that measurements of the $Y-M$ relation could provide percent-level constraints on certain combinations of feedback parameters and/or rule out a major part of the parameter space of supernova and AGN feedback models used in current state-of-the-art hydrodynamic simulations. Our results can be useful for using upcoming SZ surveys (e.g., SO, CMB-S4) and galaxy surveys (e.g., DESI and Rubin) to constrain the nature of baryonic feedback. Finally, we find that the an alternative relation, $Y-M_*$, provides complementary information on feedback than $Y-M$
△ Less
Submitted 28 April, 2023; v1 submitted 5 September, 2022;
originally announced September 2022.
-
Augmenting astrophysical scaling relations with machine learning: application to reducing the Sunyaev-Zeldovich flux-mass scatter
Authors:
Digvijay Wadekar,
Leander Thiele,
Francisco Villaescusa-Navarro,
J. Colin Hill,
Miles Cranmer,
David N. Spergel,
Nicholas Battaglia,
Daniel Anglés-Alcázar,
Lars Hernquist,
Shirley Ho
Abstract:
Complex astrophysical systems often exhibit low-scatter relations between observable properties (e.g., luminosity, velocity dispersion, oscillation period). These scaling relations illuminate the underlying physics, and can provide observational tools for estimating masses and distances. Machine learning can provide a fast and systematic way to search for new scaling relations (or for simple exten…
▽ More
Complex astrophysical systems often exhibit low-scatter relations between observable properties (e.g., luminosity, velocity dispersion, oscillation period). These scaling relations illuminate the underlying physics, and can provide observational tools for estimating masses and distances. Machine learning can provide a fast and systematic way to search for new scaling relations (or for simple extensions to existing relations) in abstract high-dimensional parameter spaces. We use a machine learning tool called symbolic regression (SR), which models patterns in a dataset in the form of analytic equations. We focus on the Sunyaev-Zeldovich flux$-$cluster mass relation ($Y_\mathrm{SZ}-M$), the scatter in which affects inference of cosmological parameters from cluster abundance data. Using SR on the data from the IllustrisTNG hydrodynamical simulation, we find a new proxy for cluster mass which combines $Y_\mathrm{SZ}$ and concentration of ionized gas ($c_\mathrm{gas}$): $M \propto Y_\mathrm{conc}^{3/5} \equiv Y_\mathrm{SZ}^{3/5} (1-A\, c_\mathrm{gas})$. $Y_\mathrm{conc}$ reduces the scatter in the predicted $M$ by $\sim 20-30$\% for large clusters ($M\gtrsim 10^{14}\, h^{-1} \, M_\odot$), as compared to using just $Y_\mathrm{SZ}$. We show that the dependence on $c_\mathrm{gas}$ is linked to cores of clusters exhibiting larger scatter than their outskirts. Finally, we test $Y_\mathrm{conc}$ on clusters from CAMELS simulations and show that $Y_\mathrm{conc}$ is robust against variations in cosmology, subgrid physics, and cosmic variance. Our results and methodology can be useful for accurate multiwavelength cluster mass estimation from upcoming CMB and X-ray surveys like ACT, SO, eROSITA and CMB-S4.
△ Less
Submitted 17 March, 2023; v1 submitted 4 January, 2022;
originally announced January 2022.
-
Tele-operative Robotic Lung Ultrasound Scanning Platform for Triage of COVID-19 Patients
Authors:
Ryosuke Tsumura,
John W. Hardin,
Keshav Bimbraw,
Olushola S. Odusanya,
Yihao Zheng,
Jeffrey C. Hill,
Beatrice Hoffmann,
Winston Soboyejo,
Haichong K. Zhang
Abstract:
Novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has become a pandemic of epic proportions and a global response to prepare health systems worldwide is of utmost importance. In addition to its cost-effectiveness in a resources-limited setting, lung ultrasound (LUS) has emerged as a rapid noninvasive imaging tool for the diagnosis of COVID-19 infected patients. Concerns surroundin…
▽ More
Novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has become a pandemic of epic proportions and a global response to prepare health systems worldwide is of utmost importance. In addition to its cost-effectiveness in a resources-limited setting, lung ultrasound (LUS) has emerged as a rapid noninvasive imaging tool for the diagnosis of COVID-19 infected patients. Concerns surrounding LUS include the disparity of infected patients and healthcare providers, relatively small number of physicians and sonographers capable of performing LUS, and most importantly, the requirement for substantial physical contact between the patient and operator, increasing the risk of transmission. Mitigation of the spread of the virus is of paramount importance. A 2-dimensional (2D) tele-operative robotic platform capable of performing LUS in for COVID-19 infected patients may be of significant benefit. The authors address the aforementioned issues surrounding the use of LUS in the application of COVID- 19 infected patients. In addition, first time application, feasibility and safety were validated in three healthy subjects, along with 2D image optimization and comparison for overall accuracy. Preliminary results demonstrate that the proposed platform allows for successful acquisition and application of LUS in humans.
△ Less
Submitted 11 November, 2020; v1 submitted 23 October, 2020;
originally announced October 2020.
-
Computer Vision-Based Health Monitoring of Mecklenburg Bridge Using 3D Digital Image Correlation
Authors:
Mehrdad S. Dizaji,
Devin K. Harris,
Bernie Kassner,
Jeffrey C. Hill
Abstract:
A collaborative investigation between the University of Virginia (UVA) and the Virginia Transportation Research Council was performed on the Mecklenburg Bridge (I-85 over Route 1 in Mecklenburg County). The research team aided the Virginia Department of Transportation - Richmond District in the characterization of the bridge behavior of one of the bridge beams that had been repaired due to a previ…
▽ More
A collaborative investigation between the University of Virginia (UVA) and the Virginia Transportation Research Council was performed on the Mecklenburg Bridge (I-85 over Route 1 in Mecklenburg County). The research team aided the Virginia Department of Transportation - Richmond District in the characterization of the bridge behavior of one of the bridge beams that had been repaired due to a previous web buckling and crippling failure. The investigation focused on collecting full-field three-dimensional digital image correlation (3D-DIC) deformation measurements during the dropping sequence (removal of jacking to support beam on bearing/pier). Additionally, measurements were taken of the section prior to and after dropping using a handheld laser scanner to assess the potential of lateral deformation or out-of-plane buckling. Results from the study demonstrated that buckling of the tested beam did not occur, but did provided a series of approaches that can be used to evaluate the effectiveness of repaired steel beam ends. Specifically, the results provided an approach that could estimate the dead load distribution through back-calculation.
△ Less
Submitted 24 April, 2020;
originally announced May 2020.
-
MADNESS: A Multiresolution, Adaptive Numerical Environment for Scientific Simulation
Authors:
Robert J. Harrison,
Gregory Beylkin,
Florian A. Bischoff,
Justus A. Calvin,
George I. Fann,
Jacob Fosso-Tande,
Diego Galindo,
Jeff R. Hammond,
Rebecca Hartman-Baker,
Judith C. Hill,
Jun Jia,
Jakob S. Kottmann,
M-J. Yvonne Ou,
Laura E. Ratcliff,
Matthew G. Reuter,
Adam C. Richie-Halford,
Nichols A. Romero,
Hideo Sekino,
William A. Shelton,
Bryan E. Sundahl,
W. Scott Thornton,
Edward F. Valeev,
Álvaro Vázquez-Mayagoitia,
Nicholas Vence,
Yukina Yokoi
Abstract:
MADNESS (multiresolution adaptive numerical environment for scientific simulation) is a high-level software environment for solving integral and differential equations in many dimensions that uses adaptive and fast harmonic analysis methods with guaranteed precision based on multiresolution analysis and separated representations. Underpinning the numerical capabilities is a powerful petascale para…
▽ More
MADNESS (multiresolution adaptive numerical environment for scientific simulation) is a high-level software environment for solving integral and differential equations in many dimensions that uses adaptive and fast harmonic analysis methods with guaranteed precision based on multiresolution analysis and separated representations. Underpinning the numerical capabilities is a powerful petascale parallel programming environment that aims to increase both programmer productivity and code scalability. This paper describes the features and capabilities of MADNESS and briefly discusses some current applications in chemistry and several areas of physics.
△ Less
Submitted 5 July, 2015;
originally announced July 2015.
