-
Contrastive Self-Supervised Learning for Spatio-Temporal Analysis of Lung Ultrasound Videos
Authors:
Li Chen,
Jonathan Rubin,
Jiahong Ouyang,
Naveen Balaraju,
Shubham Patil,
Courosh Mehanian,
Sourabh Kulhare,
Rachel Millin,
Kenton W Gregory,
Cynthia R Gregory,
Meihua Zhu,
David O Kessler,
Laurie Malia,
Almaz Dessie,
Joni Rabiner,
Di Coneybeare,
Bo Shopsin,
Andrew Hersh,
Cristian Madar,
Jeffrey Shupp,
Laura S Johnson,
Jacob Avila,
Kristin Dwyer,
Peter Weimersheimer,
Balasundar Raju
, et al. (2 additional authors not shown)
Abstract:
Self-supervised learning (SSL) methods have shown promise for medical imaging applications by learning meaningful visual representations, even when the amount of labeled data is limited. Here, we extend state-of-the-art contrastive learning SSL methods to 2D+time medical ultrasound video data by introducing a modified encoder and augmentation method capable of learning meaningful spatio-temporal r…
▽ More
Self-supervised learning (SSL) methods have shown promise for medical imaging applications by learning meaningful visual representations, even when the amount of labeled data is limited. Here, we extend state-of-the-art contrastive learning SSL methods to 2D+time medical ultrasound video data by introducing a modified encoder and augmentation method capable of learning meaningful spatio-temporal representations, without requiring constraints on the input data. We evaluate our method on the challenging clinical task of identifying lung consolidations (an important pathological feature) in ultrasound videos. Using a multi-center dataset of over 27k lung ultrasound videos acquired from over 500 patients, we show that our method can significantly improve performance on downstream localization and classification of lung consolidation. Comparisons against baseline models trained without SSL show that the proposed methods are particularly advantageous when the size of labeled training data is limited (e.g., as little as 5% of the training set).
△ Less
Submitted 14 October, 2023;
originally announced October 2023.
-
Weakly Semi-Supervised Detection in Lung Ultrasound Videos
Authors:
Jiahong Ouyang,
Li Chen,
Gary Y. Li,
Naveen Balaraju,
Shubham Patil,
Courosh Mehanian,
Sourabh Kulhare,
Rachel Millin,
Kenton W. Gregory,
Cynthia R. Gregory,
Meihua Zhu,
David O. Kessler,
Laurie Malia,
Almaz Dessie,
Joni Rabiner,
Di Coneybeare,
Bo Shopsin,
Andrew Hersh,
Cristian Madar,
Jeffrey Shupp,
Laura S. Johnson,
Jacob Avila,
Kristin Dwyer,
Peter Weimersheimer,
Balasundar Raju
, et al. (2 additional authors not shown)
Abstract:
Frame-by-frame annotation of bounding boxes by clinical experts is often required to train fully supervised object detection models on medical video data. We propose a method for improving object detection in medical videos through weak supervision from video-level labels. More concretely, we aggregate individual detection predictions into video-level predictions and extend a teacher-student train…
▽ More
Frame-by-frame annotation of bounding boxes by clinical experts is often required to train fully supervised object detection models on medical video data. We propose a method for improving object detection in medical videos through weak supervision from video-level labels. More concretely, we aggregate individual detection predictions into video-level predictions and extend a teacher-student training strategy to provide additional supervision via a video-level loss. We also introduce improvements to the underlying teacher-student framework, including methods to improve the quality of pseudo-labels based on weak supervision and adaptive schemes to optimize knowledge transfer between the student and teacher networks. We apply this approach to the clinically important task of detecting lung consolidations (seen in respiratory infections such as COVID-19 pneumonia) in medical ultrasound videos. Experiments reveal that our framework improves detection accuracy and robustness compared to baseline semi-supervised models, and improves efficiency in data and annotation usage.
△ Less
Submitted 7 August, 2023;
originally announced August 2023.
-
Dialogs Re-enacted Across Languages
Authors:
Nigel G. Ward,
Jonathan E. Avila,
Emilia Rivas,
Divette Marco
Abstract:
To support machine learning of cross-language prosodic mappings and other ways to improve speech-to-speech translation, we present a protocol for collecting closely matched pairs of utterances across languages, a description of the resulting data collection and its public release, and some observations and musings. This report is intended for: people using this corpus, people extending this corpus…
▽ More
To support machine learning of cross-language prosodic mappings and other ways to improve speech-to-speech translation, we present a protocol for collecting closely matched pairs of utterances across languages, a description of the resulting data collection and its public release, and some observations and musings. This report is intended for: people using this corpus, people extending this corpus, and people designing similar collections of bilingual dialog data.
△ Less
Submitted 12 July, 2023; v1 submitted 18 November, 2022;
originally announced November 2022.
-
Depth Evaluation for Metal Surface Defects by Eddy Current Testing using Deep Residual Convolutional Neural Networks
Authors:
Tian Meng,
Yang Tao,
Ziqi Chen,
Jorge R. Salas Avila,
Qiaoye Ran,
Yuchun Shao,
Ruochen Huang,
Yuedong Xie,
Qian Zhao,
Zhijie Zhang,
Hujun Yin,
Anthony J. Peyton,
Wuliang Yin
Abstract:
Eddy current testing (ECT) is an effective technique in the evaluation of the depth of metal surface defects. However, in practice, the evaluation primarily relies on the experience of an operator and is often carried out by manual inspection. In this paper, we address the challenges of automatic depth evaluation of metal surface defects by virtual of state-of-the-art deep learning (DL) techniques…
▽ More
Eddy current testing (ECT) is an effective technique in the evaluation of the depth of metal surface defects. However, in practice, the evaluation primarily relies on the experience of an operator and is often carried out by manual inspection. In this paper, we address the challenges of automatic depth evaluation of metal surface defects by virtual of state-of-the-art deep learning (DL) techniques. The main contributions are three-fold. Firstly, a highly-integrated portable ECT device is developed, which takes advantage of an advanced field programmable gate array (Zynq-7020 system on chip) and provides fast data acquisition and in-phase/quadrature demodulation. Secondly, a dataset, termed as MDDECT, is constructed using the ECT device by human operators and made openly available. It contains 48,000 scans from 18 defects of different depths and lift-offs. Thirdly, the depth evaluation problem is formulated as a time series classification problem, and various state-of-the-art 1-d residual convolutional neural networks are trained and evaluated on the MDDECT dataset. A 38-layer 1-d ResNeXt achieves an accuracy of 93.58% in discriminating the surface defects in a stainless steel sheet. The depths of the defects vary from 0.3 mm to 2.0 mm in a resolution of 0.1 mm. In addition, results show that the trained ResNeXt1D-38 model is immune to lift-off signals.
△ Less
Submitted 8 March, 2021;
originally announced April 2021.
-
A Novel Dual Modality Sensor With Sensitivities to Permittivity, Conductivity, and Permeability
Authors:
Jorge R. Salas Avila,
Kin Yau How,
Mingyang Lu,
Wuliang Yin
Abstract:
In this paper, an electromagnetic sensor which can operate simultaneously in capacitive and inductive modalities with sensitivities to permittivity, conductivity, and permeability is developed, and a novel measurement strategy is proposed accordingly. The sensor is composed of two planar spiral coils with a track width of 4 mm, which promotes its capacitive mode. The capacitive coupling is measure…
▽ More
In this paper, an electromagnetic sensor which can operate simultaneously in capacitive and inductive modalities with sensitivities to permittivity, conductivity, and permeability is developed, and a novel measurement strategy is proposed accordingly. The sensor is composed of two planar spiral coils with a track width of 4 mm, which promotes its capacitive mode. The capacitive coupling is measured in common mode, while the inductive coupling is measured in differential mode. In capacitive mode, the sensor is sensitive to changes in permittivity, i.e., the dielectric material distribution; while in inductive mode, it is sensitive to magnetically permeable material and electrically conductive material. Furthermore, it is demonstrated that the sensor can simultaneously measure dielectric and conductive materials. This novel sensing element has been designed and implemented. Experimental results verified its effectiveness in dual modality measurement.
△ Less
Submitted 28 May, 2019;
originally announced May 2019.
