Showing 1–50 of 436 results for author: Shin, Y

Generalized logistic model for $r$ largest order statistics, with hydrological application

Authors: Yire Shin, Jeong-Soo Park

Abstract: The effective use of available information in extreme value analysis is critical because extreme values are scarce. Thus, using the $r$ largest order statistics (rLOS) instead of the block maxima is encouraged. Based on the four-parameter kappa model for the rLOS (rK4D), we introduce a new distribution for the rLOS as a special case of the rK4D. That is the generalized logistic model for rLOS (rGL… ▽ More The effective use of available information in extreme value analysis is critical because extreme values are scarce. Thus, using the $r$ largest order statistics (rLOS) instead of the block maxima is encouraged. Based on the four-parameter kappa model for the rLOS (rK4D), we introduce a new distribution for the rLOS as a special case of the rK4D. That is the generalized logistic model for rLOS (rGLO). This distribution can be useful when the generalized extreme value model for rLOS is no longer efficient to capture the variability of extreme values. Moreover, the rGLO enriches a pool of candidate distributions to determine the best model to yield accurate and robust quantile estimates. We derive a joint probability density function, the marginal and conditional distribution functions of new model. The maximum likelihood estimation, delta method, profile likelihood, order selection by the entropy difference test, cross-validated likelihood criteria, and model averaging were considered for inferences. The usefulness and practical effectiveness of the rGLO are illustrated by the Monte Carlo simulation and an application to extreme streamflow data in Bevern Stream, UK. △ Less

Submitted 16 August, 2024; originally announced August 2024.

Journal ref: Stoch Environ Res Risk Assess 38 (2024) 1567-1581

Harmful Suicide Content Detection

Authors: Kyumin Park, Myung Jae Baik, YeongJun Hwang, Yen Shin, HoJae Lee, Ruda Lee, Sang Min Lee, Je Young Hannah Sun, Ah Rah Lee, Si Yeun Yoon, Dong-ho Lee, Jihyung Moon, JinYeong Bak, Kyunghyun Cho, Jong-Woo Paik, Sungjoon Park

Abstract: Harmful suicide content on the Internet is a significant risk factor inducing suicidal thoughts and behaviors among vulnerable populations. Despite global efforts, existing resources are insufficient, specifically in high-risk regions like the Republic of Korea. Current research mainly focuses on understanding negative effects of such content or suicide risk in individuals, rather than on automati… ▽ More Harmful suicide content on the Internet is a significant risk factor inducing suicidal thoughts and behaviors among vulnerable populations. Despite global efforts, existing resources are insufficient, specifically in high-risk regions like the Republic of Korea. Current research mainly focuses on understanding negative effects of such content or suicide risk in individuals, rather than on automatically detecting the harmfulness of content. To fill this gap, we introduce a harmful suicide content detection task for classifying online suicide content into five harmfulness levels. We develop a multi-modal benchmark and a task description document in collaboration with medical professionals, and leverage large language models (LLMs) to explore efficient methods for moderating such content. Our contributions include proposing a novel detection task, a multi-modal Korean benchmark with expert annotations, and suggesting strategies using LLMs to detect illegal and harmful content. Owing to the potential harm involved, we publicize our implementations and benchmark, incorporating an ethical verification process. △ Less

Submitted 2 June, 2024; originally announced July 2024.

Comments: 30 pages, 7 figures

A Multi-Messenger Search for Exotic Field Emission with a Global Magnetometer Network

Authors: Sami S. Khamis, Ibrahim A. Sulai, Paul Hamilton, S. Afach, B. C. Buchler, D. Budker, N. L. Figueroa, R. Folman, D. Gavilán-Martín, M. Givon, Z. D. Grujić, H. Guo, M. P. Hedges, D. F. Jackson Kimball, D. Kim, E. Klinger, T. Kornack, A. Kryemadhi, N. Kukowski, G. Lukasiewicz, H. Masia-Roig, M. Padniuk, C. A. Palm, S. Y. Park, X. Peng , et al. (16 additional authors not shown)

Abstract: We present an analysis method to search for exotic low-mass field (ELF) bursts generated during large energy astrophysical events such as supernovae, binary black hole or binary neutron star mergers, and fast radio bursts using the Global Network of Optical Magnetometers for Exotic physics searches (GNOME). In our model, the associated gravitational waves or electromagnetic signals herald the arri… ▽ More We present an analysis method to search for exotic low-mass field (ELF) bursts generated during large energy astrophysical events such as supernovae, binary black hole or binary neutron star mergers, and fast radio bursts using the Global Network of Optical Magnetometers for Exotic physics searches (GNOME). In our model, the associated gravitational waves or electromagnetic signals herald the arrival of the ELF burst that interacts via coupling to the spin of fermions in the magnetometers. This enables GNOME to serve as a tool for multi-messenger astronomy. The algorithm employs a model-agnostic excess-power method to identify network-wide candidate events to be subjected to a model-dependent generalized likelihood-ratio test to determine their statistical significance. We perform the first search with this technique on GNOME data coincident with the binary black hole merger S200311bg detected by LIGO/Virgo on the 11th of March 2020 and find no significant events. We place the first lab-based limits on combinations of ELF production and coupling parameters. △ Less

Submitted 18 July, 2024; originally announced July 2024.

A Study of Digital Appliances Accessibility for People with Visual Disabilities

Authors: Hyunjin An, Hyundoug Kim, Seungwoo Hong, Youngsun Shin

Abstract: This research aims to find where visually impaired users find appliances hard to use and suggest guideline to solve this issue. 181 visually impaired users have been surveyed, and 12 visually impaired users have been selected based on disability cause and classification. In a home-like environment, we had participants perform tasks which were sorted using Hierarchical task analysis on six major ho… ▽ More This research aims to find where visually impaired users find appliances hard to use and suggest guideline to solve this issue. 181 visually impaired users have been surveyed, and 12 visually impaired users have been selected based on disability cause and classification. In a home-like environment, we had participants perform tasks which were sorted using Hierarchical task analysis on six major home appliances. From this research we found out that home appliances sometimes only provide visual information which causes difficulty in sensory processing. Also, interfaces tactile/auditory feedbacks are the same making it hard for people to recognize which feature is processed. Blind users cannot see the provided information so they rely on long-term memory to use products. This research provides guideline for button, knob and remote control interface for visually impaired users. This information will be helpful for project planners, designers, and developers to create products which are accessible by visually impaired people. Some of the features will be applied to upcoming home appliance products. △ Less

Submitted 8 July, 2024; originally announced July 2024.

Comments: 10 pages, 3 figures

MSC Class: 68U35 ACM Class: D.2.2

Rethinking Image Skip Connections in StyleGAN2

Authors: Seung Park, Yong-Goo Shin

Abstract: Various models based on StyleGAN have gained significant traction in the field of image synthesis, attributed to their robust training stability and superior performances. Within the StyleGAN framework, the adoption of image skip connection is favored over the traditional residual connection. However, this preference is just based on empirical observations; there has not been any in-depth mathemat… ▽ More Various models based on StyleGAN have gained significant traction in the field of image synthesis, attributed to their robust training stability and superior performances. Within the StyleGAN framework, the adoption of image skip connection is favored over the traditional residual connection. However, this preference is just based on empirical observations; there has not been any in-depth mathematical analysis on it yet. To rectify this situation, this brief aims to elucidate the mathematical meaning of the image skip connection and introduce a groundbreaking methodology, termed the image squeeze connection, which significantly improves the quality of image synthesis. Specifically, we analyze the image skip connection technique to reveal its problem and introduce the proposed method which not only effectively boosts the GAN performance but also reduces the required number of network parameters. Extensive experiments on various datasets demonstrate that the proposed method consistently enhances the performance of state-of-the-art models based on StyleGAN. We believe that our findings represent a vital advancement in the field of image synthesis, suggesting a novel direction for future research and applications. △ Less

Submitted 7 July, 2024; originally announced July 2024.

Effective Heterogeneous Federated Learning via Efficient Hypernetwork-based Weight Generation

Authors: Yujin Shin, Kichang Lee, Sungmin Lee, You Rim Choi, Hyung-Sin Kim, JeongGil Ko

Abstract: While federated learning leverages distributed client resources, it faces challenges due to heterogeneous client capabilities. This necessitates allocating models suited to clients' resources and careful parameter aggregation to accommodate this heterogeneity. We propose HypeMeFed, a novel federated learning framework for supporting client heterogeneity by combining a multi-exit network architectu… ▽ More While federated learning leverages distributed client resources, it faces challenges due to heterogeneous client capabilities. This necessitates allocating models suited to clients' resources and careful parameter aggregation to accommodate this heterogeneity. We propose HypeMeFed, a novel federated learning framework for supporting client heterogeneity by combining a multi-exit network architecture with hypernetwork-based model weight generation. This approach aligns the feature spaces of heterogeneous model layers and resolves per-layer information disparity during weight aggregation. To practically realize HypeMeFed, we also propose a low-rank factorization approach to minimize computation and memory overhead associated with hypernetworks. Our evaluations on a real-world heterogeneous device testbed indicate that HypeMeFed enhances accuracy by 5.12% over FedAvg, reduces the hypernetwork memory requirements by 98.22%, and accelerates its operations by 1.86 times compared to a naive hypernetwork approach. These results demonstrate HypeMeFed's effectiveness in leveraging and engaging heterogeneous clients for federated learning. △ Less

Submitted 3 July, 2024; originally announced July 2024.

FIESTA: Fourier-Based Semantic Augmentation with Uncertainty Guidance for Enhanced Domain Generalizability in Medical Image Segmentation

Authors: Kwanseok Oh, Eunjin Jeon, Da-Woon Heo, Yooseung Shin, Heung-Il Suk

Abstract: Single-source domain generalization (SDG) in medical image segmentation (MIS) aims to generalize a model using data from only one source domain to segment data from an unseen target domain. Despite substantial advances in SDG with data augmentation, existing methods often fail to fully consider the details and uncertain areas prevalent in MIS, leading to mis-segmentation. This paper proposes a Fou… ▽ More Single-source domain generalization (SDG) in medical image segmentation (MIS) aims to generalize a model using data from only one source domain to segment data from an unseen target domain. Despite substantial advances in SDG with data augmentation, existing methods often fail to fully consider the details and uncertain areas prevalent in MIS, leading to mis-segmentation. This paper proposes a Fourier-based semantic augmentation method called FIESTA using uncertainty guidance to enhance the fundamental goals of MIS in an SDG context by manipulating the amplitude and phase components in the frequency domain. The proposed Fourier augmentative transformer addresses semantic amplitude modulation based on meaningful angular points to induce pertinent variations and harnesses the phase spectrum to ensure structural coherence. Moreover, FIESTA employs epistemic uncertainty to fine-tune the augmentation process, improving the ability of the model to adapt to diverse augmented data and concentrate on areas with higher ambiguity. Extensive experiments across three cross-domain scenarios demonstrate that FIESTA surpasses recent state-of-the-art SDG approaches in segmentation performance and significantly contributes to boosting the applicability of the model in medical imaging modalities. △ Less

Submitted 20 June, 2024; originally announced June 2024.

Comments: 40 pages, 7 figures, 5 tables

On the Feasibility of Fidelity$^-$ for Graph Pruning

Authors: Yong-Min Shin, Won-Yong Shin

Abstract: As one of popular quantitative metrics to assess the quality of explanation of graph neural networks (GNNs), fidelity measures the output difference after removing unimportant parts of the input graph. Fidelity has been widely used due to its straightforward interpretation that the underlying model should produce similar predictions when features deemed unimportant from the explanation are removed… ▽ More As one of popular quantitative metrics to assess the quality of explanation of graph neural networks (GNNs), fidelity measures the output difference after removing unimportant parts of the input graph. Fidelity has been widely used due to its straightforward interpretation that the underlying model should produce similar predictions when features deemed unimportant from the explanation are removed. This raises a natural question: "Does fidelity induce a global (soft) mask for graph pruning?" To solve this, we aim to explore the potential of the fidelity measure to be used for graph pruning, eventually enhancing the GNN models for better efficiency. To this end, we propose Fidelity$^-$-inspired Pruning (FiP), an effective framework to construct global edge masks from local explanations. Our empirical observations using 7 edge attribution methods demonstrate that, surprisingly, general eXplainable AI methods outperform methods tailored to GNNs in terms of graph pruning performance. △ Less

Submitted 17 June, 2024; originally announced June 2024.

Comments: 6 pages, 3 figures, 2 tables; IJCAI Workshop on Explainable AI (XAI 2024) (to appear) (Please cite our workshop version.)

ONNXim: A Fast, Cycle-level Multi-core NPU Simulator

Authors: Hyungkyu Ham, Wonhyuk Yang, Yunseon Shin, Okkyun Woo, Guseul Heo, Sangyeop Lee, Jongse Park, Gwangsun Kim

Abstract: As DNNs are widely adopted in various application domains while demanding increasingly higher compute and memory requirements, designing efficient and performant NPUs (Neural Processing Units) is becoming more important. However, existing architectural NPU simulators lack support for high-speed simulation, multi-core modeling, multi-tenant scenarios, detailed DRAM/NoC modeling, and/or different de… ▽ More As DNNs are widely adopted in various application domains while demanding increasingly higher compute and memory requirements, designing efficient and performant NPUs (Neural Processing Units) is becoming more important. However, existing architectural NPU simulators lack support for high-speed simulation, multi-core modeling, multi-tenant scenarios, detailed DRAM/NoC modeling, and/or different deep learning frameworks. To address these limitations, this work proposes ONNXim, a fast cycle-level simulator for multi-core NPUs in DNN serving systems. It takes DNN models represented in the ONNX graph format generated from various deep learning frameworks for ease of simulation. In addition, based on the observation that typical NPU cores process tensor tiles from on-chip scratchpad memory with deterministic compute latency, we forgo a detailed modeling for the computation while still preserving simulation accuracy. ONNXim also preserves dependencies between compute and tile DMAs. Meanwhile, the DRAM and NoC are modeled in cycle-level to properly model contention among multiple cores that can execute different DNN models for multi-tenancy. Consequently, ONNXim is significantly faster than existing simulators (e.g., by up to 384x over Accel-sim) and enables various case studies, such as multi-tenant NPUs, that were previously impractical due to slow speed and/or lack of functionalities. ONNXim is publicly available at https://github.com/PSAL-POSTECH/ONNXim. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Revisiting Attention Weights as Interpretations of Message-Passing Neural Networks

Authors: Yong-Min Shin, Siqing Li, Xin Cao, Won-Yong Shin

Abstract: The self-attention mechanism has been adopted in several widely-used message-passing neural networks (MPNNs) (e.g., GATs), which adaptively controls the amount of information that flows along the edges of the underlying graph. This usage of attention has made such models a baseline for studies on explainable AI (XAI) since interpretations via attention have been popularized in various domains (e.g… ▽ More The self-attention mechanism has been adopted in several widely-used message-passing neural networks (MPNNs) (e.g., GATs), which adaptively controls the amount of information that flows along the edges of the underlying graph. This usage of attention has made such models a baseline for studies on explainable AI (XAI) since interpretations via attention have been popularized in various domains (e.g., natural language processing and computer vision). However, existing studies often use naive calculations to derive attribution scores from attention, and do not take the precise and careful calculation of edge attribution into consideration. In our study, we aim to fill the gap between the widespread usage of attention-enabled MPNNs and their potential in largely under-explored explainability, a topic that has been actively investigated in other areas. To this end, as the first attempt, we formalize the problem of edge attribution from attention weights in GNNs. Then, we propose GATT, an edge attribution calculation method built upon the computation tree. Through comprehensive experiments, we demonstrate the effectiveness of our proposed method when evaluating attributions from GATs. Conversely, we empirically validate that simply averaging attention weights over graph attention layers is insufficient to interpret the GAT model's behavior. Code is publicly available at https://github.com/jordan7186/GAtt/tree/main. △ Less

Submitted 6 June, 2024; originally announced June 2024.

Comments: 11 pages, 3 figures, 5 tables

Monotone Equilibrium Design for Matching Markets with Signaling

Authors: Seungjin Han, Alex Sam, Youngki Shin

Abstract: We study monotone equilibrium design by a planner who chooses an interval of reactions that receivers take before senders and receivers move in matching markets with signaling. Given the convex efficiency frontier over sender surplus and receiver surplus generated by the interval delegation, the optimal reaction interval crucially depends on the ripple effect of its lower bound and on the trade-of… ▽ More We study monotone equilibrium design by a planner who chooses an interval of reactions that receivers take before senders and receivers move in matching markets with signaling. Given the convex efficiency frontier over sender surplus and receiver surplus generated by the interval delegation, the optimal reaction interval crucially depends on the ripple effect of its lower bound and on the trade-off between matching inefficiency and signaling cost savings in the top pooling region generated by its upper bound. Our analysis generates cohesive market design results that integrate the literature on minimum wage, firm size distribution, and relative risk aversion. △ Less

Submitted 23 July, 2024; v1 submitted 3 June, 2024; originally announced June 2024.

Comments: 54 pages, 14 figures

Bayesian Estimation of Hierarchical Linear Models from Incomplete Data: Cluster-Level Interaction Effects and Small Sample Sizes

Authors: Dongho Shin, Yongyun Shin, Nao Hagiwara

Abstract: We consider Bayesian estimation of a hierarchical linear model (HLM) from small sample sizes where 37 patient-physician encounters are repeatedly measured at four time points. The continuous response $Y$ and continuous covariates $C$ are partially observed and assumed missing at random. With $C$ having linear effects, the HLM may be efficiently estimated by available methods. When $C$ includes clu… ▽ More We consider Bayesian estimation of a hierarchical linear model (HLM) from small sample sizes where 37 patient-physician encounters are repeatedly measured at four time points. The continuous response $Y$ and continuous covariates $C$ are partially observed and assumed missing at random. With $C$ having linear effects, the HLM may be efficiently estimated by available methods. When $C$ includes cluster-level covariates having interactive or other nonlinear effects given small sample sizes, however, maximum likelihood estimation is suboptimal, and existing Gibbs samplers are based on a Bayesian joint distribution compatible with the HLM, but impute missing values of $C$ by a Metropolis algorithm via a proposal density having a constant variance while the target conditional distribution has a nonconstant variance. Therefore, the samplers are not guaranteed to be compatible with the joint distribution and, thus, not guaranteed to always produce unbiased estimation of the HLM. We introduce a compatible Gibbs sampler that imputes parameters and missing values directly from the exact conditional distributions. We analyze repeated measurements from patient-physician encounters by our sampler, and compare our estimators with those of existing methods by simulation. △ Less

Submitted 31 May, 2024; originally announced May 2024.

Double-sided van der Waals epitaxy of topological insulators across an atomically thin membrane

Authors: Joon Young Park, Young Jae Shin, Jeacheol Shin, Jehyun Kim, Janghyun Jo, Hyobin Yoo, Danial Haei, Chohee Hyun, Jiyoung Yun, Robert M. Huber, Arijit Gupta, Kenji Watanabe, Takashi Taniguchi, Wan Kyu Park, Hyeon Suk Shin, Miyoung Kim, Dohun Kim, Gyu-Chul Yi, Philip Kim

Abstract: Atomically thin van der Waals (vdW) films provide a novel material platform for epitaxial growth of quantum heterostructures. However, unlike the remote epitaxial growth of three-dimensional bulk crystals, the growth of two-dimensional (2D) material heterostructures across atomic layers has been limited due to the weak vdW interaction. Here, we report the double-sided epitaxy of vdW layered materi… ▽ More Atomically thin van der Waals (vdW) films provide a novel material platform for epitaxial growth of quantum heterostructures. However, unlike the remote epitaxial growth of three-dimensional bulk crystals, the growth of two-dimensional (2D) material heterostructures across atomic layers has been limited due to the weak vdW interaction. Here, we report the double-sided epitaxy of vdW layered materials through atomic membranes. We grow vdW topological insulators (TIs) Sb$_2$Te$_3$ and Bi$_2$Se$_3$ by molecular beam epitaxy on both surfaces of atomically thin graphene or hBN, which serve as suspended 2D vdW "$\textit{substrate}$" layers. Both homo- and hetero- double-sided vdW TI tunnel junctions are fabricated, with the atomically thin hBN acting as a crystal-momentum-conserving tunnelling barrier with abrupt and epitaxial interface. By performing field-angle dependent magneto-tunnelling spectroscopy on these devices, we reveal the energy-momentum-spin resonant tunnelling of massless Dirac electrons between helical Landau levels developed in the topological surface states at the interface. △ Less

Submitted 30 May, 2024; originally announced May 2024.

Comments: 24 pages, 4 main figures, 7 extended data figures

Topological Floquet engineering of a three-band optical lattice with dual-mode resonant driving

Authors: Dalmin Bae, Junyoung Park, Myeonghyeon Kim, Haneul Kwak, Junhwan Kwon, Yong-il Shin

Abstract: We present a Floquet framework for controlling topological features of a one-dimensional optical lattice system with dual-mode resonant driving, in which both the amplitude and phase of the lattice potential are modulated simultaneously. We investigate a three-band model consisting of the three lowest orbitals and elucidate the formation of a cross-linked two-leg ladder through an indirect interba… ▽ More We present a Floquet framework for controlling topological features of a one-dimensional optical lattice system with dual-mode resonant driving, in which both the amplitude and phase of the lattice potential are modulated simultaneously. We investigate a three-band model consisting of the three lowest orbitals and elucidate the formation of a cross-linked two-leg ladder through an indirect interband coupling via an off-resonant band. We numerically demonstrate the emergence of topologically nontrivial bands within the driven system, and a topological charge pumping phenomenon with cyclic parameter changes in the dual-mode resonant driving. Finally, we show that the band topology in the driven three-band system is protected by parity-time reversal symmetry. △ Less

Submitted 16 May, 2024; originally announced May 2024.

Comments: 10 pages, 6 figures

Data-Efficient Molecular Generation with Hierarchical Textual Inversion

Authors: Seojin Kim, Jaehyun Nam, Sihyun Yu, Younghoon Shin, Jinwoo Shin

Abstract: Developing an effective molecular generation framework even with a limited number of molecules is often important for its practical deployment, e.g., drug discovery, since acquiring task-related molecular data requires expensive and time-consuming experimental costs. To tackle this issue, we introduce Hierarchical textual Inversion for Molecular generation (HI-Mol), a novel data-efficient molecula… ▽ More Developing an effective molecular generation framework even with a limited number of molecules is often important for its practical deployment, e.g., drug discovery, since acquiring task-related molecular data requires expensive and time-consuming experimental costs. To tackle this issue, we introduce Hierarchical textual Inversion for Molecular generation (HI-Mol), a novel data-efficient molecular generation method. HI-Mol is inspired by the importance of hierarchical information, e.g., both coarse- and fine-grained features, in understanding the molecule distribution. We propose to use multi-level embeddings to reflect such hierarchical features based on the adoption of the recent textual inversion technique in the visual domain, which achieves data-efficient image generation. Compared to the conventional textual inversion method in the image domain using a single-level token embedding, our multi-level token embeddings allow the model to effectively learn the underlying low-shot molecule distribution. We then generate molecules based on the interpolation of the multi-level token embeddings. Extensive experiments demonstrate the superiority of HI-Mol with notable data-efficiency. For instance, on QM9, HI-Mol outperforms the prior state-of-the-art method with 50x less training data. We also show the effectiveness of molecules generated by HI-Mol in low-shot molecular property prediction. △ Less

Submitted 16 July, 2024; v1 submitted 5 May, 2024; originally announced May 2024.

Comments: ICML 2024

Low-overhead General-purpose Near-Data Processing in CXL Memory Expanders

Authors: Hyungkyu Ham, Jeongmin Hong, Geonwoo Park, Yunseon Shin, Okkyun Woo, Wonhyuk Yang, Jinhoon Bae, Eunhyeok Park, Hyojin Sung, Euicheol Lim, Gwangsun Kim

Abstract: Emerging Compute Express Link (CXL) enables cost-efficient memory expansion beyond the local DRAM of processors. While its CXL.mem protocol provides minimal latency overhead through an optimized protocol stack, frequent CXL memory accesses can result in significant slowdowns for memory-bound applications whether they are latency-sensitive or bandwidth-intensive. The near-data processing (NDP) in t… ▽ More Emerging Compute Express Link (CXL) enables cost-efficient memory expansion beyond the local DRAM of processors. While its CXL.mem protocol provides minimal latency overhead through an optimized protocol stack, frequent CXL memory accesses can result in significant slowdowns for memory-bound applications whether they are latency-sensitive or bandwidth-intensive. The near-data processing (NDP) in the CXL controller promises to overcome such limitations of passive CXL memory. However, prior work on NDP in CXL memory proposes application-specific units that are not suitable for practical CXL memory-based systems that should support various applications. On the other hand, existing CPU or GPU cores are not cost-effective for NDP because they are not optimized for memory-bound applications. In addition, the communication between the host processor and CXL controller for NDP offloading should achieve low latency, but existing CXL$.$io/PCIe-based mechanisms incur $μ$s-scale latency and are not suitable for fine-grained NDP. To achieve high-performance NDP end-to-end, we propose a low-overhead general-purpose NDP architecture for CXL memory referred to as Memory-Mapped NDP (M$^2$NDP), which comprises memory-mapped functions (M$^2$func) and memory-mapped $μ$threading (M$^2μ$thr). M$^2$func is a CXL.mem-compatible low-overhead communication mechanism between the host processor and NDP controller in CXL memory. M$^2μ$thr enables low-cost, general-purpose NDP unit design by introducing lightweight $μ$threads that support highly concurrent execution of kernels with minimal resource wastage. Combining them, M$^2$NDP achieves significant speedups for various workloads by up to 128x (14.5x overall) and reduces energy by up to 87.9% (80.3% overall) compared to baseline CPU/GPU hosts with passive CXL memory. △ Less

Submitted 19 July, 2024; v1 submitted 30 April, 2024; originally announced April 2024.

Turbo-CF: Matrix Decomposition-Free Graph Filtering for Fast Recommendation

Authors: Jin-Duk Park, Yong-Min Shin, Won-Yong Shin

Abstract: A series of graph filtering (GF)-based collaborative filtering (CF) showcases state-of-the-art performance on the recommendation accuracy by using a low-pass filter (LPF) without a training process. However, conventional GF-based CF approaches mostly perform matrix decomposition on the item-item similarity graph to realize the ideal LPF, which results in a non-trivial computational cost and thus m… ▽ More A series of graph filtering (GF)-based collaborative filtering (CF) showcases state-of-the-art performance on the recommendation accuracy by using a low-pass filter (LPF) without a training process. However, conventional GF-based CF approaches mostly perform matrix decomposition on the item-item similarity graph to realize the ideal LPF, which results in a non-trivial computational cost and thus makes them less practical in scenarios where rapid recommendations are essential. In this paper, we propose Turbo-CF, a GF-based CF method that is both training-free and matrix decomposition-free. Turbo-CF employs a polynomial graph filter to circumvent the issue of expensive matrix decompositions, enabling us to make full use of modern computer hardware components (i.e., GPU). Specifically, Turbo-CF first constructs an item-item similarity graph whose edge weights are effectively regulated. Then, our own polynomial LPFs are designed to retain only low-frequency signals without explicit matrix decompositions. We demonstrate that Turbo-CF is extremely fast yet accurate, achieving a runtime of less than 1 second on real-world benchmark datasets while achieving recommendation accuracies comparable to best competitors. △ Less

Submitted 22 April, 2024; originally announced April 2024.

Comments: 5 pages, 4 figures, 4 tables; 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2024) (to appear) (Please cite our conference version.)

Structural properties of amorphous Na$_3$OCl electrolyte by first-principles and machine learning molecular dynamics

Authors: T. -L. Pham, M. Guerboub, S. D. Wansi Wendj, A. Bouzid, C. Tugène, M. Boero, C. Massobrio, Y. -H. Shin, G. Ori

Abstract: Solid-state electrolytes mark a significant leap forward in the field of electrochemical energy storage, offering improved safety and efficiency compared to conventional liquid electrolytes. Among these, antiperovskite electrolytes, particularly those based on Li and Na, have emerged as promising candidates due to their superior ionic conductivity and straightforward synthesis processes. This stud… ▽ More Solid-state electrolytes mark a significant leap forward in the field of electrochemical energy storage, offering improved safety and efficiency compared to conventional liquid electrolytes. Among these, antiperovskite electrolytes, particularly those based on Li and Na, have emerged as promising candidates due to their superior ionic conductivity and straightforward synthesis processes. This study focuses on the amorphous phase of antiperovskite Na$_3$OCl, assessing its structural properties through a combination of first-principles molecular dynamics (FPMD) and machine learning interatomic potential (MLIP) simulations. Our comprehensive analysis spans models ranging from 135 to 3645 atoms, allowing for a detailed examination of X-ray and neutron structure factors, total and partial pair correlation functions, coordination numbers, and structural unit distributions. We demonstrate the minimal, albeit partially present, size effects on these structural features and validate the accuracy of the MLIP model in reproducing the intricate details of the amorphous Na$_3$OCl structure described at the FPMD level. △ Less

Submitted 17 April, 2024; originally announced April 2024.

Comments: 6 Figures and 3 Tables

Flattening a trapped atomic gas using a programmable optical potential in a feedback loop

Authors: Sol Kim, Kyuhwan Lee, Jongmin Kim, Y. Shin

Abstract: We present a method for producing a flat, large-area Fermi gas of $^6$Li with a uniform area density. The method uses a programmable optical potential within a feedback loop to flatten the in-plane trapping potential for atoms. The optical potential is generated using a laser beam, whose intensity profile is adjusted by a spatial light modulator and optimized through measurements of the density di… ▽ More We present a method for producing a flat, large-area Fermi gas of $^6$Li with a uniform area density. The method uses a programmable optical potential within a feedback loop to flatten the in-plane trapping potential for atoms. The optical potential is generated using a laser beam, whose intensity profile is adjusted by a spatial light modulator and optimized through measurements of the density distribution of the sample. The resulting planar sample exhibits a uniform area density within a region of about 480 $μ$m in diameter and the standard deviation of the trap bottom potential is estimated to be $\approx k_B \times$ 6.1 nK, which is less than 20$\%$ of the transverse confinement energy. We discuss a dimensional crossover toward 2D regime by reducing the number of atoms in the planar trap, including the effect of the spatial variation of the transverse trapping frequency in the large-area sample. △ Less

Submitted 30 March, 2024; originally announced April 2024.

Comments: 8 pages, 6 figures

Dual-sided transparent display

Authors: Suman Halder, Yunho Shin, Yidan Peng, Long Wang, Liye Duan, Paul Schmalenberg, Guangkui Qin, Yuxi Gao, Ercan M. Dede, Deng-Ke Yang, Sean P. Rodrigues

Abstract: In the past decade, display technology has been reimagined to meet the needs of the virtual world. By mapping information onto a scene through a transparent display, users can simultaneously visualize both the real world and layers of virtual elements. However, advances in augmented reality (AR) technology have primarily focused on wearable gear or personal devices. Here we present a single displa… ▽ More In the past decade, display technology has been reimagined to meet the needs of the virtual world. By mapping information onto a scene through a transparent display, users can simultaneously visualize both the real world and layers of virtual elements. However, advances in augmented reality (AR) technology have primarily focused on wearable gear or personal devices. Here we present a single display capable of delivering visual information to observers positioned on either side of the transparent device. This dual-sided display system employs a polymer stabilized liquid crystal waveguide technology to achieve a transparency window of 65% while offering active-matrix control. An early-stage prototype exhibits full-color information via time-sequential processing of a red-green-blue (RGB) light-emitting diode (LED) strip. The dual-sided display provides a perspective on transparent mediums as display devices for human-centric and service-related experiences that can support both enhanced bi-directional user interactions and new media platforms. △ Less

Submitted 7 March, 2024; originally announced March 2024.

Exploring Dielectric Properties in Models of Amorphous Boron Nitride

Authors: Thomas Galvani, Ali K. Hamze, Laura Caputo, Onurcan Kaya, Simon Dubois, Luigi Colombo, Viet-Hung Nguyen, Yongwoo Shin, Hyeon-Jin Shin, Jean-Christophe Charlier, Stephan Roche

Abstract: We report a theoretical study of dielectric properties of models of amorphous Boron Nitride, using interatomic potentials generated by machine learning. We first perform first-principles simulations on small (about $100$ atoms in the periodic cell) sample sizes to explore the emergence of mid-gap states and its correlation with structural features. Next, by using a simplified tight-binding electro… ▽ More We report a theoretical study of dielectric properties of models of amorphous Boron Nitride, using interatomic potentials generated by machine learning. We first perform first-principles simulations on small (about $100$ atoms in the periodic cell) sample sizes to explore the emergence of mid-gap states and its correlation with structural features. Next, by using a simplified tight-binding electronic model, we analyse the dielectric functions for complex three dimensional models (containing about $10.000$ atoms) embedding varying concentrations of ${\rm sp^{1}, sp^{2}}$ and ${\rm sp^3}$ bonds between B and N atoms. Within the limits of these methodologies, the resulting value of the zero-frequency dielectric constant is shown to be influenced by the population density of such mid-gap states and their localization characteristics. We observe nontrivial correlations between the structure-induced electronic fluctuations and the resulting dielectric constant values. Our findings are however just a first step in the quest of accessing fully accurate dielectric properties of as-grown amorphous BN of relevance for interconnect technologies and beyond. △ Less

Submitted 18 March, 2024; originally announced March 2024.

Comments: 27 pages, 10 figures

A Comprehensive Review of Latent Space Dynamics Identification Algorithms for Intrusive and Non-Intrusive Reduced-Order-Modeling

Authors: Christophe Bonneville, Xiaolong He, April Tran, Jun Sur Park, William Fries, Daniel A. Messenger, Siu Wun Cheung, Yeonjong Shin, David M. Bortz, Debojyoti Ghosh, Jiun-Shyan Chen, Jonathan Belof, Youngsoo Choi

Abstract: Numerical solvers of partial differential equations (PDEs) have been widely employed for simulating physical systems. However, the computational cost remains a major bottleneck in various scientific and engineering applications, which has motivated the development of reduced-order models (ROMs). Recently, machine-learning-based ROMs have gained significant popularity and are promising for addressi… ▽ More Numerical solvers of partial differential equations (PDEs) have been widely employed for simulating physical systems. However, the computational cost remains a major bottleneck in various scientific and engineering applications, which has motivated the development of reduced-order models (ROMs). Recently, machine-learning-based ROMs have gained significant popularity and are promising for addressing some limitations of traditional ROM methods, especially for advection dominated systems. In this chapter, we focus on a particular framework known as Latent Space Dynamics Identification (LaSDI), which transforms the high-fidelity data, governed by a PDE, to simpler and low-dimensional latent-space data, governed by ordinary differential equations (ODEs). These ODEs can be learned and subsequently interpolated to make ROM predictions. Each building block of LaSDI can be easily modulated depending on the application, which makes the LaSDI framework highly flexible. In particular, we present strategies to enforce the laws of thermodynamics into LaSDI models (tLaSDI), enhance robustness in the presence of noise through the weak form (WLaSDI), select high-fidelity training data efficiently through active learning (gLaSDI, GPLaSDI), and quantify the ROM prediction uncertainty through Gaussian processes (GPLaSDI). We demonstrate the performance of different LaSDI approaches on Burgers equation, a non-linear heat conduction problem, and a plasma physics problem, showing that LaSDI algorithms can achieve relative errors of less than a few percent and up to thousands of times speed-ups. △ Less

Submitted 15 March, 2024; originally announced March 2024.

tLaSDI: Thermodynamics-informed latent space dynamics identification

Authors: Jun Sur Richard Park, Siu Wun Cheung, Youngsoo Choi, Yeonjong Shin

Abstract: We propose a latent space dynamics identification method, namely tLaSDI, that embeds the first and second principles of thermodynamics. The latent variables are learned through an autoencoder as a nonlinear dimension reduction model. The latent dynamics are constructed by a neural network-based model that precisely preserves certain structures for the thermodynamic laws through the GENERIC formali… ▽ More We propose a latent space dynamics identification method, namely tLaSDI, that embeds the first and second principles of thermodynamics. The latent variables are learned through an autoencoder as a nonlinear dimension reduction model. The latent dynamics are constructed by a neural network-based model that precisely preserves certain structures for the thermodynamic laws through the GENERIC formalism. An abstract error estimate is established, which provides a new loss formulation involving the Jacobian computation of autoencoder. The autoencoder and the latent dynamics are simultaneously trained to minimize the new loss. Computational examples demonstrate the effectiveness of tLaSDI, which exhibits robust generalization ability, even in extrapolation. In addition, an intriguing correlation is empirically observed between a quantity from tLaSDI in the latent space and the behaviors of the full-state solution. △ Less

Submitted 21 March, 2024; v1 submitted 9 March, 2024; originally announced March 2024.

Comments: 32 pages, 8 figures

Chaos-assisted Turbulence in Spinor Bose-Einstein Condensates

Authors: Jongmin Kim, Jongheum Jung, Junghoon Lee, Deokhwa Hong, Yong-il Shin

Abstract: We present a turbulence-sustaining mechanism in a spinor Bose-Einstein condensate, which is based on the chaotic nature of internal spin dynamics. Magnetic driving induces a complete chaotic evolution of the local spin state, thereby continuously randomizing the spin texture of the condensate to maintain the turbulent state. We experimentally demonstrate the onset of turbulence in the driven conde… ▽ More We present a turbulence-sustaining mechanism in a spinor Bose-Einstein condensate, which is based on the chaotic nature of internal spin dynamics. Magnetic driving induces a complete chaotic evolution of the local spin state, thereby continuously randomizing the spin texture of the condensate to maintain the turbulent state. We experimentally demonstrate the onset of turbulence in the driven condensate as the driving frequency changes and show that it is consistent with the regular-to-chaotic transition of the local spin dynamics. This chaos-assisted turbulence establishes the spin-driven spinor condensate as an intriguing platform for exploring quantum chaos and related superfluid turbulence phenomena. △ Less

Submitted 1 March, 2024; originally announced March 2024.

Comments: 10 pages, 8 figures

Extensive search for axion dark matter over 1\,GHz with CAPP's Main Axion eXperiment

Authors: Saebyeok Ahn, JinMyeong Kim, Boris I. Ivanov, Ohjoon Kwon, HeeSu Byun, Arjan F. van Loo, SeongTae Par, Junu Jeong, Soohyung Lee, Jinsu Kim, Çağlar Kutlu, Andrew K. Yi, Yasunobu Nakamura, Seonjeong Oh, Danho Ahn, SungJae Bae, Hyoungsoon Choi, Jihoon Choi, Yonuk Chong, Woohyun Chung, Violeta Gkika, Jihn E. Kim, Younggeun Kim, Byeong Rok Ko, Lino Miceli , et al. (11 additional authors not shown)

Abstract: We report an extensive high-sensitivity search for axion dark matter above 1\,GHz at the Center for Axion and Precision Physics Research (CAPP). The cavity resonant search, exploiting the coupling between axions and photons, explored the frequency (mass) range of 1.025\,GHz (4.24\,$μ$eV) to 1.185\,GHz (4.91\,$μ$eV). We have introduced a number of innovations in this field, demonstrating the practi… ▽ More We report an extensive high-sensitivity search for axion dark matter above 1\,GHz at the Center for Axion and Precision Physics Research (CAPP). The cavity resonant search, exploiting the coupling between axions and photons, explored the frequency (mass) range of 1.025\,GHz (4.24\,$μ$eV) to 1.185\,GHz (4.91\,$μ$eV). We have introduced a number of innovations in this field, demonstrating the practical approach of optimizing all the relevant parameters of axion haloscopes, extending presently available technology. The CAPP 12\,T magnet with an aperture of 320\,mm made of Nb$_3$Sn and NbTi superconductors surrounding a 37-liter ultralight-weight copper cavity is expected to convert DFSZ axions into approximately $10^2$ microwave photons per second. A powerful dilution refrigerator, capable of keeping the core system below 40\,mK, combined with quantum-noise limited readout electronics, achieved a total system noise of about 200\,mK or below, which corresponds to a background of roughly $4\times 10^3$ photons per second within the axion bandwidth. The combination of all those improvements provides unprecedented search performance, imposing the most stringent exclusion limits on axion--photon coupling in this frequency range to date. These results also suggest an experimental capability suitable for highly-sensitive searches for axion dark matter above 1\,GHz. △ Less

Submitted 20 February, 2024; originally announced February 2024.

Comments: A detailed axion dark matter article with 27 pages, 22 figures

H2O-SDF: Two-phase Learning for 3D Indoor Reconstruction using Object Surface Fields

Authors: Minyoung Park, Mirae Do, YeonJae Shin, Jaeseok Yoo, Jongkwang Hong, Joongrock Kim, Chul Lee

Abstract: Advanced techniques using Neural Radiance Fields (NeRF), Signed Distance Fields (SDF), and Occupancy Fields have recently emerged as solutions for 3D indoor scene reconstruction. We introduce a novel two-phase learning approach, H2O-SDF, that discriminates between object and non-object regions within indoor environments. This method achieves a nuanced balance, carefully preserving the geometric in… ▽ More Advanced techniques using Neural Radiance Fields (NeRF), Signed Distance Fields (SDF), and Occupancy Fields have recently emerged as solutions for 3D indoor scene reconstruction. We introduce a novel two-phase learning approach, H2O-SDF, that discriminates between object and non-object regions within indoor environments. This method achieves a nuanced balance, carefully preserving the geometric integrity of room layouts while also capturing intricate surface details of specific objects. A cornerstone of our two-phase learning framework is the introduction of the Object Surface Field (OSF), a novel concept designed to mitigate the persistent vanishing gradient problem that has previously hindered the capture of high-frequency details in other methods. Our proposed approach is validated through several experiments that include ablation studies. △ Less

Submitted 8 March, 2024; v1 submitted 12 February, 2024; originally announced February 2024.

Probing early phase coarsening in a rapidly quenched Bose gas using off-resonant matter-wave interferometry

Authors: Tenzin Rabga, Yangheon Lee, Yong-il Shin

Abstract: We experimentally investigate the evolution of spatial phase correlations in a rapidly quenched inhomogeneous Bose gas of rubidium using off-resonant matter-wave interferometry. We measure the phase coherence length $\ell$ of the sample and directly probe its increase during the early stage of condensate growth before vortices are formed. Once the vortices are formed stably in the quenched condens… ▽ More We experimentally investigate the evolution of spatial phase correlations in a rapidly quenched inhomogeneous Bose gas of rubidium using off-resonant matter-wave interferometry. We measure the phase coherence length $\ell$ of the sample and directly probe its increase during the early stage of condensate growth before vortices are formed. Once the vortices are formed stably in the quenched condensate, the measured value of $\ell$ is shown to be linearly proportional to the mean distance between the vortex. These results confirm the presence of phase coarsening prior to vortex formation, which is crucial for a quantitative understanding of the resultant defect density in samples undergoing critical phase transitions. △ Less

Submitted 25 August, 2024; v1 submitted 6 February, 2024; originally announced February 2024.

Comments: 8 pages, 4 figures

Towards Generating Executable Metamorphic Relations Using Large Language Models

Authors: Seung Yeob Shin, Fabrizio Pastore, Domenico Bianculli, Alexandra Baicoianu

Abstract: Metamorphic testing (MT) has proven to be a successful solution to automating testing and addressing the oracle problem. However, it entails manually deriving metamorphic relations (MRs) and converting them into an executable form; these steps are time-consuming and may prevent the adoption of MT. In this paper, we propose an approach for automatically deriving executable MRs (EMRs) from requireme… ▽ More Metamorphic testing (MT) has proven to be a successful solution to automating testing and addressing the oracle problem. However, it entails manually deriving metamorphic relations (MRs) and converting them into an executable form; these steps are time-consuming and may prevent the adoption of MT. In this paper, we propose an approach for automatically deriving executable MRs (EMRs) from requirements using large language models (LLMs). Instead of merely asking the LLM to produce EMRs, our approach relies on a few-shot prompting strategy to instruct the LLM to perform activities in the MT process, by providing requirements and API specifications, as one would do with software engineers. To assess the feasibility of our approach, we conducted a questionnaire-based survey in collaboration with Siemens Industry Software, a worldwide leader in providing industry software and services, focusing on four of their software applications. Additionally, we evaluated the accuracy of the generated EMRs for a Web application. The outcomes of our study are highly promising, as they demonstrate the capability of our approach to generate MRs and EMRs that are both comprehensible and pertinent for testing purposes. △ Less

Submitted 7 June, 2024; v1 submitted 30 January, 2024; originally announced January 2024.

Comments: preprint - accepted for QUATIC 2024

Highest weight modules over Borcherds-Bozec superalgebras and their character formula

Authors: Zhaobing Fan, Jiaqi Huang, Seok-Jin Kang, Yong-Su Shin

Abstract: We present and prove the Weyl-Kac type character formula for the irreducible highest weight modules over Borcherds-Bozec superalgebras with dominant integral highest weights. We present and prove the Weyl-Kac type character formula for the irreducible highest weight modules over Borcherds-Bozec superalgebras with dominant integral highest weights. △ Less

Submitted 28 January, 2024; originally announced January 2024.

Log canonical thresholds of Burniat surfaces with $K^2 = 5$

Authors: Nguyen Bin, Jheng-Jie Chen, YongJoo Shin

Abstract: In the paper we compute the global log canonical thresholds of the secondary Burniat surfaces with $K^2 = 5$. Furthermore, we establish optimal lower bounds for the log canonical thresholds of members in pluricanonical sublinear systems of the secondary Burniat surfaces with $K^2 = 5$. In the paper we compute the global log canonical thresholds of the secondary Burniat surfaces with $K^2 = 5$. Furthermore, we establish optimal lower bounds for the log canonical thresholds of members in pluricanonical sublinear systems of the secondary Burniat surfaces with $K^2 = 5$. △ Less

Submitted 8 January, 2024; originally announced January 2024.

Comments: 25 pages, comments are welcome

Universal Time-Series Representation Learning: A Survey

Authors: Patara Trirat, Yooju Shin, Junhyeok Kang, Youngeun Nam, Jihye Na, Minyoung Bae, Joeun Kim, Byunghyun Kim, Jae-Gil Lee

Abstract: Time-series data exists in every corner of real-world systems and services, ranging from satellites in the sky to wearable devices on human bodies. Learning representations by extracting and inferring valuable information from these time series is crucial for understanding the complex dynamics of particular phenomena and enabling informed decisions. With the learned representations, we can perform… ▽ More Time-series data exists in every corner of real-world systems and services, ranging from satellites in the sky to wearable devices on human bodies. Learning representations by extracting and inferring valuable information from these time series is crucial for understanding the complex dynamics of particular phenomena and enabling informed decisions. With the learned representations, we can perform numerous downstream analyses more effectively. Among several approaches, deep learning has demonstrated remarkable performance in extracting hidden patterns and features from time-series data without manual feature engineering. This survey first presents a novel taxonomy based on three fundamental elements in designing state-of-the-art universal representation learning methods for time series. According to the proposed taxonomy, we comprehensively review existing studies and discuss their intuitions and insights into how these methods enhance the quality of learned representations. Finally, as a guideline for future studies, we summarize commonly used experimental setups and datasets and discuss several promising research directions. An up-to-date corresponding resource is available at https://github.com/itouchz/awesome-deep-time-series-representations. △ Less

Submitted 27 August, 2024; v1 submitted 8 January, 2024; originally announced January 2024.

Comments: 41 pages, 7 figures

Assessing AI Detectors in Identifying AI-Generated Code: Implications for Education

Authors: Wei Hung Pan, Ming Jie Chok, Jonathan Leong Shan Wong, Yung Xin Shin, Yeong Shian Poon, Zhou Yang, Chun Yong Chong, David Lo, Mei Kuan Lim

Abstract: Educators are increasingly concerned about the usage of Large Language Models (LLMs) such as ChatGPT in programming education, particularly regarding the potential exploitation of imperfections in Artificial Intelligence Generated Content (AIGC) Detectors for academic misconduct. In this paper, we present an empirical study where the LLM is examined for its attempts to bypass detection by AIGC Det… ▽ More Educators are increasingly concerned about the usage of Large Language Models (LLMs) such as ChatGPT in programming education, particularly regarding the potential exploitation of imperfections in Artificial Intelligence Generated Content (AIGC) Detectors for academic misconduct. In this paper, we present an empirical study where the LLM is examined for its attempts to bypass detection by AIGC Detectors. This is achieved by generating code in response to a given question using different variants. We collected a dataset comprising 5,069 samples, with each sample consisting of a textual description of a coding problem and its corresponding human-written Python solution codes. These samples were obtained from various sources, including 80 from Quescol, 3,264 from Kaggle, and 1,725 from LeetCode. From the dataset, we created 13 sets of code problem variant prompts, which were used to instruct ChatGPT to generate the outputs. Subsequently, we assessed the performance of five AIGC detectors. Our results demonstrate that existing AIGC Detectors perform poorly in distinguishing between human-written code and AI-generated code. △ Less

Submitted 8 January, 2024; originally announced January 2024.

Comments: 11 pages, paper accepted at 46th International Conference on Software Engineering, Software Engineering Education and Training Track (ICSE-SEET 2024)

Actuator-Constrained Reinforcement Learning for High-Speed Quadrupedal Locomotion

Authors: Young-Ha Shin, Tae-Gyu Song, Gwanghyeon Ji, Hae-Won Park

Abstract: This paper presents a method for achieving high-speed running of a quadruped robot by considering the actuator torque-speed operating region in reinforcement learning. The physical properties and constraints of the actuator are included in the training process to reduce state transitions that are infeasible in the real world due to motor torque-speed limitations. The gait reward is designed to dis… ▽ More This paper presents a method for achieving high-speed running of a quadruped robot by considering the actuator torque-speed operating region in reinforcement learning. The physical properties and constraints of the actuator are included in the training process to reduce state transitions that are infeasible in the real world due to motor torque-speed limitations. The gait reward is designed to distribute motor torque evenly across all legs, contributing to more balanced power usage and mitigating performance bottlenecks due to single-motor saturation. Additionally, we designed a lightweight foot to enhance the robot's agility. We observed that applying the motor operating region as a constraint helps the policy network avoid infeasible areas during sampling. With the trained policy, KAIST Hound, a 45 kg quadruped robot, can run up to 6.5 m/s, which is the fastest speed among electric motor-based quadruped robots. △ Less

Submitted 29 December, 2023; originally announced December 2023.

Continuous-time Autoencoders for Regular and Irregular Time Series Imputation

Authors: Hyowon Wi, Yehjin Shin, Noseong Park

Abstract: Time series imputation is one of the most fundamental tasks for time series. Real-world time series datasets are frequently incomplete (or irregular with missing observations), in which case imputation is strongly required. Many different time series imputation methods have been proposed. Recent self-attention-based methods show the state-of-the-art imputation performance. However, it has been ove… ▽ More Time series imputation is one of the most fundamental tasks for time series. Real-world time series datasets are frequently incomplete (or irregular with missing observations), in which case imputation is strongly required. Many different time series imputation methods have been proposed. Recent self-attention-based methods show the state-of-the-art imputation performance. However, it has been overlooked for a long time to design an imputation method based on continuous-time recurrent neural networks (RNNs), i.e., neural controlled differential equations (NCDEs). To this end, we redesign time series (variational) autoencoders based on NCDEs. Our method, called continuous-time autoencoder (CTA), encodes an input time series sample into a continuous hidden path (rather than a hidden vector) and decodes it to reconstruct and impute the input. In our experiments with 4 datasets and 19 baselines, our method shows the best imputation performance in almost all cases. △ Less

Submitted 24 June, 2024; v1 submitted 27 December, 2023; originally announced December 2023.

Comments: Published as a WSDM'24 full paper (oral presentation)

Informatics-based learning of oxygen vacancy ordering principles in oxygen-deficient perovskites

Authors: Yongjin Shin, Kenneth R. Poeppelmeier, James M. Rondinelli

Abstract: Ordered oxygen vacancies (OOVs) in perovskites can exhibit long-range order and may be used to direct materials properties through modifications in electronic structures and broken symmetries. Based on the various vacancy patterns observed in previously known compounds, we explore the ordering principles of OOVs in oxygen-deficient perovskite oxides with $AB\mathrm{O}_{2.5}$ stoichiometry to ident… ▽ More Ordered oxygen vacancies (OOVs) in perovskites can exhibit long-range order and may be used to direct materials properties through modifications in electronic structures and broken symmetries. Based on the various vacancy patterns observed in previously known compounds, we explore the ordering principles of OOVs in oxygen-deficient perovskite oxides with $AB\mathrm{O}_{2.5}$ stoichiometry to identify other OOV variants. We performed first-principles calculations to assess the OOV stability on a dataset of 50 OOV structures generated from our bespoke algorithm. The algorithm employs uniform planar vacancy patterns on (111) pseudocubic perovskite layers and the approach proves effective for generating stable OOV patterns with minimal computational loads. We find as expected that the major factors determining the stability of OOV structures include coordination preferences of transition metals and elastic penalties resulting from the assemblies of polyhedra. Cooperative rotational modes of polyhedra within OOV structures reduce elastic instabilities by optimizing the bond valence of $A$- and $B$-cations. This finding explains the observed formation of vacancy channels along low-index crystallographic directions in prototypical OOV phases. The identified ordering principles enable us to devise other stable vacancy patterns with longer periodicity for targeted property design in yet to be synthesized compounds. △ Less

Submitted 19 December, 2023; originally announced December 2023.

An Attentive Inductive Bias for Sequential Recommendation beyond the Self-Attention

Authors: Yehjin Shin, Jeongwhan Choi, Hyowon Wi, Noseong Park

Abstract: Sequential recommendation (SR) models based on Transformers have achieved remarkable successes. The self-attention mechanism of Transformers for computer vision and natural language processing suffers from the oversmoothing problem, i.e., hidden representations becoming similar to tokens. In the SR domain, we, for the first time, show that the same problem occurs. We present pioneering investigati… ▽ More Sequential recommendation (SR) models based on Transformers have achieved remarkable successes. The self-attention mechanism of Transformers for computer vision and natural language processing suffers from the oversmoothing problem, i.e., hidden representations becoming similar to tokens. In the SR domain, we, for the first time, show that the same problem occurs. We present pioneering investigations that reveal the low-pass filtering nature of self-attention in the SR, which causes oversmoothing. To this end, we propose a novel method called $\textbf{B}$eyond $\textbf{S}$elf-$\textbf{A}$ttention for Sequential $\textbf{Rec}$ommendation (BSARec), which leverages the Fourier transform to i) inject an inductive bias by considering fine-grained sequential patterns and ii) integrate low and high-frequency information to mitigate oversmoothing. Our discovery shows significant advancements in the SR domain and is expected to bridge the gap for existing Transformer-based SR models. We test our proposed approach through extensive experiments on 6 benchmark datasets. The experimental results demonstrate that our model outperforms 7 baseline methods in terms of recommendation performance. Our code is available at https://github.com/yehjin-shin/BSARec. △ Less

Submitted 17 February, 2024; v1 submitted 16 December, 2023; originally announced December 2023.

Comments: Accepted by AAAI 2024. Yehjin Shin and Jeongwhan Choi are co-first authors with equal contribution

Assessing the Usability of GutGPT: A Simulation Study of an AI Clinical Decision Support System for Gastrointestinal Bleeding Risk

Authors: Colleen Chan, Kisung You, Sunny Chung, Mauro Giuffrè, Theo Saarinen, Niroop Rajashekar, Yuan Pu, Yeo Eun Shin, Loren Laine, Ambrose Wong, René Kizilcec, Jasjeet Sekhon, Dennis Shung

Abstract: Applications of large language models (LLMs) like ChatGPT have potential to enhance clinical decision support through conversational interfaces. However, challenges of human-algorithmic interaction and clinician trust are poorly understood. GutGPT, a LLM for gastrointestinal (GI) bleeding risk prediction and management guidance, was deployed in clinical simulation scenarios alongside the electroni… ▽ More Applications of large language models (LLMs) like ChatGPT have potential to enhance clinical decision support through conversational interfaces. However, challenges of human-algorithmic interaction and clinician trust are poorly understood. GutGPT, a LLM for gastrointestinal (GI) bleeding risk prediction and management guidance, was deployed in clinical simulation scenarios alongside the electronic health record (EHR) with emergency medicine physicians, internal medicine physicians, and medical students to evaluate its effect on physician acceptance and trust in AI clinical decision support systems (AI-CDSS). GutGPT provides risk predictions from a validated machine learning model and evidence-based answers by querying extracted clinical guidelines. Participants were randomized to GutGPT and an interactive dashboard, or the interactive dashboard and a search engine. Surveys and educational assessments taken before and after measured technology acceptance and content mastery. Preliminary results showed mixed effects on acceptance after using GutGPT compared to the dashboard or search engine but appeared to improve content mastery based on simulation performance. Overall, this study demonstrates LLMs like GutGPT could enhance effective AI-CDSS if implemented optimally and paired with interactive interfaces. △ Less

Submitted 6 December, 2023; originally announced December 2023.

Comments: Extended Abstract presented at Machine Learning for Health (ML4H) symposium 2023, December 10, 2023, New Orleans, United States, 11 pages

IR-UWB Radar-Based Contactless Silent Speech Recognition of Vowels, Consonants, Words, and Phrases

Authors: Sunghwa Lee, Younghoon Shin, Myungjong Kim, Jiwon Seo

Abstract: Several sensing techniques have been proposed for silent speech recognition (SSR); however, many of these methods require invasive processes or sensor attachment to the skin using adhesive tape or glue, rendering them unsuitable for frequent use in daily life. By contrast, impulse radio ultra-wideband (IR-UWB) radar can operate without physical contact with users' articulators and related body par… ▽ More Several sensing techniques have been proposed for silent speech recognition (SSR); however, many of these methods require invasive processes or sensor attachment to the skin using adhesive tape or glue, rendering them unsuitable for frequent use in daily life. By contrast, impulse radio ultra-wideband (IR-UWB) radar can operate without physical contact with users' articulators and related body parts, offering several advantages for SSR. These advantages include high range resolution, high penetrability, low power consumption, robustness to external light or sound interference, and the ability to be embedded in space-constrained handheld devices. This study demonstrated IR-UWB radar-based contactless SSR using four types of speech stimuli (vowels, consonants, words, and phrases). To achieve this, a novel speech feature extraction algorithm specifically designed for IR-UWB radar-based SSR is proposed. Each speech stimulus is recognized by applying a classification algorithm to the extracted speech features. Two different algorithms, multidimensional dynamic time warping (MD-DTW) and deep neural network-hidden Markov model (DNN-HMM), were compared for the classification task. Additionally, a favorable radar antenna position, either in front of the user's lips or below the user's chin, was determined to achieve higher recognition accuracy. Experimental results demonstrated the efficacy of the proposed speech feature extraction algorithm combined with DNN-HMM for classifying vowels, consonants, words, and phrases. Notably, this study represents the first demonstration of phoneme-level SSR using contactless radar. △ Less

Submitted 15 December, 2023; originally announced December 2023.

Comments: Submitted to IEEE Access

Adaptive Shortcut Debiasing for Online Continual Learning

Authors: Doyoung Kim, Dongmin Park, Yooju Shin, Jihwan Bang, Hwanjun Song, Jae-Gil Lee

Abstract: We propose a novel framework DropTop that suppresses the shortcut bias in online continual learning (OCL) while being adaptive to the varying degree of the shortcut bias incurred by continuously changing environment. By the observed high-attention property of the shortcut bias, highly-activated features are considered candidates for debiasing. More importantly, resolving the limitation of the onli… ▽ More We propose a novel framework DropTop that suppresses the shortcut bias in online continual learning (OCL) while being adaptive to the varying degree of the shortcut bias incurred by continuously changing environment. By the observed high-attention property of the shortcut bias, highly-activated features are considered candidates for debiasing. More importantly, resolving the limitation of the online environment where prior knowledge and auxiliary data are not ready, two novel techniques -- feature map fusion and adaptive intensity shifting -- enable us to automatically determine the appropriate level and proportion of the candidate shortcut features to be dropped. Extensive experiments on five benchmark datasets demonstrate that, when combined with various OCL algorithms, DropTop increases the average accuracy by up to 10.4% and decreases the forgetting by up to 63.2%. △ Less

Submitted 14 December, 2023; originally announced December 2023.

Polynomial-based Self-Attention for Table Representation learning

Authors: Jayoung Kim, Yehjin Shin, Jeongwhan Choi, Hyowon Wi, Noseong Park

Abstract: Structured data, which constitutes a significant portion of existing data types, has been a long-standing research topic in the field of machine learning. Various representation learning methods for tabular data have been proposed, ranging from encoder-decoder structures to Transformers. Among these, Transformer-based methods have achieved state-of-the-art performance not only in tabular data but… ▽ More Structured data, which constitutes a significant portion of existing data types, has been a long-standing research topic in the field of machine learning. Various representation learning methods for tabular data have been proposed, ranging from encoder-decoder structures to Transformers. Among these, Transformer-based methods have achieved state-of-the-art performance not only in tabular data but also in various other fields, including computer vision and natural language processing. However, recent studies have revealed that self-attention, a key component of Transformers, can lead to an oversmoothing issue. We show that Transformers for tabular data also face this problem, and to address the problem, we propose a novel matrix polynomial-based self-attention layer as a substitute for the original self-attention layer, which enhances model scalability. In our experiments with three representative table learning models equipped with our proposed layer, we illustrate that the layer effectively mitigates the oversmoothing problem and enhances the representation performance of the existing methods, outperforming the state-of-the-art table representation methods. △ Less

Submitted 18 December, 2023; v1 submitted 12 December, 2023; originally announced December 2023.

Graph Convolutions Enrich the Self-Attention in Transformers!

Authors: Jeongwhan Choi, Hyowon Wi, Jayoung Kim, Yehjin Shin, Kookjin Lee, Nathaniel Trask, Noseong Park

Abstract: Transformers, renowned for their self-attention mechanism, have achieved state-of-the-art performance across various tasks in natural language processing, computer vision, time-series modeling, etc. However, one of the challenges with deep Transformer models is the oversmoothing problem, where representations across layers converge to indistinguishable values, leading to significant performance de… ▽ More Transformers, renowned for their self-attention mechanism, have achieved state-of-the-art performance across various tasks in natural language processing, computer vision, time-series modeling, etc. However, one of the challenges with deep Transformer models is the oversmoothing problem, where representations across layers converge to indistinguishable values, leading to significant performance degradation. We interpret the original self-attention as a simple graph filter and redesign it from a graph signal processing (GSP) perspective. We propose a graph-filter-based self-attention (GFSA) to learn a general yet effective one, whose complexity, however, is slightly larger than that of the original self-attention mechanism. We demonstrate that GFSA improves the performance of Transformers in various fields, including computer vision, natural language processing, graph regression, speech recognition, and code classification. △ Less

Submitted 30 May, 2024; v1 submitted 7 December, 2023; originally announced December 2023.

On Optimal Consistency-Robustness Trade-Off for Learning-Augmented Multi-Option Ski Rental

Authors: Yongho Shin, Changyeol Lee, Hyung-Chan An

Abstract: The learning-augmented multi-option ski rental problem generalizes the classical ski rental problem in two ways: the algorithm is provided with a prediction on the number of days we can ski, and the ski rental options now come with a variety of rental periods and prices to choose from, unlike the classical two-option setting. Subsequent to the initial study of the multi-option ski rental problem (… ▽ More The learning-augmented multi-option ski rental problem generalizes the classical ski rental problem in two ways: the algorithm is provided with a prediction on the number of days we can ski, and the ski rental options now come with a variety of rental periods and prices to choose from, unlike the classical two-option setting. Subsequent to the initial study of the multi-option ski rental problem (without learning augmentation) due to Zhang, Poon, and Xu, significant progress has been made for this problem recently in particular. The problem is very well understood when we relinquish one of the two generalizations -- for the learning-augmented classical ski rental problem, algorithms giving best-possible trade-off between consistency and robustness exist; for the multi-option ski rental problem without learning augmentation, deterministic/randomized algorithms giving the best-possible competitiveness have been found. However, in presence of both generalizations, there remained a huge gap between the algorithmic and impossibility results. In fact, for randomized algorithms, we did not have any nontrivial lower bounds on the consistency-robustness trade-off before. This paper bridges this gap for both deterministic and randomized algorithms. For deterministic algorithms, we present a best-possible algorithm that completely matches the known lower bound. For randomized algorithms, we show the first nontrivial lower bound on the consistency-robustness trade-off, and also present an improved randomized algorithm. Our algorithm matches our lower bound on robustness within a factor of e/2 when the consistency is at most 1.086. △ Less

Submitted 5 December, 2023; originally announced December 2023.

Comments: 16 pages, 2 figures

MSC Class: 68W27; 68T05 ACM Class: F.2.2; I.2.6

Propagate & Distill: Towards Effective Graph Learners Using Propagation-Embracing MLPs

Authors: Yong-Min Shin, Won-Yong Shin

Abstract: Recent studies attempted to utilize multilayer perceptrons (MLPs) to solve semisupervised node classification on graphs, by training a student MLP by knowledge distillation from a teacher graph neural network (GNN). While previous studies have focused mostly on training the student MLP by matching the output probability distributions between the teacher and student models during distillation, it h… ▽ More Recent studies attempted to utilize multilayer perceptrons (MLPs) to solve semisupervised node classification on graphs, by training a student MLP by knowledge distillation from a teacher graph neural network (GNN). While previous studies have focused mostly on training the student MLP by matching the output probability distributions between the teacher and student models during distillation, it has not been systematically studied how to inject the structural information in an explicit and interpretable manner. Inspired by GNNs that separate feature transformation $T$ and propagation $Π$, we re-frame the distillation process as making the student MLP learn both $T$ and $Π$. Although this can be achieved by applying the inverse propagation $Π^{-1}$ before distillation from the teacher, it still comes with a high computational cost from large matrix multiplications during training. To solve this problem, we propose Propagate & Distill (P&D), which propagates the output of the teacher before distillation, which can be interpreted as an approximate process of the inverse propagation. We demonstrate that P&D can readily improve the performance of the student MLP. △ Less

Submitted 29 November, 2023; originally announced November 2023.

Comments: 17 pages, 2 figures, 8 tables; 2nd Learning on Graphs Conference (LoG 2023) (Please cite our conference version.). arXiv admin note: substantial text overlap with arXiv:2311.11759

Unveiling the Unseen Potential of Graph Learning through MLPs: Effective Graph Learners Using Propagation-Embracing MLPs

Authors: Yong-Min Shin, Won-Yong Shin

Abstract: Recent studies attempted to utilize multilayer perceptrons (MLPs) to solve semi-supervised node classification on graphs, by training a student MLP by knowledge distillation (KD) from a teacher graph neural network (GNN). While previous studies have focused mostly on training the student MLP by matching the output probability distributions between the teacher and student models during KD, it has n… ▽ More Recent studies attempted to utilize multilayer perceptrons (MLPs) to solve semi-supervised node classification on graphs, by training a student MLP by knowledge distillation (KD) from a teacher graph neural network (GNN). While previous studies have focused mostly on training the student MLP by matching the output probability distributions between the teacher and student models during KD, it has not been systematically studied how to inject the structural information in an explicit and interpretable manner. Inspired by GNNs that separate feature transformation $T$ and propagation $Π$, we re-frame the KD process as enabling the student MLP to explicitly learn both $T$ and $Π$. Although this can be achieved by applying the inverse propagation $Π^{-1}$ before distillation from the teacher GNN, it still comes with a high computational cost from large matrix multiplications during training. To solve this problem, we propose Propagate & Distill (P&D), which propagates the output of the teacher GNN before KD and can be interpreted as an approximate process of the inverse propagation $Π^{-1}$. Through comprehensive evaluations using real-world benchmark datasets, we demonstrate the effectiveness of P&D by showing further performance boost of the student MLP. △ Less

Submitted 20 November, 2023; originally announced November 2023.

Comments: 35 pages, 5 figures, 8 tables

A Machine Learning Approach to Understanding the Physical Properties of Magnetic Flux Ropes in the Solar Wind at 1 AU

Authors: Hameedullah Farooki, Yasser Abduallah, Sung Jun Noh, Hyomin Kim, George Bizos, Youra Shin, Jason T. L. Wang, Haimin Wang

Abstract: Interplanetary magnetic flux ropes (MFRs) are commonly observed structures in the solar wind, categorized as magnetic clouds (MCs) and small-scale MFRs (SMFRs) depending on whether they are associated with coronal mass ejections. We apply machine learning to systematically compare SMFRs, MCs, and ambient solar wind plasma properties. We construct a dataset of 3-minute averaged sequential data poin… ▽ More Interplanetary magnetic flux ropes (MFRs) are commonly observed structures in the solar wind, categorized as magnetic clouds (MCs) and small-scale MFRs (SMFRs) depending on whether they are associated with coronal mass ejections. We apply machine learning to systematically compare SMFRs, MCs, and ambient solar wind plasma properties. We construct a dataset of 3-minute averaged sequential data points of the solar wind's instantaneous bulk fluid plasma properties using about twenty years of measurements from \emph{Wind}. We label samples by the presence and type of MFRs containing them using a catalog based on Grad-Shafranov (GS) automated detection for SMFRs and NASA's catalog for MCs (with samples in neither labeled non-MFRs). We apply the random forest machine learning algorithm to find which categories can be more easily distinguished and by what features. MCs were distinguished from non-MFRs with an AUC of 94% and SMFRs with an AUC of 89% and had distinctive plasma properties. In contrast, while SMFRs were distinguished from non-MFRs with an AUC of 86%, this appears to rely solely on the $\langle B \rangle$ > 5 nT threshold applied by the GS catalog. The results indicate that SMFRs have virtually the same plasma properties as the ambient solar wind, unlike the distinct plasma regimes of MCs. We interpret our findings as additional evidence that most SMFRs at 1 au are generated within the solar wind, and furthermore, suggesting that they should be considered a salient feature of the solar wind's magnetic structure rather than transient events. △ Less

Submitted 15 November, 2023; originally announced November 2023.

Comments: Accepted for publication to ApJ

Sample-Efficient and Safe Deep Reinforcement Learning via Reset Deep Ensemble Agents

Authors: Woojun Kim, Yongjae Shin, Jongeui Park, Youngchul Sung

Abstract: Deep reinforcement learning (RL) has achieved remarkable success in solving complex tasks through its integration with deep neural networks (DNNs) as function approximators. However, the reliance on DNNs has introduced a new challenge called primacy bias, whereby these function approximators tend to prioritize early experiences, leading to overfitting. To mitigate this primacy bias, a reset method… ▽ More Deep reinforcement learning (RL) has achieved remarkable success in solving complex tasks through its integration with deep neural networks (DNNs) as function approximators. However, the reliance on DNNs has introduced a new challenge called primacy bias, whereby these function approximators tend to prioritize early experiences, leading to overfitting. To mitigate this primacy bias, a reset method has been proposed, which performs periodic resets of a portion or the entirety of a deep RL agent while preserving the replay buffer. However, the use of the reset method can result in performance collapses after executing the reset, which can be detrimental from the perspective of safe RL and regret minimization. In this paper, we propose a new reset-based method that leverages deep ensemble learning to address the limitations of the vanilla reset method and enhance sample efficiency. The proposed method is evaluated through various experiments including those in the domain of safe RL. Numerical results show its effectiveness in high sample efficiency and safety considerations. △ Less

Submitted 31 October, 2023; originally announced October 2023.

Comments: NeurIPS 2023 camera-ready

Randomized Forward Mode of Automatic Differentiation For Optimization Algorithms

Authors: Khemraj Shukla, Yeonjong Shin

Abstract: We present a randomized forward mode gradient (RFG) as an alternative to backpropagation. RFG is a random estimator for the gradient that is constructed based on the directional derivative along a random vector. The forward mode automatic differentiation (AD) provides an efficient computation of RFG. The probability distribution of the random vector determines the statistical properties of RFG. Th… ▽ More We present a randomized forward mode gradient (RFG) as an alternative to backpropagation. RFG is a random estimator for the gradient that is constructed based on the directional derivative along a random vector. The forward mode automatic differentiation (AD) provides an efficient computation of RFG. The probability distribution of the random vector determines the statistical properties of RFG. Through the second moment analysis, we found that the distribution with the smallest kurtosis yields the smallest expected relative squared error. By replacing gradient with RFG, a class of RFG-based optimization algorithms is obtained. By focusing on gradient descent (GD) and Polyak's heavy ball (PHB) methods, we present a convergence analysis of RFG-based optimization algorithms for quadratic functions. Computational experiments are presented to demonstrate the performance of the proposed algorithms and verify the theoretical findings. △ Less

Submitted 1 February, 2024; v1 submitted 22 October, 2023; originally announced October 2023.

Comments: 22 Pages, 7 Figures

MSC Class: 65K05; 65B99; 65Y20

Object-Aware Impedance Control for Human-Robot Collaborative Task with Online Object Parameter Estimation

Authors: Jinseong Park, Yong-Sik Shin, Sanghyun Kim

Abstract: Physical human-robot interactions (pHRIs) can improve robot autonomy and reduce physical demands on humans. In this paper, we consider a collaborative task with a considerably long object and no prior knowledge of the object's parameters. An integrated control framework with an online object parameter estimator and a Cartesian object-aware impedance controller is proposed to realize complicated sc… ▽ More Physical human-robot interactions (pHRIs) can improve robot autonomy and reduce physical demands on humans. In this paper, we consider a collaborative task with a considerably long object and no prior knowledge of the object's parameters. An integrated control framework with an online object parameter estimator and a Cartesian object-aware impedance controller is proposed to realize complicated scenarios. During the transportation task, the object parameters are estimated online while a robot and human lift an object. The perturbation motion is incorporated into the null space of the desired trajectory to enhance the estimator accuracy. An object-aware impedance controller is designed using the real-time estimation results to effectively transmit the intended human motion to the robot through the object. Experimental demonstrations of collaborative tasks, including object transportation and assembly tasks, are implemented to show the effectiveness of our proposed method. △ Less

Submitted 18 October, 2023; originally announced October 2023.

Comments: 11 pages, 5 figures, for associated video, see https://youtu.be/bGH6GAFlRgA?si=wXj_SRzEE8BYoV2a

Domain Generalization for Medical Image Analysis: A Survey

Authors: Jee Seok Yoon, Kwanseok Oh, Yooseung Shin, Maciej A. Mazurowski, Heung-Il Suk

Abstract: Medical image analysis (MedIA) has become an essential tool in medicine and healthcare, aiding in disease diagnosis, prognosis, and treatment planning, and recent successes in deep learning (DL) have made significant contributions to its advances. However, deploying DL models for MedIA in real-world situations remains challenging due to their failure to generalize across the distributional gap bet… ▽ More Medical image analysis (MedIA) has become an essential tool in medicine and healthcare, aiding in disease diagnosis, prognosis, and treatment planning, and recent successes in deep learning (DL) have made significant contributions to its advances. However, deploying DL models for MedIA in real-world situations remains challenging due to their failure to generalize across the distributional gap between training and testing samples - a problem known as domain shift. Researchers have dedicated their efforts to developing various DL methods to adapt and perform robustly on unknown and out-of-distribution data distributions. This paper comprehensively reviews domain generalization studies specifically tailored for MedIA. We provide a holistic view of how domain generalization techniques interact within the broader MedIA system, going beyond methodologies to consider the operational implications on the entire MedIA workflow. Specifically, we categorize domain generalization methods into data-level, feature-level, model-level, and analysis-level methods. We show how those methods can be used in various stages of the MedIA workflow with DL equipped from data acquisition to model prediction and analysis. Furthermore, we critically analyze the strengths and weaknesses of various methods, unveiling future research opportunities. △ Less

Submitted 15 February, 2024; v1 submitted 5 October, 2023; originally announced October 2023.

Observation of universal Kibble-Zurek scaling in an atomic Fermi superfluid

Authors: Kyuhwan Lee, Sol Kim, Taehoon Kim, Yong-il Shin

Abstract: Half a century ago, T. Kibble proposed a scenario for topological defect formation from symmetry breaking during the expansion of the early Universe. W. Zurek later crystallized the concept to superfluid helium, predicting a power-law relation between the number of quantum vortices and the rate at which the system passes through the lambda transition. Here, we report the observation of Kibble-Zure… ▽ More Half a century ago, T. Kibble proposed a scenario for topological defect formation from symmetry breaking during the expansion of the early Universe. W. Zurek later crystallized the concept to superfluid helium, predicting a power-law relation between the number of quantum vortices and the rate at which the system passes through the lambda transition. Here, we report the observation of Kibble-Zurek scaling in a homogeneous, strongly interacting Fermi gas undergoing a superfluid phase transition. We investigate the superfluid transition using two distinct control parameters: temperature and interaction strength. The microscopic physics of condensate formation is markedly different for the two quench parameters, signaled by their two orders of magnitude difference in the condensate formation timescale. However, regardless of the thermodynamic direction in which the system passes through a phase transition, the Kibble-Zurek exponent is identically observed to be about 0.68 and shows good agreement with theoretical predictions that describe superfluid phase transitions. This work demonstrates the gedanken experiment Zurek proposed for liquid helium that shares the same universality class with strongly interacting Fermi gases. △ Less

Submitted 9 October, 2023; originally announced October 2023.