Showing 1–50 of 570 results for author: Han, M

The MICADO first light imager for the ELT: overview and current Status

Authors: E. Sturm, R. Davies, J. Alves, Y. Clénet, J. Kotilainen, A. Monna, H. Nicklas, J. -U. Pott, E. Tolstoy, B. Vulcani, J. Achren, S. Annadevara, H. Anwand-Heerwart, C. Arcidiacono, S. Barboza, L. Barl, P. Baudoz, R. Bender, N. Bezawada, F. Biondi, P. Bizenberger, A. Blin, A. Boné, P. Bonifacio, B. Borgo , et al. (129 additional authors not shown)

Abstract: MICADO is a first light instrument for the Extremely Large Telescope (ELT), set to start operating later this decade. It will provide diffraction limited imaging, astrometry, high contrast imaging, and long slit spectroscopy at near-infrared wavelengths. During the initial phase operations, adaptive optics (AO) correction will be provided by its own natural guide star wavefront sensor. In its fina… ▽ More MICADO is a first light instrument for the Extremely Large Telescope (ELT), set to start operating later this decade. It will provide diffraction limited imaging, astrometry, high contrast imaging, and long slit spectroscopy at near-infrared wavelengths. During the initial phase operations, adaptive optics (AO) correction will be provided by its own natural guide star wavefront sensor. In its final configuration, that AO system will be retained and complemented by the laser guide star multi-conjugate adaptive optics module MORFEO (formerly known as MAORY). Among many other things, MICADO will study exoplanets, distant galaxies and stars, and investigate black holes, such as Sagittarius A* at the centre of the Milky Way. After their final design phase, most components of MICADO have moved on to the manufacturing and assembly phase. Here we summarize the final design of the instrument and provide an overview about its current manufacturing status and the timeline. Some lessons learned from the final design review process will be presented in order to help future instrumentation projects to cope with the challenges arising from the substantial differences between projects for 8-10m class telescopes (e.g. ESO-VLT) and the next generation Extremely Large Telescopes (e.g. ESO-ELT). Finally, the expected performance will be discussed in the context of the current landscape of astronomical observatories and instruments. For instance, MICADO will have similar sensitivity as the James Webb Space Telescope (JWST), but with six times the spatial resolution. △ Less

Submitted 29 August, 2024; originally announced August 2024.

Comments: Proceedings of the SPIE, Volume 13096, id. 1309611 11 pp. (2024)

Systematic Evaluation of LLM-as-a-Judge in LLM Alignment Tasks: Explainable Metrics and Diverse Prompt Templates

Authors: Hui Wei, Shenghua He, Tian Xia, Andy Wong, Jingyang Lin, Mei Han

Abstract: Alignment approaches such as RLHF and DPO are actively investigated to align large language models (LLMs) with human preferences. Commercial large language models (LLMs) like GPT-4 have been recently employed to evaluate and compare different LLM alignment approaches. These models act as surrogates for human evaluators due to their promising abilities to approximate human preferences with remarkab… ▽ More Alignment approaches such as RLHF and DPO are actively investigated to align large language models (LLMs) with human preferences. Commercial large language models (LLMs) like GPT-4 have been recently employed to evaluate and compare different LLM alignment approaches. These models act as surrogates for human evaluators due to their promising abilities to approximate human preferences with remarkably faster feedback and lower costs. This methodology is referred to as LLM-as-a-judge. However, concerns regarding its reliability have emerged, attributed to LLM judges' biases and inconsistent decision-making. Previous research has sought to develop robust evaluation frameworks for assessing the reliability of LLM judges and their alignment with human preferences. However, the employed evaluation metrics often lack adequate explainability and fail to address the internal inconsistency of LLMs. Additionally, existing studies inadequately explore the impact of various prompt templates when applying LLM-as-a-judge methods, which leads to potentially inconsistent comparisons between different alignment algorithms. In this work, we systematically evaluate LLM judges on alignment tasks (e.g. summarization) by defining evaluation metrics with improved theoretical interpretability and disentangling reliability metrics with LLM internal inconsistency. We develop a framework to evaluate, compare, and visualize the reliability and alignment of LLM judges to provide informative observations that help choose LLM judges for alignment tasks. Our results indicate a significant impact of prompt templates on LLM judge performance, as well as a mediocre alignment level between the tested LLM judges and human evaluators. △ Less

Submitted 23 August, 2024; originally announced August 2024.

Comments: Preprint, under review. 17 pages, 7 figures, 16 tables

MSCPT: Few-shot Whole Slide Image Classification with Multi-scale and Context-focused Prompt Tuning

Authors: Minghao Han, Linhao Qu, Dingkang Yang, Xukun Zhang, Xiaoying Wang, Lihua Zhang

Abstract: Multiple instance learning (MIL) has become a standard paradigm for weakly supervised classification of whole slide images (WSI). However, this paradigm relies on the use of a large number of labelled WSIs for training. The lack of training data and the presence of rare diseases present significant challenges for these methods. Prompt tuning combined with the pre-trained Vision-Language models (VL… ▽ More Multiple instance learning (MIL) has become a standard paradigm for weakly supervised classification of whole slide images (WSI). However, this paradigm relies on the use of a large number of labelled WSIs for training. The lack of training data and the presence of rare diseases present significant challenges for these methods. Prompt tuning combined with the pre-trained Vision-Language models (VLMs) is an effective solution to the Few-shot Weakly Supervised WSI classification (FSWC) tasks. Nevertheless, applying prompt tuning methods designed for natural images to WSIs presents three significant challenges: 1) These methods fail to fully leverage the prior knowledge from the VLM's text modality; 2) They overlook the essential multi-scale and contextual information in WSIs, leading to suboptimal results; and 3) They lack exploration of instance aggregation methods. To address these problems, we propose a Multi-Scale and Context-focused Prompt Tuning (MSCPT) method for FSWC tasks. Specifically, MSCPT employs the frozen large language model to generate pathological visual language prior knowledge at multi-scale, guiding hierarchical prompt tuning. Additionally, we design a graph prompt tuning module to learn essential contextual information within WSI, and finally, a non-parametric cross-guided instance aggregation module has been introduced to get the WSI-level features. Based on two VLMs, extensive experiments and visualizations on three datasets demonstrated the powerful performance of our MSCPT. △ Less

Submitted 21 August, 2024; originally announced August 2024.

Comments: 11 pages, 5 figures, 5tables

NutrifyAI: An AI-Powered System for Real-Time Food Detection, Nutritional Analysis, and Personalized Meal Recommendations

Authors: Michelle Han, Junyao Chen

Abstract: With diet and nutrition apps reaching 1.4 billion users in 2022 [1], it's no surprise that health apps like MyFitnessPal, Noom, and Calorie Counter, are surging in popularity. However, one major setback [2] of nearly all nutrition applications is that users must enter food data manually, which is time-consuming and tedious. Thus, there has been an increasing demand for applications that can accura… ▽ More With diet and nutrition apps reaching 1.4 billion users in 2022 [1], it's no surprise that health apps like MyFitnessPal, Noom, and Calorie Counter, are surging in popularity. However, one major setback [2] of nearly all nutrition applications is that users must enter food data manually, which is time-consuming and tedious. Thus, there has been an increasing demand for applications that can accurately identify food items, analyze their nutritional content, and offer dietary recommendations in real-time. This paper introduces a comprehensive system that combines advanced computer vision techniques with nutrition analysis, implemented in a versatile mobile and web application. The system is divided into three key components: 1) food detection using the YOLOv8 model, 2) nutrient analysis via the Edamam Nutrition Analysis API, and 3) personalized meal recommendations using the Edamam Meal Planning and Recipe Search APIs. Designed for both mobile and web platforms, the application ensures fast processing times with an intuitive user interface, with features such as data visualizations using Chart.js, a login system, and personalized settings for dietary preferences, allergies, and cuisine choices. Preliminary results showcase the system's effectiveness, making it a valuable tool for users to make informed dietary decisions. △ Less

Submitted 20 August, 2024; originally announced August 2024.

Comments: 7 pages, 12 figures

DDIM Redux: Mathematical Foundation and Some Extension

Authors: Manhyung Han

Abstract: This note provides a critical review of the mathematical concepts underlying the generalized diffusion denoising implicit model (gDDIM) and the exponential integrator (EI) scheme. We present enhanced mathematical results, including an exact expression for the reverse trajectory in the probability flow ODE and an exact expression for the covariance matrix in the gDDIM scheme. Furthermore, we offer… ▽ More This note provides a critical review of the mathematical concepts underlying the generalized diffusion denoising implicit model (gDDIM) and the exponential integrator (EI) scheme. We present enhanced mathematical results, including an exact expression for the reverse trajectory in the probability flow ODE and an exact expression for the covariance matrix in the gDDIM scheme. Furthermore, we offer an improved understanding of the EI scheme's efficiency in terms of the change of variables. The noising process in DDIM is analyzed from the perspective of non-equilibrium statistical physics. Additionally, we propose a new scheme for DDIM, called the principal-axis DDIM (paDDIM). △ Less

Submitted 2 August, 2024; originally announced August 2024.

One-dimensional spin-flipping topological edge state laser

Authors: Jhih-Sheng Wu, Zhen-Ting Huang, Meng-Ting Han, Yen-Hsun Chen, Tien-Chang Lu

Abstract: Topological edge states manifest spin-momentum-locking propagation as a primary consequence of topological crystals. However, experimental studies on spin manipulation and the resulting propagation of these states are lacking. Here, we demonstrate experimentally spin manipulation of topological edge states by the boundary conditions of the one-dimensional path. Armchair boundaries at the endpoints… ▽ More Topological edge states manifest spin-momentum-locking propagation as a primary consequence of topological crystals. However, experimental studies on spin manipulation and the resulting propagation of these states are lacking. Here, we demonstrate experimentally spin manipulation of topological edge states by the boundary conditions of the one-dimensional path. Armchair boundaries at the endpoints of the path induce spin-flipping back-scattering, resulting in a novel one-dimensional resonance -- traveling resonance. Remarkably, we demonstrate lasing of this one-dimensional traveling resonance. Our findings hold significant potential for practical applications in spin manipulation of light. △ Less

Submitted 9 August, 2024; originally announced August 2024.

Comments: 9 pages, 6 figures

Self-tuning moving horizon estimation of nonlinear systems via physics-informed machine learning Koopman modeling

Authors: Mingxue Yan, Minghao Han, Adrian Wing-Keung Law, Xunyuan Yin

Abstract: In this paper, we propose a physics-informed learning-based Koopman modeling approach and present a Koopman-based self-tuning moving horizon estimation design for a class of nonlinear systems. Specifically, we train Koopman operators and two neural networks - the state lifting network and the noise characterization network - using both data and available physical information. The two neural networ… ▽ More In this paper, we propose a physics-informed learning-based Koopman modeling approach and present a Koopman-based self-tuning moving horizon estimation design for a class of nonlinear systems. Specifically, we train Koopman operators and two neural networks - the state lifting network and the noise characterization network - using both data and available physical information. The two neural networks account for the nonlinear lifting functions for Koopman modeling and describing system noise distributions, respectively. Accordingly, a stochastic linear Koopman model is established in the lifted space to forecast the dynamic behavior of the nonlinear system. Based on the Koopman model, a self-tuning linear moving horizon estimation (MHE) scheme is developed. The weighting matrices of the MHE design are updated using the pre-trained noise characterization network at each sampling instant. The proposed estimation scheme is computationally efficient because only convex optimization is involved during online implementation, and updating the weighting matrices of the MHE scheme does not require re-training the neural networks. We verify the effectiveness and evaluate the performance of the proposed method via the application to a simulated chemical process. △ Less

Submitted 7 August, 2024; originally announced August 2024.

Comments: 31 pages, 7 figures

Machine learning-based input-augmented Koopman modeling and predictive control of nonlinear processes

Authors: Zhaoyang Li, Minghao Han, Dat-Nguyen Vo, Xunyuan Yin

Abstract: Koopman-based modeling and model predictive control have been a promising alternative for optimal control of nonlinear processes. Good Koopman modeling performance significantly depends on an appropriate nonlinear mapping from the original state-space to a lifted state space. In this work, we propose an input-augmented Koopman modeling and model predictive control approach. Both the states and the… ▽ More Koopman-based modeling and model predictive control have been a promising alternative for optimal control of nonlinear processes. Good Koopman modeling performance significantly depends on an appropriate nonlinear mapping from the original state-space to a lifted state space. In this work, we propose an input-augmented Koopman modeling and model predictive control approach. Both the states and the known inputs are lifted using two deep neural networks (DNNs), and a Koopman model with nonlinearity in inputs is trained within the higher-dimensional state-space. A Koopman-based model predictive control problem is formulated. To bypass non-convex optimization induced by the nonlinearity in the Koopman model, we further present an iterative implementation algorithm, which approximates the optimal control input via solving a convex optimization problem iteratively. The proposed method is applied to a chemical process and a biological water treatment process via simulations. The efficacy and advantages of the proposed modeling and control approach are demonstrated. △ Less

Submitted 5 August, 2024; originally announced August 2024.

Escape Sensing Games: Detection-vs-Evasion in Security Applications

Authors: Niclas Boehmer, Minbiao Han, Haifeng Xu, Milind Tambe

Abstract: Traditional game-theoretic research for security applications primarily focuses on the allocation of external protection resources to defend targets. This work puts forward the study of a new class of games centered around strategically arranging targets to protect them against a constrained adversary, with motivations from varied domains such as peacekeeping resource transit and cybersecurity. Sp… ▽ More Traditional game-theoretic research for security applications primarily focuses on the allocation of external protection resources to defend targets. This work puts forward the study of a new class of games centered around strategically arranging targets to protect them against a constrained adversary, with motivations from varied domains such as peacekeeping resource transit and cybersecurity. Specifically, we introduce Escape Sensing Games (ESGs). In ESGs, a blue player manages the order in which targets pass through a channel, while her opponent tries to capture the targets using a set of sensors that need some time to recharge after each activation. We present a thorough computational study of ESGs. Among others, we show that it is NP-hard to compute best responses and equilibria. Nevertheless, we propose a variety of effective (heuristic) algorithms whose quality we demonstrate in extensive computational experiments. △ Less

Submitted 30 July, 2024; originally announced July 2024.

ByteCheckpoint: A Unified Checkpointing System for LLM Development

Authors: Borui Wan, Mingji Han, Yiyao Sheng, Zhichao Lai, Mofan Zhang, Junda Zhang, Yanghua Peng, Haibin Lin, Xin Liu, Chuan Wu

Abstract: The development of real-world Large Language Models (LLMs) necessitates checkpointing of training states in persistent storage to mitigate potential software and hardware failures, as well as to facilitate checkpoint transferring within the training pipeline and across various tasks. Due to the immense size of LLMs, saving and loading checkpoints often incur intolerable minute-level stalls, signif… ▽ More The development of real-world Large Language Models (LLMs) necessitates checkpointing of training states in persistent storage to mitigate potential software and hardware failures, as well as to facilitate checkpoint transferring within the training pipeline and across various tasks. Due to the immense size of LLMs, saving and loading checkpoints often incur intolerable minute-level stalls, significantly diminishing training efficiency. Besides, when transferring checkpoints across tasks, checkpoint resharding, defined as loading checkpoints into parallel configurations differing from those used for saving, is often required according to the characteristics and resource quota of specific tasks. Previous checkpointing systems [16,3,33,6] assume consistent parallel configurations, failing to address the complexities of checkpoint transformation during resharding. Furthermore, in the industry platform, developers create checkpoints from different training frameworks[23,36,21,11], each with its own unique storage and I/O logic. This diversity complicates the implementation of unified checkpoint management and optimization. To address these challenges, we introduce ByteCheckpoint, a PyTorch-native multi-framework LLM checkpointing system that supports automatic online checkpoint resharding. ByteCheckpoint employs a data/metadata disaggregated storage architecture, decoupling checkpoint storage from the adopted parallelism strategies and training frameworks. We design an efficient asynchronous tensor merging technique to settle the irregular tensor sharding problem and propose several I/O performance optimizations to significantly enhance the efficiency of checkpoint saving and loading. Experimental results demonstrate ByteCheckpoint's substantial advantages in reducing checkpoint saving (by up to 529.22X) and loading (by up to 3.51X) costs, compared to baseline methods. △ Less

Submitted 29 July, 2024; originally announced July 2024.

Diff-Shadow: Global-guided Diffusion Model for Shadow Removal

Authors: Jinting Luo, Ru Li, Chengzhi Jiang, Mingyan Han, Xiaoming Zhang, Ting Jiang, Haoqiang Fan, Shuaicheng Liu

Abstract: We propose Diff-Shadow, a global-guided diffusion model for high-quality shadow removal. Previous transformer-based approaches can utilize global information to relate shadow and non-shadow regions but are limited in their synthesis ability and recover images with obvious boundaries. In contrast, diffusion-based methods can generate better content but ignore global information, resulting in incons… ▽ More We propose Diff-Shadow, a global-guided diffusion model for high-quality shadow removal. Previous transformer-based approaches can utilize global information to relate shadow and non-shadow regions but are limited in their synthesis ability and recover images with obvious boundaries. In contrast, diffusion-based methods can generate better content but ignore global information, resulting in inconsistent illumination. In this work, we combine the advantages of diffusion models and global guidance to realize shadow-free restoration. Specifically, we propose a parallel UNets architecture: 1) the local branch performs the patch-based noise estimation in the diffusion process, and 2) the global branch recovers the low-resolution shadow-free images. A Reweight Cross Attention (RCA) module is designed to integrate global contextural information of non-shadow regions into the local branch. We further design a Global-guided Sampling Strategy (GSS) that mitigates patch boundary issues and ensures consistent illumination across shaded and unshaded regions in the recovered image. Comprehensive experiments on three publicly standard datasets ISTD, ISTD+, and SRD have demonstrated the effectiveness of Diff-Shadow. Compared to state-of-the-art methods, our method achieves a significant improvement in terms of PSNR, increasing from 32.33dB to 33.69dB on the SRD dataset. Codes will be released. △ Less

Submitted 23 July, 2024; originally announced July 2024.

Figure it Out: Analyzing-based Jailbreak Attack on Large Language Models

Authors: Shi Lin, Rongchang Li, Xun Wang, Changting Lin, Wenpeng Xing, Meng Han

Abstract: The rapid development of Large Language Models (LLMs) has brought remarkable generative capabilities across diverse tasks. However, despite the impressive achievements, these LLMs still have numerous inherent vulnerabilities, particularly when faced with jailbreak attacks. By investigating jailbreak attacks, we can uncover hidden weaknesses in LLMs and inform the development of more robust defense… ▽ More The rapid development of Large Language Models (LLMs) has brought remarkable generative capabilities across diverse tasks. However, despite the impressive achievements, these LLMs still have numerous inherent vulnerabilities, particularly when faced with jailbreak attacks. By investigating jailbreak attacks, we can uncover hidden weaknesses in LLMs and inform the development of more robust defense mechanisms to fortify their security. In this paper, we further explore the boundary of jailbreak attacks on LLMs and propose Analyzing-based Jailbreak (ABJ). This effective jailbreak attack method takes advantage of LLMs' growing analyzing and reasoning capability and reveals their underlying vulnerabilities when facing analyzing-based tasks. We conduct a detailed evaluation of ABJ across various open-source and closed-source LLMs, which achieves 94.8% attack success rate (ASR) and 1.06 attack efficiency (AE) on GPT-4-turbo-0409, demonstrating state-of-the-art attack effectiveness and efficiency. Our research highlights the importance of prioritizing and enhancing the safety of LLMs to mitigate the risks of misuse. The code is publicly available at hhttps://github.com/theshi-1128/ABJ-Attack. Warning: This paper contains examples of LLMs that might be offensive or harmful. △ Less

Submitted 13 August, 2024; v1 submitted 23 July, 2024; originally announced July 2024.

Fact-Aware Multimodal Retrieval Augmentation for Accurate Medical Radiology Report Generation

Authors: Liwen Sun, James Zhao, Megan Han, Chenyan Xiong

Abstract: Multimodal foundation models hold significant potential for automating radiology report generation, thereby assisting clinicians in diagnosing cardiac diseases. However, generated reports often suffer from serious factual inaccuracy. In this paper, we introduce a fact-aware multimodal retrieval-augmented pipeline in generating accurate radiology reports (FactMM-RAG). We first leverage RadGraph to… ▽ More Multimodal foundation models hold significant potential for automating radiology report generation, thereby assisting clinicians in diagnosing cardiac diseases. However, generated reports often suffer from serious factual inaccuracy. In this paper, we introduce a fact-aware multimodal retrieval-augmented pipeline in generating accurate radiology reports (FactMM-RAG). We first leverage RadGraph to mine factual report pairs, then integrate factual knowledge to train a universal multimodal retriever. Given a radiology image, our retriever can identify high-quality reference reports to augment multimodal foundation models, thus enhancing the factual completeness and correctness of report generation. Experiments on two benchmark datasets show that our multimodal retriever outperforms state-of-the-art retrievers on both language generation and radiology-specific metrics, up to 6.5% and 2% score in F1CheXbert and F1RadGraph. Further analysis indicates that employing our factually-informed training strategy imposes an effective supervision signal, without relying on explicit diagnostic label guidance, and successfully propagates fact-aware capabilities from the multimodal retriever to the multimodal foundation model in radiology report generation. △ Less

Submitted 21 July, 2024; originally announced July 2024.

Atomic-Layer-Controlled Magnetic Orders in MnBi2Te4-Bi2Te3 Topological Heterostructures

Authors: Xiong Yao, Qirui Cui, Zengle Huang, Xiaoyu Yuan, Hee Taek Yi, Deepti Jain, Kim Kisslinger, Myung-Geun Han, Weida Wu, Hongxin Yang, Seongshik Oh

Abstract: The natural van der Waals superlattice MnBi2Te4-(Bi2Te3)m provides an optimal platform to combine topology and magnetism in one system with minimal structural disorder. Here, we show that this system can harbor both ferromagnetic (FM) and antiferromagnetic (AFM) orders and that these magnetic orders can be controlled in two different ways by either varying the Mn-Mn distance while keeping the Bi2T… ▽ More The natural van der Waals superlattice MnBi2Te4-(Bi2Te3)m provides an optimal platform to combine topology and magnetism in one system with minimal structural disorder. Here, we show that this system can harbor both ferromagnetic (FM) and antiferromagnetic (AFM) orders and that these magnetic orders can be controlled in two different ways by either varying the Mn-Mn distance while keeping the Bi2Te3/MnBi2Te4 ratio constant or vice versa. We achieve this by creating atomically engineered sandwich structures composed of Bi2Te3 and MnBi2Te4 layers. We show that the AFM order is exclusively determined by the Mn-Mn distance whereas the FM order depends only on the overall Bi2Te3/MnBi2Te4 ratio regardless of the distance between the MnBi2Te4 layers. Our results shed light on the origins of the AFM and FM orders and provide insights into how to manipulate magnetic orders not only for the MnBi2Te4-Bi2Te3 system but also for other magneto-topological materials. △ Less

Submitted 20 July, 2024; originally announced July 2024.

Comments: 25 pages, 5 figures, accepted to Nano Letters

Overview of AI-Debater 2023: The Challenges of Argument Generation Tasks

Authors: Jiayu Lin, Guanrong Chen, Bojun Jin, Chenyang Li, Shutong Jia, Wancong Lin, Yang Sun, Yuhang He, Caihua Yang, Jianzhu Bao, Jipeng Wu, Wen Su, Jinglu Chen, Xinyi Li, Tianyu Chen, Mingjie Han, Shuaiwen Du, Zijian Wang, Jiyin Li, Fuzhong Suo, Hao Wang, Nuanchen Lin, Xuanjing Huang, Changjian Jiang, RuiFeng Xu , et al. (4 additional authors not shown)

Abstract: In this paper we present the results of the AI-Debater 2023 Challenge held by the Chinese Conference on Affect Computing (CCAC 2023), and introduce the related datasets. We organize two tracks to handle the argumentative generation tasks in different scenarios, namely, Counter-Argument Generation (Track 1) and Claim-based Argument Generation (Track 2). Each track is equipped with its distinct data… ▽ More In this paper we present the results of the AI-Debater 2023 Challenge held by the Chinese Conference on Affect Computing (CCAC 2023), and introduce the related datasets. We organize two tracks to handle the argumentative generation tasks in different scenarios, namely, Counter-Argument Generation (Track 1) and Claim-based Argument Generation (Track 2). Each track is equipped with its distinct dataset and baseline model respectively. In total, 32 competing teams register for the challenge, from which we received 11 successful submissions. In this paper, we will present the results of the challenge and a summary of the systems, highlighting commonalities and innovations among participating systems. Datasets and baseline models of the AI-Debater 2023 Challenge have been already released and can be accessed through the official website of the challenge. △ Less

Submitted 24 July, 2024; v1 submitted 20 July, 2024; originally announced July 2024.

The object detection method aids in image reconstruction evaluation and clinical interpretation of meniscal abnormalities

Authors: Natalia Konovalova, Aniket Tolpadi, Felix Liu, Zehra Akkaya, Felix Gassert, Paula Giesler, Johanna Luitjens, Misung Han, Emma Bahroos, Sharmila Majumdar, Valentina Pedoia

Abstract: This study investigates the relationship between deep learning (DL) image reconstruction quality and anomaly detection performance, and evaluates the efficacy of an artificial intelligence (AI) assistant in enhancing radiologists' interpretation of meniscal anomalies on reconstructed images. A retrospective study was conducted using an in-house reconstruction and anomaly detection pipeline to asse… ▽ More This study investigates the relationship between deep learning (DL) image reconstruction quality and anomaly detection performance, and evaluates the efficacy of an artificial intelligence (AI) assistant in enhancing radiologists' interpretation of meniscal anomalies on reconstructed images. A retrospective study was conducted using an in-house reconstruction and anomaly detection pipeline to assess knee MR images from 896 patients. The original and 14 sets of DL-reconstructed images were evaluated using standard reconstruction and object detection metrics, alongside newly developed box-based reconstruction metrics. Two clinical radiologists reviewed a subset of 50 patients' images, both original and AI-assisted reconstructed, with subsequent assessment of their accuracy and performance characteristics. Results indicated that the structural similarity index (SSIM) showed a weaker correlation with anomaly detection metrics (mAP, r=0.64, p=0.01; F1 score, r=0.38, p=0.18), while box-based SSIM had a stronger association with detection performance (mAP, r=0.81, p<0.01; F1 score, r=0.65, p=0.01). Minor SSIM fluctuations did not affect detection outcomes, but significant changes reduced performance. Radiologists' AI-assisted evaluations demonstrated improved accuracy (86.0% without assistance vs. 88.3% with assistance, p<0.05) and interrater agreement (Cohen's kappa, 0.39 without assistance vs. 0.57 with assistance). An additional review led to the incorporation of 17 more lesions into the dataset. The proposed anomaly detection method shows promise in evaluating reconstruction algorithms for automated tasks and aiding radiologists in interpreting DL-reconstructed MR images. △ Less

Submitted 16 July, 2024; originally announced July 2024.

Analytical Expression for Continuum-continuum Transition Amplitude of Hydrogen-like Atoms with Angular-momentum Dependence

Authors: Jia-Bao Ji, Kiyoshi Ueda, Meng Han, Hans Jakob Wörner

Abstract: Attosecond chronoscopy typically utilises interfering two-photon transitions to access the phase information. Simulating these two-photon transitions is challenging due to the continuum-continuum transition term. The hydrogenic approximation within second-order perturbation theory has been widely used due to the existence of analytical expressions of the wave functions. So far, only (partially) as… ▽ More Attosecond chronoscopy typically utilises interfering two-photon transitions to access the phase information. Simulating these two-photon transitions is challenging due to the continuum-continuum transition term. The hydrogenic approximation within second-order perturbation theory has been widely used due to the existence of analytical expressions of the wave functions. So far, only (partially) asymptotic results have been derived, which fail to correctly describe the low-kinetic-energy behaviour, especially for high angular-momentum states. Here, we report an analytical expression that overcome these limitations. They are based on the Appell's F1 function and use the confluent hypergeometric function of the second kind as the intermediate states. We show that the derived formula can be applied to arbitrarily high angular momenta and quantitatively agrees with the numerical simulations using the time-dependent Schrödinger equation. Furthermore, we give an angular-momentum-dependent asymptotic form of the outgoing wavefunction and their continuum-continuum dipole transition amplitudes. △ Less

Submitted 12 July, 2024; originally announced July 2024.

Seed-ASR: Understanding Diverse Speech and Contexts with LLM-based Speech Recognition

Authors: Ye Bai, Jingping Chen, Jitong Chen, Wei Chen, Zhuo Chen, Chuang Ding, Linhao Dong, Qianqian Dong, Yujiao Du, Kepan Gao, Lu Gao, Yi Guo, Minglun Han, Ting Han, Wenchao Hu, Xinying Hu, Yuxiang Hu, Deyu Hua, Lu Huang, Mingkun Huang, Youjia Huang, Jishuo Jin, Fanliu Kong, Zongwei Lan, Tianyu Li , et al. (30 additional authors not shown)

Abstract: Modern automatic speech recognition (ASR) model is required to accurately transcribe diverse speech signals (from different domains, languages, accents, etc) given the specific contextual information in various application scenarios. Classic end-to-end models fused with extra language models perform well, but mainly in data matching scenarios and are gradually approaching a bottleneck. In this wor… ▽ More Modern automatic speech recognition (ASR) model is required to accurately transcribe diverse speech signals (from different domains, languages, accents, etc) given the specific contextual information in various application scenarios. Classic end-to-end models fused with extra language models perform well, but mainly in data matching scenarios and are gradually approaching a bottleneck. In this work, we introduce Seed-ASR, a large language model (LLM) based speech recognition model. Seed-ASR is developed based on the framework of audio conditioned LLM (AcLLM), leveraging the capabilities of LLMs by inputting continuous speech representations together with contextual information into the LLM. Through stage-wise large-scale training and the elicitation of context-aware capabilities in LLM, Seed-ASR demonstrates significant improvement over end-to-end models on comprehensive evaluation sets, including multiple domains, accents/dialects and languages. Additionally, Seed-ASR can be further deployed to support specific needs in various scenarios without requiring extra language models. Compared to recently released large ASR models, Seed-ASR achieves 10%-40% reduction in word (or character, for Chinese) error rates on Chinese and English public test sets, further demonstrating its powerful performance. △ Less

Submitted 10 July, 2024; v1 submitted 5 July, 2024; originally announced July 2024.

E-SAGE: Explainability-based Defense Against Backdoor Attacks on Graph Neural Networks

Authors: Dingqiang Yuan, Xiaohua Xu, Lei Yu, Tongchang Han, Rongchang Li, Meng Han

Abstract: Graph Neural Networks (GNNs) have recently been widely adopted in multiple domains. Yet, they are notably vulnerable to adversarial and backdoor attacks. In particular, backdoor attacks based on subgraph insertion have been shown to be effective in graph classification tasks while being stealthy, successfully circumventing various existing defense methods. In this paper, we propose E-SAGE, a novel… ▽ More Graph Neural Networks (GNNs) have recently been widely adopted in multiple domains. Yet, they are notably vulnerable to adversarial and backdoor attacks. In particular, backdoor attacks based on subgraph insertion have been shown to be effective in graph classification tasks while being stealthy, successfully circumventing various existing defense methods. In this paper, we propose E-SAGE, a novel approach to defending GNN backdoor attacks based on explainability. We find that the malicious edges and benign edges have significant differences in the importance scores for explainability evaluation. Accordingly, E-SAGE adaptively applies an iterative edge pruning process on the graph based on the edge scores. Through extensive experiments, we demonstrate the effectiveness of E-SAGE against state-of-the-art graph backdoor attacks in different attack settings. In addition, we investigate the effectiveness of E-SAGE against adversarial attacks. △ Less

Submitted 15 June, 2024; originally announced June 2024.

Stoichiometry-induced ferromagnetism in altermagnetic candidate MnTe

Authors: Michael Chilcote, Alessandro R. Mazza, Qiangsheng Lu, Isaiah Gray, Qi Tian, Qinwen Deng, Duncan Moseley, An-Hsi Chen, Jason Lapano, Jason S. Gardner, Gyula Eres, T. Zac Ward, Erxi Feng, Huibo Cao, Valeria Lauter, Michael A. McGuire, Raphael Hermann, David Parker, Myung-Geun Han, Asghar Kayani, Gaurab Rimal, Liang Wu, Timothy R. Charlton, Robert G. Moore, Matthew Brahlek

Abstract: The field of spintronics has seen a surge of interest in altermagnetism due to novel predictions and many possible applications. MnTe is a leading altermagnetic candidate that is of significant interest across spintronics due to its layered antiferromagnetic structure, high Neel temperature (TN ~ 310 K) and semiconducting properties. We present results on molecular beam epitaxy (MBE) grown MnTe/In… ▽ More The field of spintronics has seen a surge of interest in altermagnetism due to novel predictions and many possible applications. MnTe is a leading altermagnetic candidate that is of significant interest across spintronics due to its layered antiferromagnetic structure, high Neel temperature (TN ~ 310 K) and semiconducting properties. We present results on molecular beam epitaxy (MBE) grown MnTe/InP(111) films. Here, it is found that the electronic and magnetic properties are driven by the natural stoichiometry of MnTe. Electronic transport and in situ angle-resolved photoemission spectroscopy show the films are natively metallic with the Fermi level in the valence band and the band structure is in good agreement with first principles calculations for altermagnetic spin-splitting. Neutron diffraction confirms that the film is antiferromagnetic with planar anisotropy and polarized neutron reflectometry indicates weak ferromagnetism, which is linked to a slight Mn-richness that is intrinsic to the MBE grown samples. When combined with the anomalous Hall effect, this work shows that the electronic response is strongly affected by the ferromagnetic moment. Altogether, this highlights potential mechanisms for controlling altermagnetic ordering for diverse spintronic applications. △ Less

Submitted 6 June, 2024; originally announced June 2024.

Comments: Accepted in Advanced Functional Materials

Electric-Field Control of Magnetic Skyrmion Chirality in a Centrosymmetric 2D van der Waals Magnet

Authors: Myung-Geun Han, Joachim Dahl Thomsen, John P. Philbin, Junsik Mun, Eugene Park, Fernando Camino, Lukáš Děkanovský, Chuhang Liu, Zdenek Sofer, Prineha Narang, Frances M. Ross, Yimei Zhu

Abstract: Two-dimensional van der Waals magnets hosting topological magnetic textures, such as skyrmions, show promise for applications in spintronics and quantum computing. Electrical control of these topological spin textures would enable novel devices with enhanced performance and functionality. Here, using electron microscopy combined with in situ electric and magnetic biasing, we show that the skyrmion… ▽ More Two-dimensional van der Waals magnets hosting topological magnetic textures, such as skyrmions, show promise for applications in spintronics and quantum computing. Electrical control of these topological spin textures would enable novel devices with enhanced performance and functionality. Here, using electron microscopy combined with in situ electric and magnetic biasing, we show that the skyrmion chirality, whether left-handed or right-handed, in insulating Cr2Ge2Te6, is controlled by external electric field direction applied during magnetic field cooling process. The electric-field-tuned chirality remains stable, even amid variations in magnetic and electric fields. Our theoretical investigation reveals that nonzero Dzyaloshinskii-Moriya interactions between the nearest neighbors, induced by the external electric field, change their sign upon reversing the electric field direction, thereby facilitating chirality selection. The electrical control of magnetic chirality demonstrated in this study can be extended to other non-metallic centrosymmetric skyrmion-hosting magnets, opening avenues for future device designs in topological spintronics and quantum computing. △ Less

Submitted 2 June, 2024; originally announced June 2024.

InterPreT: Interactive Predicate Learning from Language Feedback for Generalizable Task Planning

Authors: Muzhi Han, Yifeng Zhu, Song-Chun Zhu, Ying Nian Wu, Yuke Zhu

Abstract: Learning abstract state representations and knowledge is crucial for long-horizon robot planning. We present InterPreT, an LLM-powered framework for robots to learn symbolic predicates from language feedback of human non-experts during embodied interaction. The learned predicates provide relational abstractions of the environment state, facilitating the learning of symbolic operators that capture… ▽ More Learning abstract state representations and knowledge is crucial for long-horizon robot planning. We present InterPreT, an LLM-powered framework for robots to learn symbolic predicates from language feedback of human non-experts during embodied interaction. The learned predicates provide relational abstractions of the environment state, facilitating the learning of symbolic operators that capture action preconditions and effects. By compiling the learned predicates and operators into a PDDL domain on-the-fly, InterPreT allows effective planning toward arbitrary in-domain goals using a PDDL planner. In both simulated and real-world robot manipulation domains, we demonstrate that InterPreT reliably uncovers the key predicates and operators governing the environment dynamics. Although learned from simple training tasks, these predicates and operators exhibit strong generalization to novel tasks with significantly higher complexity. In the most challenging generalization setting, InterPreT attains success rates of 73% in simulation and 40% in the real world, substantially outperforming baseline methods. △ Less

Submitted 30 May, 2024; originally announced May 2024.

Comments: RSS 2024; https://interpret-robot.github.io

The BlackGEM telescope array I: Overview

Authors: Paul J. Groot, S. Bloemen, P. Vreeswijk, J. van Roestel, P. G. Jonker, G. Nelemans, M. Klein-Wolt, R. Le Poole, D. Pieterse, M. Rodenhuis, W. Boland, M. Haverkorn, C. Aerts, R. Bakker, H. Balster, M. Bekema, E. Dijkstra, P. Dolron, E. Elswijk, A. van Elteren, A. Engels, M. Fokker, M. de Haan, F. Hahn, R. ter Horst , et al. (49 additional authors not shown)

Abstract: The main science aim of the BlackGEM array is to detect optical counterparts to gravitational wave mergers. Additionally, the array will perform a set of synoptic surveys to detect Local Universe transients and short time-scale variability in stars and binaries, as well as a six-filter all-sky survey down to ~22nd mag. The BlackGEM Phase-I array consists of three optical wide-field unit telescopes… ▽ More The main science aim of the BlackGEM array is to detect optical counterparts to gravitational wave mergers. Additionally, the array will perform a set of synoptic surveys to detect Local Universe transients and short time-scale variability in stars and binaries, as well as a six-filter all-sky survey down to ~22nd mag. The BlackGEM Phase-I array consists of three optical wide-field unit telescopes. Each unit uses an f/5.5 modified Dall-Kirkham (Harmer-Wynne) design with a triplet corrector lens, and a 65cm primary mirror, coupled with a 110Mpix CCD detector, that provides an instantaneous field-of-view of 2.7~square degrees, sampled at 0.564\arcsec/pixel. The total field-of-view for the array is 8.2 square degrees. Each telescope is equipped with a six-slot filter wheel containing an optimised Sloan set (BG-u, BG-g, BG-r, BG-i, BG-z) and a wider-band 440-720 nm (BG-q) filter. Each unit telescope is independent from the others. Cloud-based data processing is done in real time, and includes a transient-detection routine as well as a full-source optimal-photometry module. BlackGEM has been installed at the ESO La Silla observatory as of October 2019. After a prolonged COVID-19 hiatus, science operations started on April 1, 2023 and will run for five years. Aside from its core scientific program, BlackGEM will give rise to a multitude of additional science cases in multi-colour time-domain astronomy, to the benefit of a variety of topics in astrophysics, such as infant supernovae, luminous red novae, asteroseismology of post-main-sequence objects, (ultracompact) binary stars, and the relation between gravitational wave counterparts and other classes of transients △ Less

Submitted 30 May, 2024; v1 submitted 29 May, 2024; originally announced May 2024.

Comments: 14 pages, submitted to Astronomy & Astrophysics

Dishonest Approximate Computing: A Coming Crisis for Cloud Clients

Authors: Ye Wang, Jian Dong, Ming Han, Jin Wu, Gang Qu

Abstract: Approximate Computing (AC) has emerged as a promising technique for achieving energy-efficient architectures and is expected to become an effective technique for reducing the electricity cost for cloud service providers (CSP). However, the potential misuse of AC has not received adequate attention, which is a coming crisis behind the blueprint of AC. Driven by the pursuit of illegal financial prof… ▽ More Approximate Computing (AC) has emerged as a promising technique for achieving energy-efficient architectures and is expected to become an effective technique for reducing the electricity cost for cloud service providers (CSP). However, the potential misuse of AC has not received adequate attention, which is a coming crisis behind the blueprint of AC. Driven by the pursuit of illegal financial profits, untrusted CSPs may deploy low-cost AC devices and deceive clients by presenting AC services as promised accurate computing products, while falsely claiming AC outputs as accurate results. This misuse of AC will cause both financial loss and computing degradation to cloud clients. In this paper, we define this malicious attack as DisHonest Approximate Computing (DHAC) and analyze the technical challenges faced by clients in detecting such attacks. To address this issue, we propose two golden model free detection methods: Residual Class Check (RCC) and Forward-Backward Check (FBC). RCC provides clients a low-cost approach to infer the residual class to which a legitimate accurate output should belong. By comparing the residual class of the returned result, clients can determine whether a computing service contains any AC elements. FBC detects potential DHAC by computing an invertible check branch using the intermediate values of the program. It compares the values before entering and after returning from the check branch to identify any discrepancies. Both RCC and FBC can be executed concurrently with real computing tasks, enabling real-time DHAC detection with current inputs. Our experimental results show that both RCC and FBC can detect over 96%-99% of DHAC cases without misjudging any legitimate accurate results. △ Less

Submitted 24 May, 2024; originally announced May 2024.

Comments: 12 pages, 9 figures

Efficient Economic Model Predictive Control of Water Treatment Process with Learning-based Koopman Operator

Authors: Minghao Han, Jingshi Yao, Adrian Wing-Keung Law, Xunyuan Yin

Abstract: Used water treatment plays a pivotal role in advancing environmental sustainability. Economic model predictive control holds the promise of enhancing the overall operational performance of the water treatment facilities. In this study, we propose a data-driven economic predictive control approach within the Koopman modeling framework. First, we propose a deep learning-enabled input-output Koopman… ▽ More Used water treatment plays a pivotal role in advancing environmental sustainability. Economic model predictive control holds the promise of enhancing the overall operational performance of the water treatment facilities. In this study, we propose a data-driven economic predictive control approach within the Koopman modeling framework. First, we propose a deep learning-enabled input-output Koopman modeling approach, which predicts the overall economic operational cost of the wastewater treatment process based on input data and available output measurements that are directly linked to the operational costs. Subsequently, by leveraging this learned input-output Koopman model, a convex economic predictive control scheme is developed. The resulting predictive control problem can be efficiently solved by leveraging quadratic programming solvers, and complex non-convex optimization problems are bypassed. The proposed method is applied to a benchmark wastewater treatment process. The proposed method significantly improves the overall economic operational performance of the water treatment process. Additionally, the computational efficiency of the proposed method is significantly enhanced as compared to benchmark control solutions. △ Less

Submitted 14 July, 2024; v1 submitted 20 May, 2024; originally announced May 2024.

PARALLELGPUOS: A Concurrent OS-level GPU Checkpoint and Restore System using Validated Speculation

Authors: Zhuobin Huang, Xingda Wei, Yingyi Hao, Rong Chen, Mingcong Han, Jinyu Gu, Haibo Chen

Abstract: Checkpointing (C) and restoring (R) are key components for GPU tasks. POS is an OS-level GPU C/R system: It can transparently checkpoint or restore processes that use the GPU, without requiring any cooperation from the application, a key feature required by modern systems like the cloud. Moreover, POS is the first OS-level C/R system that can concurrently execute C/R with the application execution… ▽ More Checkpointing (C) and restoring (R) are key components for GPU tasks. POS is an OS-level GPU C/R system: It can transparently checkpoint or restore processes that use the GPU, without requiring any cooperation from the application, a key feature required by modern systems like the cloud. Moreover, POS is the first OS-level C/R system that can concurrently execute C/R with the application execution: a critical feature that can be trivially achieved when the processes only running on the CPU, but becomes challenging when the processes use GPU. The problem is how to ensure consistency during concurrent execution with the lack of application semantics due to transparency. CPU processes can leverage OS and hardware paging to fix inconsistency without application semantics. Unfortunately, GPU bypasses OS and paging for high performance. POS fills the semantic gap by speculatively extracting buffer access information of GPU kernels during runtime. Thanks to the simple and well-structured nature of GPU kernels, our speculative extraction (with runtime validation) achieves 100% accuracy on applications from training to inference whose domains span from vision, large language models, and reinforcement learning. Based on the extracted semantics, we systematically overlap C/R with application execution, and achieves orders of magnitude higher performance under various tasks compared with the state-of-the-art OS-level GPU C/R, including training fault tolerance, live GPU process migration, and cold starts acceleration in GPU-based serverless computing. △ Less

Submitted 20 May, 2024; originally announced May 2024.

No-Regret Learning of Nash Equilibrium for Black-Box Games via Gaussian Processes

Authors: Minbiao Han, Fengxue Zhang, Yuxin Chen

Abstract: This paper investigates the challenge of learning in black-box games, where the underlying utility function is unknown to any of the agents. While there is an extensive body of literature on the theoretical analysis of algorithms for computing the Nash equilibrium with complete information about the game, studies on Nash equilibrium in black-box games are less common. In this paper, we focus on le… ▽ More This paper investigates the challenge of learning in black-box games, where the underlying utility function is unknown to any of the agents. While there is an extensive body of literature on the theoretical analysis of algorithms for computing the Nash equilibrium with complete information about the game, studies on Nash equilibrium in black-box games are less common. In this paper, we focus on learning the Nash equilibrium when the only available information about an agent's payoff comes in the form of empirical queries. We provide a no-regret learning algorithm that utilizes Gaussian processes to identify the equilibrium in such games. Our approach not only ensures a theoretical convergence rate but also demonstrates effectiveness across a variety collection of games through experimental validation. △ Less

Submitted 14 May, 2024; originally announced May 2024.

Comments: 40th Conference on Uncertainty in Artificial Intelligence (UAI 2024)

HILCodec: High Fidelity and Lightweight Neural Audio Codec

Authors: Sunghwan Ahn, Beom Jun Woo, Min Hyun Han, Chanyeong Moon, Nam Soo Kim

Abstract: The recent advancement of end-to-end neural audio codecs enables compressing audio at very low bitrates while reconstructing the output audio with high fidelity. Nonetheless, such improvements often come at the cost of increased model complexity. In this paper, we identify and address the problems of existing neural audio codecs. We show that the performance of Wave-U-Net does not increase consist… ▽ More The recent advancement of end-to-end neural audio codecs enables compressing audio at very low bitrates while reconstructing the output audio with high fidelity. Nonetheless, such improvements often come at the cost of increased model complexity. In this paper, we identify and address the problems of existing neural audio codecs. We show that the performance of Wave-U-Net does not increase consistently as the network depth increases. We analyze the root cause of such a phenomenon and suggest a variance-constrained design. Also, we reveal various distortions in previous waveform domain discriminators and propose a novel distortion-free discriminator. The resulting model, \textit{HILCodec}, is a real-time streaming audio codec that demonstrates state-of-the-art quality across various bitrates and audio types. △ Less

Submitted 7 May, 2024; originally announced May 2024.

Multi-Scale Heterogeneity-Aware Hypergraph Representation for Histopathology Whole Slide Images

Authors: Minghao Han, Xukun Zhang, Dingkang Yang, Tao Liu, Haopeng Kuang, Jinghui Feng, Lihua Zhang

Abstract: Survival prediction is a complex ordinal regression task that aims to predict the survival coefficient ranking among a cohort of patients, typically achieved by analyzing patients' whole slide images. Existing deep learning approaches mainly adopt multiple instance learning or graph neural networks under weak supervision. Most of them are unable to uncover the diverse interactions between differen… ▽ More Survival prediction is a complex ordinal regression task that aims to predict the survival coefficient ranking among a cohort of patients, typically achieved by analyzing patients' whole slide images. Existing deep learning approaches mainly adopt multiple instance learning or graph neural networks under weak supervision. Most of them are unable to uncover the diverse interactions between different types of biological entities(\textit{e.g.}, cell cluster and tissue block) across multiple scales, while such interactions are crucial for patient survival prediction. In light of this, we propose a novel multi-scale heterogeneity-aware hypergraph representation framework. Specifically, our framework first constructs a multi-scale heterogeneity-aware hypergraph and assigns each node with its biological entity type. It then mines diverse interactions between nodes on the graph structure to obtain a global representation. Experimental results demonstrate that our method outperforms state-of-the-art approaches on three benchmark datasets. Code is publicly available at \href{https://github.com/Hanminghao/H2GT}{https://github.com/Hanminghao/H2GT}. △ Less

Submitted 30 April, 2024; originally announced April 2024.

Comments: 9 pages, 6 figures, accepted by ICME2024

Ag2Manip: Learning Novel Manipulation Skills with Agent-Agnostic Visual and Action Representations

Authors: Puhao Li, Tengyu Liu, Yuyang Li, Muzhi Han, Haoran Geng, Shu Wang, Yixin Zhu, Song-Chun Zhu, Siyuan Huang

Abstract: Autonomous robotic systems capable of learning novel manipulation tasks are poised to transform industries from manufacturing to service automation. However, modern methods (e.g., VIP and R3M) still face significant hurdles, notably the domain gap among robotic embodiments and the sparsity of successful task executions within specific action spaces, resulting in misaligned and ambiguous task repre… ▽ More Autonomous robotic systems capable of learning novel manipulation tasks are poised to transform industries from manufacturing to service automation. However, modern methods (e.g., VIP and R3M) still face significant hurdles, notably the domain gap among robotic embodiments and the sparsity of successful task executions within specific action spaces, resulting in misaligned and ambiguous task representations. We introduce Ag2Manip (Agent-Agnostic representations for Manipulation), a framework aimed at surmounting these challenges through two key innovations: a novel agent-agnostic visual representation derived from human manipulation videos, with the specifics of embodiments obscured to enhance generalizability; and an agent-agnostic action representation abstracting a robot's kinematics to a universal agent proxy, emphasizing crucial interactions between end-effector and object. Ag2Manip's empirical validation across simulated benchmarks like FrankaKitchen, ManiSkill, and PartManip shows a 325% increase in performance, achieved without domain-specific demonstrations. Ablation studies underline the essential contributions of the visual and action representations to this success. Extending our evaluations to the real world, Ag2Manip significantly improves imitation learning success rates from 50% to 77.5%, demonstrating its effectiveness and generalizability across both simulated and physical environments. △ Less

Submitted 26 April, 2024; originally announced April 2024.

Comments: Project website and open-source code: https://xiaoyao-li.github.io/research/ag2manip

A Mathematica program for numerically computing real and complex critical points in 4-dimensional Lorentzian spinfoam amplitude

Authors: Muxin Han, Hongguang Liu, Dongxue Qu

Abstract: This work develops a comprehensive algorithm and a Mathematica program to construct boundary data and compute real and complex critical points in spinfoam amplitudes. Our approach covers both spacelike tetrahedra and triangles in the EPRL model and timelike tetrahedra and triangles in the Conrady-Hnybida extension, aiming at addressing a wide range of physical scenarios such as cosmology and black… ▽ More This work develops a comprehensive algorithm and a Mathematica program to construct boundary data and compute real and complex critical points in spinfoam amplitudes. Our approach covers both spacelike tetrahedra and triangles in the EPRL model and timelike tetrahedra and triangles in the Conrady-Hnybida extension, aiming at addressing a wide range of physical scenarios such as cosmology and black holes. Starting with a single 4-simplex, we explain how to numerically construct boundary data and corresponding real critical points from any nondegenerate 4-simplex geometry. Extending this to the simplicial complex, we demonstrate the algorithm for constructing boundary data and critical points using examples with two 4-simplices sharing an internal tetrahedron. By revisiting the $Δ_3$ triangulation with curved geometry, we demonstrate the numerical computation of the real critical point corresponding to the flat geometry and the deformation to the complex critical points. Additionally, the program evaluates the spinfoam action at the critical points and compare to the Regge action. △ Less

Submitted 16 April, 2024; originally announced April 2024.

Comments: 34 pages, 36 figures

Improving Bracket Image Restoration and Enhancement with Flow-guided Alignment and Enhanced Feature Aggregation

Authors: Wenjie Lin, Zhen Liu, Chengzhi Jiang, Mingyan Han, Ting Jiang, Shuaicheng Liu

Abstract: In this paper, we address the Bracket Image Restoration and Enhancement (BracketIRE) task using a novel framework, which requires restoring a high-quality high dynamic range (HDR) image from a sequence of noisy, blurred, and low dynamic range (LDR) multi-exposure RAW inputs. To overcome this challenge, we present the IREANet, which improves the multiple exposure alignment and aggregation with a Fl… ▽ More In this paper, we address the Bracket Image Restoration and Enhancement (BracketIRE) task using a novel framework, which requires restoring a high-quality high dynamic range (HDR) image from a sequence of noisy, blurred, and low dynamic range (LDR) multi-exposure RAW inputs. To overcome this challenge, we present the IREANet, which improves the multiple exposure alignment and aggregation with a Flow-guide Feature Alignment Module (FFAM) and an Enhanced Feature Aggregation Module (EFAM). Specifically, the proposed FFAM incorporates the inter-frame optical flow as guidance to facilitate the deformable alignment and spatial attention modules for better feature alignment. The EFAM further employs the proposed Enhanced Residual Block (ERB) as a foundational component, wherein a unidirectional recurrent network aggregates the aligned temporal features to better reconstruct the results. To improve model generalization and performance, we additionally employ the Bayer preserving augmentation (BayerAug) strategy to augment the multi-exposure RAW inputs. Our experimental evaluations demonstrate that the proposed IREANet shows state-of-the-art performance compared with previous methods. △ Less

Submitted 16 April, 2024; originally announced April 2024.

Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4V

Authors: Peiyuan Zhi, Zhiyuan Zhang, Muzhi Han, Zeyu Zhang, Zhitian Li, Ziyuan Jiao, Baoxiong Jia, Siyuan Huang

Abstract: Autonomous robot navigation and manipulation in open environments require reasoning and replanning with closed-loop feedback. We present COME-robot, the first closed-loop framework utilizing the GPT-4V vision-language foundation model for open-ended reasoning and adaptive planning in real-world scenarios. We meticulously construct a library of action primitives for robot exploration, navigation, a… ▽ More Autonomous robot navigation and manipulation in open environments require reasoning and replanning with closed-loop feedback. We present COME-robot, the first closed-loop framework utilizing the GPT-4V vision-language foundation model for open-ended reasoning and adaptive planning in real-world scenarios. We meticulously construct a library of action primitives for robot exploration, navigation, and manipulation, serving as callable execution modules for GPT-4V in task planning. On top of these modules, GPT-4V serves as the brain that can accomplish multimodal reasoning, generate action policy with code, verify the task progress, and provide feedback for replanning. Such design enables COME-robot to (i) actively perceive the environments, (ii) perform situated reasoning, and (iii) recover from failures. Through comprehensive experiments involving 8 challenging real-world tabletop and manipulation tasks, COME-robot demonstrates a significant improvement in task success rate (~25%) compared to state-of-the-art baseline methods. We further conduct comprehensive analyses to elucidate how COME-robot's design facilitates failure recovery, free-form instruction following, and long-horizon task planning. △ Less

Submitted 15 April, 2024; originally announced April 2024.

Ultra-Wide Dual-band Rydberg Atomic Receiver Based on Space Division Multiplexing RF-Chip Modules

Authors: Li-Hua Zhang, Bang Liu, Zong-Kai Liu, Zheng-Yuan Zhang, Shi-Yao Shao, Qi-Feng Wang, Ma YuTian-Yu Han, Guang-Can Guo, Dong-Sheng Ding, Bao-Sen Shi

Abstract: Detecting microwave signals over a wide frequency range has numerous advantages as it enables simultaneous transmission of a large amount of information and access to more spectrum resources. This capability is crucial for applications such as microwave communication, remote sensing, and radar. However, conventional microwave receiving systems are limited by amplifiers and band-pass filters that c… ▽ More Detecting microwave signals over a wide frequency range has numerous advantages as it enables simultaneous transmission of a large amount of information and access to more spectrum resources. This capability is crucial for applications such as microwave communication, remote sensing, and radar. However, conventional microwave receiving systems are limited by amplifiers and band-pass filters that can only operate efficiently in a specific frequency range. Typically, these systems can only process signals within a three-fold frequency range, which limits the data transfer bandwidth of the microwave communication systems. Developing novel atom-integrated microwave sensors, for example, radio frequency (RF)-chip coupled Rydberg atomic receiver, provides opportunities for a large working bandwidth of microwave sensing at the atomic level. Here, an ultra-wide dual-band RF sensing scheme is demonstrated by space-division multiplexing two RF-chip-integrated atomic receiver modules. The system can simultaneously receive dual-band microwave signals that span a frequency range exceeding 6 octaves (300 MHz and 24 GHz). This work paves the way for multi-band microwave reception applications within an ultra-wide range by RF-chip-integrated Rydberg atomic sensor. △ Less

Submitted 16 April, 2024; v1 submitted 15 April, 2024; originally announced April 2024.

Comments: 11 pages, 5 figures

Upper Limit of Sound Speed in Nuclear Matter: A Harmonious Interplay of Transport Calculation and Perturbative QCD Constraint

Authors: Shao-Peng Tang, Yong-Jia Huang, Ming-Zhe Han, Yi-Zhong Fan

Abstract: Very recently, it has been shown that there is an upper bound on the squared sound speed of nuclear matter from the transport, which reads $c_{\rm s}^2 \leq 0.781$. In this work, we demonstrate that this upper bound is corroborated by the reconstructed equation of state (EOS; modeled with a nonparametric method) for ultra-dense matter. The reconstruction integrates multi-messenger observation for… ▽ More Very recently, it has been shown that there is an upper bound on the squared sound speed of nuclear matter from the transport, which reads $c_{\rm s}^2 \leq 0.781$. In this work, we demonstrate that this upper bound is corroborated by the reconstructed equation of state (EOS; modeled with a nonparametric method) for ultra-dense matter. The reconstruction integrates multi-messenger observation for neutron stars (NSs), in particular, the latest radius measurements for PSR J0437-4715 ($11.36^{+0.95}_{-0.63}$ km), PSR J0030+0451 ($11.71^{+0.88}_{-0.83}$ km, in the ST+PDT model), and PSR J0740+6620 ($12.49^{+1.28}_{-0.88}$ km) by NICER have been adopted. The result shows in all cases, the $c_{\rm s}^2 \leq 0.781$ upper limit for EOS will naturally yield the properties of matter near center of the massive neutron star consistent with the causality-driven constraint from pQCD, where in practice, the constraint is applied at ten nuclear saturation density ($n_{\rm L}=10n_{\rm s}$). We also note that there is a strong correlation for the maximum $c_s^2$ with $n_{\rm L}$, and $c_{\rm s}^2 \leq 0.781$ is somehow violated when $n_{\rm L} = n_{\rm c,TOV}$. The result indicates that a higher density in implementing the pQCD constraint, even considering the uncertainties from statistics, is more natural. Moreover, the remarkable agreement between the outcomes derived from these two distinct and independent constraints (i.e., the transport calculation and pQCD boundary) lends strong support to their validity. Besides, the latest joint constraint for $R_{1.4}$, $R_{2.0}$, $R_{1.4}-R_{2.0}$, and $M_{\rm TOV}$ are $11.94_{-0.68}^{+0.77}$ km, $11.99_{-0.67}^{+0.88}$ km, $-0.1_{-0.27}^{+0.42}$ km, and $2.24_{-0.10}^{+0.13}M_\odot$ (at $90\%$ credible level), respectively. △ Less

Submitted 22 July, 2024; v1 submitted 15 April, 2024; originally announced April 2024.

Comments: 9 pages, 9 figures, updated with the most recent data release from NICER

Report number: RIKEN-iTHEMS-Report-24

LongVLM: Efficient Long Video Understanding via Large Language Models

Authors: Yuetian Weng, Mingfei Han, Haoyu He, Xiaojun Chang, Bohan Zhuang

Abstract: Empowered by Large Language Models (LLMs), recent advancements in Video-based LLMs (VideoLLMs) have driven progress in various video understanding tasks. These models encode video representations through pooling or query aggregation over a vast number of visual tokens, making computational and memory costs affordable. Despite successfully providing an overall comprehension of video content, existi… ▽ More Empowered by Large Language Models (LLMs), recent advancements in Video-based LLMs (VideoLLMs) have driven progress in various video understanding tasks. These models encode video representations through pooling or query aggregation over a vast number of visual tokens, making computational and memory costs affordable. Despite successfully providing an overall comprehension of video content, existing VideoLLMs still face challenges in achieving detailed understanding due to overlooking local information in long-term videos. To tackle this challenge, we introduce LongVLM, a simple yet powerful VideoLLM for long video understanding, building upon the observation that long videos often consist of sequential key events, complex actions, and camera movements. Our approach proposes to decompose long videos into multiple short-term segments and encode local features for each segment via a hierarchical token merging module. These features are concatenated in temporal order to maintain the storyline across sequential short-term segments. Additionally, we propose to integrate global semantics into each local feature to enhance context understanding. In this way, we encode video representations that incorporate both local and global information, enabling the LLM to generate comprehensive responses for long-term videos. Experimental results on the VideoChatGPT benchmark and zero-shot video question-answering datasets demonstrate the superior capabilities of our model over the previous state-of-the-art methods. Qualitative examples show that our model produces more precise responses for long video understanding. Code is available at https://github.com/ziplab/LongVLM. △ Less

Submitted 20 July, 2024; v1 submitted 4 April, 2024; originally announced April 2024.

Comments: Accepted by ECCV 2024

Site-specific Deterministic Temperature and Humidity Forecasts with Explainable and Reliable Machine Learning

Authors: MengMeng Han, Tennessee Leeuwenburg, Brad Murphy

Abstract: Site-specific weather forecasts are essential to accurate prediction of power demand and are consequently of great interest to energy operators. However, weather forecasts from current numerical weather prediction (NWP) models lack the fine-scale detail to capture all important characteristics of localised real-world sites. Instead they provide weather information representing a rectangular gridbo… ▽ More Site-specific weather forecasts are essential to accurate prediction of power demand and are consequently of great interest to energy operators. However, weather forecasts from current numerical weather prediction (NWP) models lack the fine-scale detail to capture all important characteristics of localised real-world sites. Instead they provide weather information representing a rectangular gridbox (usually kilometres in size). Even after post-processing and bias correction, area-averaged information is usually not optimal for specific sites. Prior work on site optimised forecasts has focused on linear methods, weighted consensus averaging, time-series methods, and others. Recent developments in machine learning (ML) have prompted increasing interest in applying ML as a novel approach towards this problem. In this study, we investigate the feasibility of optimising forecasts at sites by adopting the popular machine learning model gradient boosting decision tree, supported by the Python version of the XGBoost package. Regression trees have been trained with historical NWP and site observations as training data, aimed at predicting temperature and dew point at multiple site locations across Australia. We developed a working ML framework, named 'Multi-SiteBoost' and initial testing results show a significant improvement compared with gridded values from bias-corrected NWP models. The improvement from XGBoost is found to be comparable with non-ML methods reported in literature. With the insights provided by SHapley Additive exPlanations (SHAP), this study also tests various approaches to understand the ML predictions and increase the reliability of the forecasts generated by ML. △ Less

Submitted 4 April, 2024; originally announced April 2024.

Comments: 27 Pages, 16 Figures, 11 Tables

Journal ref: Applied Sciences, Volume 14, Issue 14, Article Number 6314, July 2024

Spin foam amplitude of the black-to-white hole transition

Authors: Muxin Han, Dongxue Qu, Cong Zhang

Abstract: It has been conjectured that quantum gravity effects may cause the black-to-white hole transition due to quantum tunneling. The transition amplitude of this process is explored within the framework of the spin foam model on a 2-complex containing 56 vertices. We develop a systematic way to construct the bulk triangulation from the boundary triangulation to obtain the 2-complex. By using Thiemann's… ▽ More It has been conjectured that quantum gravity effects may cause the black-to-white hole transition due to quantum tunneling. The transition amplitude of this process is explored within the framework of the spin foam model on a 2-complex containing 56 vertices. We develop a systematic way to construct the bulk triangulation from the boundary triangulation to obtain the 2-complex. By using Thiemann's complexifier coherent state as the boundary state to resemble the semiclassical geometry, we introduce a procedure to calculate the parameters labeling the coherent state from the continuous curved geometry. Considering that triad fields of different orientations, i.e., $e_i^a$ and $-e_i^a$, give the same intrinsic geometry of the boundary, we creatively adopt the boundary state as a superposition of the coherent states associated with both orientations. We employ the method of complex critical point to numerically compute the transition amplitude. Despite the numerical results, it is interestingly found that the transition amplitude is dominated by the terms allowing the change in orientation. This suggests that the black-to-white hole transition should be accompanied by quantum tunneling process of a change in orientation. △ Less

Submitted 3 April, 2024; originally announced April 2024.

Comments: 17+15 Pages

Inversion and Tunability of Van Hove Singularities in $A$V$_{3}$Sb$_{5}$ ($A$ = K, Rb, and Cs) kagome metals

Authors: Sangjun Sim, Min Yong Jeong, Hyunggeun Lee, Dong Hyun David Lee, Myung Joon Han

Abstract: To understand the alkali-metal-dependent material properties of recently discovered $A$V$_{3}$Sb$_{5}$ ($A$ = K, Rb, and Cs), we conducted a detailed electronic structure analysis based on first-principles density functional theory calculations. Contrary to the case of $A$ = K and Rb, the energetic positions of the low-lying Van Hove singularities are reversed in CsV$_{3}$Sb$_{5}$, and the charact… ▽ More To understand the alkali-metal-dependent material properties of recently discovered $A$V$_{3}$Sb$_{5}$ ($A$ = K, Rb, and Cs), we conducted a detailed electronic structure analysis based on first-principles density functional theory calculations. Contrary to the case of $A$ = K and Rb, the energetic positions of the low-lying Van Hove singularities are reversed in CsV$_{3}$Sb$_{5}$, and the characteristic higher-order Van Hove point gets closer to the Fermi level. We found that this notable difference can be attributed to the chemical effect, apart from structural differences. Due to their different orbital compositions, Van Hove points show qualitatively different responses to the structure changes. A previously unnoticed highest lying point can be lowered, locating close to or even below the other ones in response to a reasonable range of bi- and uni-axial strain. Our results can be useful in better understanding the material-dependent features reported in this family and in realizing experimental control of exotic quantum phases. △ Less

Submitted 1 April, 2024; originally announced April 2024.

Comments: Physical Chemistry Chemical Physics (PCCP) in press

Reduced-order Koopman modeling and predictive control of nonlinear processes

Authors: Xuewen Zhang, Minghao Han, Xunyuan Yin

Abstract: In this paper, we propose an efficient data-driven predictive control approach for general nonlinear processes based on a reduced-order Koopman operator. A Kalman-based sparse identification of nonlinear dynamics method is employed to select lifting functions for Koopman identification. The selected lifting functions are used to project the original nonlinear state-space into a higher-dimensional… ▽ More In this paper, we propose an efficient data-driven predictive control approach for general nonlinear processes based on a reduced-order Koopman operator. A Kalman-based sparse identification of nonlinear dynamics method is employed to select lifting functions for Koopman identification. The selected lifting functions are used to project the original nonlinear state-space into a higher-dimensional linear function space, in which Koopman-based linear models can be constructed for the underlying nonlinear process. To curb the significant increase in the dimensionality of the resulting full-order Koopman models caused by the use of lifting functions, we propose a reduced-order Koopman modeling approach based on proper orthogonal decomposition. A computationally efficient linear robust predictive control scheme is established based on the reduced-order Koopman model. A case study on a benchmark chemical process is conducted to illustrate the effectiveness of the proposed method. Comprehensive comparisons are conducted to demonstrate the advantage of the proposed method. △ Less

Submitted 30 March, 2024; originally announced April 2024.

Comments: 29 pages, 8 figures

Journal ref: Computers & Chemical Engineering, 2023, 179, p.108440

Towards 3D Vision with Low-Cost Single-Photon Cameras

Authors: Fangzhou Mu, Carter Sifferman, Sacha Jungerman, Yiquan Li, Mark Han, Michael Gleicher, Mohit Gupta, Yin Li

Abstract: We present a method for reconstructing 3D shape of arbitrary Lambertian objects based on measurements by miniature, energy-efficient, low-cost single-photon cameras. These cameras, operating as time resolved image sensors, illuminate the scene with a very fast pulse of diffuse light and record the shape of that pulse as it returns back from the scene at a high temporal resolution. We propose to mo… ▽ More We present a method for reconstructing 3D shape of arbitrary Lambertian objects based on measurements by miniature, energy-efficient, low-cost single-photon cameras. These cameras, operating as time resolved image sensors, illuminate the scene with a very fast pulse of diffuse light and record the shape of that pulse as it returns back from the scene at a high temporal resolution. We propose to model this image formation process, account for its non-idealities, and adapt neural rendering to reconstruct 3D geometry from a set of spatially distributed sensors with known poses. We show that our approach can successfully recover complex 3D shapes from simulated data. We further demonstrate 3D object reconstruction from real-world captures, utilizing measurements from a commodity proximity sensor. Our work draws a connection between image-based modeling and active range scanning and is a step towards 3D vision with single-photon cameras. △ Less

Submitted 29 March, 2024; v1 submitted 26 March, 2024; originally announced March 2024.

Bulk properties of PSR J0030+0451 inferred with the compactness measurement of NICER

Authors: Chuan-Ning Luo, Shao-Peng Tang, Ming-Zhe Han, Jin-Liang Jiang, Wei-Hong Gao, Da-Ming Wei

Abstract: In 2019, Neutron star Interior Composition ExploreR (NICER) mission released its findings on the mass and radius of the isolated neutron star (INS) PSR J0030+0451, revealing a mass of approximately 1.4 solar masses ($M_{\odot}$) and a radius near 13 kilometers. However, the recent re-analysis by the NICER collaboration \citep{vinciguerra2024updated} suggests that the available data primarily yield… ▽ More In 2019, Neutron star Interior Composition ExploreR (NICER) mission released its findings on the mass and radius of the isolated neutron star (INS) PSR J0030+0451, revealing a mass of approximately 1.4 solar masses ($M_{\odot}$) and a radius near 13 kilometers. However, the recent re-analysis by the NICER collaboration \citep{vinciguerra2024updated} suggests that the available data primarily yields a precise inference of the compactness for this source while the resulting mass and radius are strongly model-dependent and diverse (the 68.3\% credible regions just overlap slightly for the ST+PDT and PDT-U models). By integrating this compactness data with the equation of state (EoS) refined by our latest investigations, we have deduced the mass and radius for PSR J0030+0451, delivering estimates of $M=1.48^{+0.09}_{-0.10}~M_\odot$ and $R=12.38_{-0.70}^{+0.51}~{\rm km}$ for the compactness found in ST+PDT model, alongside $M=1.47^{+0.14}_{-0.20}~M_\odot$ and $R=12.37_{-0.69}^{+0.50}~{\rm km}$ for the compactness in PDT-U model. These two groups of results are well consistent with each other and the direct X-ray data inference within the ST+PDT model seems to be favored. Additionally, we have calculated the tidal deformability, moment of inertia, and gravitational binding energy for this NS. Furthermore, employing these refined EoS models, we have updated mass-radius estimates for three INSs with established gravitational redshifts. △ Less

Submitted 2 April, 2024; v1 submitted 20 March, 2024; originally announced March 2024.

Comments: 10 pages,6 figures, Accepted for publication in ApJ

LLM3:Large Language Model-based Task and Motion Planning with Motion Failure Reasoning

Authors: Shu Wang, Muzhi Han, Ziyuan Jiao, Zeyu Zhang, Ying Nian Wu, Song-Chun Zhu, Hangxin Liu

Abstract: Conventional Task and Motion Planning (TAMP) approaches rely on manually crafted interfaces connecting symbolic task planning with continuous motion generation. These domain-specific and labor-intensive modules are limited in addressing emerging tasks in real-world settings. Here, we present LLM^3, a novel Large Language Model (LLM)-based TAMP framework featuring a domain-independent interface. Sp… ▽ More Conventional Task and Motion Planning (TAMP) approaches rely on manually crafted interfaces connecting symbolic task planning with continuous motion generation. These domain-specific and labor-intensive modules are limited in addressing emerging tasks in real-world settings. Here, we present LLM^3, a novel Large Language Model (LLM)-based TAMP framework featuring a domain-independent interface. Specifically, we leverage the powerful reasoning and planning capabilities of pre-trained LLMs to propose symbolic action sequences and select continuous action parameters for motion planning. Crucially, LLM^3 incorporates motion planning feedback through prompting, allowing the LLM to iteratively refine its proposals by reasoning about motion failure. Consequently, LLM^3 interfaces between task planning and motion planning, alleviating the intricate design process of handling domain-specific messages between them. Through a series of simulations in a box-packing domain, we quantitatively demonstrate the effectiveness of LLM^3 in solving TAMP problems and the efficiency in selecting action parameters. Ablation studies underscore the significant contribution of motion failure reasoning to the success of LLM^3. Furthermore, we conduct qualitative experiments on a physical manipulator, demonstrating the practical applicability of our approach in real-world settings. △ Less

Submitted 21 August, 2024; v1 submitted 18 March, 2024; originally announced March 2024.

Comments: IROS 2024. Codes available: https://github.com/AssassinWS/LLM-TAMP

Debatrix: Multi-dimensional Debate Judge with Iterative Chronological Analysis Based on LLM

Authors: Jingcong Liang, Rong Ye, Meng Han, Ruofei Lai, Xinyu Zhang, Xuanjing Huang, Zhongyu Wei

Abstract: How can we construct an automated debate judge to evaluate an extensive, vibrant, multi-turn debate? This task is challenging, as judging a debate involves grappling with lengthy texts, intricate argument relationships, and multi-dimensional assessments. At the same time, current research mainly focuses on short dialogues, rarely touching upon the evaluation of an entire debate. In this paper, by… ▽ More How can we construct an automated debate judge to evaluate an extensive, vibrant, multi-turn debate? This task is challenging, as judging a debate involves grappling with lengthy texts, intricate argument relationships, and multi-dimensional assessments. At the same time, current research mainly focuses on short dialogues, rarely touching upon the evaluation of an entire debate. In this paper, by leveraging Large Language Models (LLMs), we propose Debatrix, which makes the analysis and assessment of multi-turn debates more aligned with majority preferences. Specifically, Debatrix features a vertical, iterative chronological analysis and a horizontal, multi-dimensional evaluation collaboration. To align with real-world debate scenarios, we introduced the PanelBench benchmark, comparing our system's performance to actual debate outcomes. The findings indicate a notable enhancement over directly using LLMs for debate evaluation. Source code and benchmark data are available online at https://github.com/ljcleo/debatrix . △ Less

Submitted 19 June, 2024; v1 submitted 12 March, 2024; originally announced March 2024.

Quantum Zeno Monte Carlo for computing observables

Authors: Mancheon Han, Hyowon Park, Sangkook Choi

Abstract: The recent development of logical quantum processors signifies a pivotal moment in the progression from the noisy intermediate-scale quantum (NISQ) era to the fault-tolerant quantum computing (FTQC) era. These advanced devices are poised to alter the approach to problems that challenge classical computation methods. By transforming such problems into Hamiltonian frameworks and exploiting quantum m… ▽ More The recent development of logical quantum processors signifies a pivotal moment in the progression from the noisy intermediate-scale quantum (NISQ) era to the fault-tolerant quantum computing (FTQC) era. These advanced devices are poised to alter the approach to problems that challenge classical computation methods. By transforming such problems into Hamiltonian frameworks and exploiting quantum mechanical properties, these processors have the potential to address complex issues within a polynomial computational time. However, despite their advancements, these processors remain vulnerable to disruptive noise, highlighting the need for robust quantum algorithms designed to manage noise effectively. In response to this need, we introduce a new classical-quantum hybrid algorithm termed Quantum Zeno Monte Carlo (QZMC). QZMC is capable of handling device noises and Trotter errors while demonstrating polynomial computational cost. This algorithm combines the quantum Zeno effect with Monte Carlo integration techniques, facilitating multi-step transitions toward targeted eigenstates of the Hamiltonian problem. Notably, QZMC does not require overlap between the initial state and the target state, nor does it depend on variational parameters. It can compute static and dynamic properties of the targeted states, including ground state energy, excited state energies, and Green's functions. Compared to quantum phase estimation, QZMC offers a significantly reduced quantum circuit depth. These features make QZMC an important algorithm for navigating the current transitional phase in quantum computing and beyond. △ Less

Submitted 9 May, 2024; v1 submitted 5 March, 2024; originally announced March 2024.

Comments: 15 figures

DiffMOT: A Real-time Diffusion-based Multiple Object Tracker with Non-linear Prediction

Authors: Weiyi Lv, Yuhang Huang, Ning Zhang, Ruei-Sung Lin, Mei Han, Dan Zeng

Abstract: In Multiple Object Tracking, objects often exhibit non-linear motion of acceleration and deceleration, with irregular direction changes. Tacking-by-detection (TBD) trackers with Kalman Filter motion prediction work well in pedestrian-dominant scenarios but fall short in complex situations when multiple objects perform non-linear and diverse motion simultaneously. To tackle the complex non-linear m… ▽ More In Multiple Object Tracking, objects often exhibit non-linear motion of acceleration and deceleration, with irregular direction changes. Tacking-by-detection (TBD) trackers with Kalman Filter motion prediction work well in pedestrian-dominant scenarios but fall short in complex situations when multiple objects perform non-linear and diverse motion simultaneously. To tackle the complex non-linear motion, we propose a real-time diffusion-based MOT approach named DiffMOT. Specifically, for the motion predictor component, we propose a novel Decoupled Diffusion-based Motion Predictor (D$^2$MP). It models the entire distribution of various motion presented by the data as a whole. It also predicts an individual object's motion conditioning on an individual's historical motion information. Furthermore, it optimizes the diffusion process with much fewer sampling steps. As a MOT tracker, the DiffMOT is real-time at 22.7FPS, and also outperforms the state-of-the-art on DanceTrack and SportsMOT datasets with $62.3\%$ and $76.2\%$ in HOTA metrics, respectively. To the best of our knowledge, DiffMOT is the first to introduce a diffusion probabilistic model into the MOT to tackle non-linear motion prediction. △ Less

Submitted 20 March, 2024; v1 submitted 4 March, 2024; originally announced March 2024.

Comments: CVPR2024

Robust deep labeling of radiological emphysema subtypes using squeeze and excitation convolutional neural networks: The MESA Lung and SPIROMICS Studies

Authors: Artur Wysoczanski, Nabil Ettehadi, Soroush Arabshahi, Yifei Sun, Karen Hinkley Stukovsky, Karol E. Watson, MeiLan K. Han, Erin D Michos, Alejandro P. Comellas, Eric A. Hoffman, Andrew F. Laine, R. Graham Barr, Elsa D. Angelini

Abstract: Pulmonary emphysema, the progressive, irreversible loss of lung tissue, is conventionally categorized into three subtypes identifiable on pathology and on lung computed tomography (CT) images. Recent work has led to the unsupervised learning of ten spatially-informed lung texture patterns (sLTPs) on lung CT, representing distinct patterns of emphysematous lung parenchyma based on both textural app… ▽ More Pulmonary emphysema, the progressive, irreversible loss of lung tissue, is conventionally categorized into three subtypes identifiable on pathology and on lung computed tomography (CT) images. Recent work has led to the unsupervised learning of ten spatially-informed lung texture patterns (sLTPs) on lung CT, representing distinct patterns of emphysematous lung parenchyma based on both textural appearance and spatial location within the lung, and which aggregate into 6 robust and reproducible CT Emphysema Subtypes (CTES). Existing methods for sLTP segmentation, however, are slow and highly sensitive to changes in CT acquisition protocol. In this work, we present a robust 3-D squeeze-and-excitation CNN for supervised classification of sLTPs and CTES on lung CT. Our results demonstrate that this model achieves accurate and reproducible sLTP segmentation on lung CTscans, across two independent cohorts and independently of scanner manufacturer and model. △ Less

Submitted 29 February, 2024; originally announced March 2024.

Attosecond X-ray Chronoscopy of Core-level Photoemission

Authors: Jia-Bao Ji, Zhaoheng Guo, Taran Driver, Cynthia S. Trevisan, David Cesar, Xinxin Cheng, Joseph Duris, Paris L. Franz, James Glownia, Xiaochun Gong, Daniel Hammerland, Meng Han, Saijoscha Heck, Matthias Hoffmann, Andrei Kamalov, Kirk A. Larsen, Xiang Li, Ming-Fu Lin, Yuchen Liu, C. William McCurdy, Razib Obaid, Jordan T. ONeal, Thomas N. Rescigno, River R. Robles, Nicholas Sudar , et al. (10 additional authors not shown)

Abstract: Attosecond photoemission or photoionization delays are a unique probe of the structure and the electronic dynamics of matter. However, spectral congestion and spatial delocalization of valence electron wave functions set fundamental limits to the complexity of systems that can be studied and the information that can be retrieved, respectively. Using attosecond X-ray pulses from LCLS, we demonstrat… ▽ More Attosecond photoemission or photoionization delays are a unique probe of the structure and the electronic dynamics of matter. However, spectral congestion and spatial delocalization of valence electron wave functions set fundamental limits to the complexity of systems that can be studied and the information that can be retrieved, respectively. Using attosecond X-ray pulses from LCLS, we demonstrate the key advantages of measuring core-level delays: the photoelectron spectra remain atom-like, the measurements become element specific and the observed scattering dynamics originate from a point-like source. We exploit these unique features to reveal the effects of electronegativity and symmetry on attosecond scattering dynamics by measuring the photoionization delays between N-1s and C-1s core shells of a series of aromatic azabenzene molecules. Remarkably, the delays systematically increase with the number of nitrogen atoms in the molecule and reveal multiple resonances. We identify two previously unknown mechanisms regulating the associated attosecond dynamics, namely the enhanced confinement of the trapped wavefunction with increasing electronegativity of the atoms and the decrease of the coupling strength among the photoemitted partial waves with increasing symmetry. This study demonstrates the unique opportunities opened by measurements of core-level photoionization delays for unravelling attosecond electron dynamics in complex matter. △ Less

Submitted 8 April, 2024; v1 submitted 27 February, 2024; originally announced February 2024.

M-ideals of compact operators and Norm attaining operators

Authors: Manwook Han, Sun Kwang Kim

Abstract: We investigate M-ideals of compact operators and two distinct properties in norm-attaining operator theory related with M-ideals of compact operators called the weak maximizing property and the compact perturbation property. For Banach spaces $X$ and $Y$, it is previously known that if $\mathcal{K}(X,Y)$ is an M-ideal or $(X,Y)$ has the weak maximizing property, then $(X,Y)$ has the adjoint compac… ▽ More We investigate M-ideals of compact operators and two distinct properties in norm-attaining operator theory related with M-ideals of compact operators called the weak maximizing property and the compact perturbation property. For Banach spaces $X$ and $Y$, it is previously known that if $\mathcal{K}(X,Y)$ is an M-ideal or $(X,Y)$ has the weak maximizing property, then $(X,Y)$ has the adjoint compact perturbation property. We see that their converses are not true, and the condition that $\mathcal{K}(X,Y)$ is an M-ideal does not imply the weak maximizing property, nor vice versa. Nevertheless, we see that all of these are closely related to property $(M)$, and as a consequence, we show that if $\mathcal{K}(\ell_p,Y)$ $(1<p<\infty)$ is an M-ideal, then $(\ell_p,Y)$ has the weak maximizing property. We also prove that $(\ell_1,\ell_1)$ does not have the adjoint compact perturbation property, and neither does $(\ell_1,Y)$ for an infinite dimensional Banach space $Y$ without an isomorphic copy of $\ell_1$ if $Y$ does not have the local diameter 2 property. As a consequence, we show that if $Y$ is an infinite dimensional Banach space such that $\mathcal{L}(\ell_1,Y)$ is an M-ideal, then it has the local diameter 2 property. Furthermore, we also studied various geometric properties of Banach spaces such as the Opial property with moduli of asymptotic uniform smoothness and uniform convexity. △ Less

Submitted 19 February, 2024; originally announced February 2024.

Comments: 25 pages

Representations of a quantum-deformed Lorentz algebra, Clebsch-Gordan map, and Fenchel-Nielsen representation of quantum complex flat connections at level-$k$

Authors: Muxin Han

Abstract: A family of infinite-dimensional irreducible $\star$-representations on $\mathcal{H}\simeq L^2(\mathbb{R})\otimes\mathbb{C}^k$ is defined for a quantum-deformed Lorentz algebra $U_\mathbf{q}(sl_2)\otimes U_{\tilde{\mathbf{q}}}(sl_2)$, where $\mathbf{q}=\exp[\frac{2πi}{k}(1+b^2)]$ and $\tilde{\mathbf{q}}=\exp[\frac{2πi}{k}(1+b^{-2})]$ with $k\in\mathbb{Z}_+$ and $|b|=1$. The representations are con… ▽ More A family of infinite-dimensional irreducible $\star$-representations on $\mathcal{H}\simeq L^2(\mathbb{R})\otimes\mathbb{C}^k$ is defined for a quantum-deformed Lorentz algebra $U_\mathbf{q}(sl_2)\otimes U_{\tilde{\mathbf{q}}}(sl_2)$, where $\mathbf{q}=\exp[\frac{2πi}{k}(1+b^2)]$ and $\tilde{\mathbf{q}}=\exp[\frac{2πi}{k}(1+b^{-2})]$ with $k\in\mathbb{Z}_+$ and $|b|=1$. The representations are constructed with the irreducible representation of quantum torus algebra at level-$k$, which is developed from the quantization of $\mathrm{SL}(2,\mathbb{C})$ Chern-Simons theory. We study the Clebsch-Gordan decomposition of the tensor product representation, and we show that it reduces to the same problem as diagonalizing the complex Fenchel-Nielson length operators in quantizing $\mathrm{SL}(2,\mathbb{C})$ flat connections on 4-holed sphere. Finally, the spectral decomposition of the complex Fenchel-Nielson length operators results in the direct-integral representation of the Hilbert space $\mathcal{H}$, which we call the Fenchel-Nielson representation. △ Less

Submitted 12 February, 2024; originally announced February 2024.

Comments: 28 pages, 4 figures