Showing 1–50 of 1,085 results for author: Zhao, T

$L^p$ maximal estimates for Weyl sums with $k\ge3$ on $\mathbb{T}$

Authors: Xuezhi Chen, Changxing Miao, Jiye Yuan, Tengfei Zhao

Abstract: In this paper, we study the $L^p$ maximal estimates for the Weyl sums $\sum_{n=1}^{N}e^{2πi(nx + n^{k}t)}$ with higher-order $k\ge3$ on $\mathbb{T}$, and obtain the positive and negative results. Especially for the case $k=3$, our result is sharp up to the endpoint. The main idea is to investigate the structure of the set where large values of Weyl sums are achieved by making use of the rational a… ▽ More In this paper, we study the $L^p$ maximal estimates for the Weyl sums $\sum_{n=1}^{N}e^{2πi(nx + n^{k}t)}$ with higher-order $k\ge3$ on $\mathbb{T}$, and obtain the positive and negative results. Especially for the case $k=3$, our result is sharp up to the endpoint. The main idea is to investigate the structure of the set where large values of Weyl sums are achieved by making use of the rational approximation and the refined estimate for the exponential sums. △ Less

Submitted 28 August, 2024; originally announced August 2024.

Comments: 17 pages

MSC Class: 42B25; 42B37; 35Q41

Decoding Pedestrian Stress on Urban Streets using Electrodermal Activity Monitoring in Virtual Immersive Reality

Authors: Mohsen Nazemi, Bara Rababah, Daniel Ramos, Tangxu Zhao, Bilal Farooq

Abstract: The pedestrian stress level is shown to significantly influence human cognitive processes and, subsequently, decision-making, e.g., the decision to select a gap and cross a street. This paper systematically studies the stress experienced by a pedestrian when crossing a street under different experimental manipulations by monitoring the ElectroDermal Activity (EDA) using the Galvanic Skin Response… ▽ More The pedestrian stress level is shown to significantly influence human cognitive processes and, subsequently, decision-making, e.g., the decision to select a gap and cross a street. This paper systematically studies the stress experienced by a pedestrian when crossing a street under different experimental manipulations by monitoring the ElectroDermal Activity (EDA) using the Galvanic Skin Response (GSR) sensor. To fulfil the research objectives, a dynamic and immersive virtual reality (VR) platform was used, which is suitable for eliciting and capturing pedestrian's emotional responses in conjunction with monitoring their EDA. A total of 171 individuals participated in the experiment, tasked to cross a two-way street at mid-block with no signal control. Mixed effects models were employed to compare the influence of socio-demographics, social influence, vehicle technology, environment, road design, and traffic variables on the stress levels of the participants. The results indicated that having a street median in the middle of the road operates as a refuge and significantly reduced stress. Younger participants were (18-24 years) calmer than the relatively older participants (55-65 years). Arousal levels were higher when it came to the characteristics of the avatar (virtual pedestrian) in the simulation, especially for those avatars with adventurous traits. The pedestrian location influenced stress since the stress was higher on the street while crossing than waiting on the sidewalk. Significant causes of arousal were fear of accidents and an actual accident for pedestrians. The estimated random effects show a high degree of physical and mental learning by the participants while going through the scenarios. △ Less

Submitted 21 August, 2024; originally announced August 2024.

Benchmarking LLMs for Translating Classical Chinese Poetry:Evaluating Adequacy, Fluency, and Elegance

Authors: Andong Chen, Lianzhang Lou, Kehai Chen, Xuefeng Bai, Yang Xiang, Muyun Yang, Tiejun Zhao, Min Zhang

Abstract: Large language models (LLMs) have shown remarkable performance in general translation tasks. However, the increasing demand for high-quality translations that are not only adequate but also fluent and elegant. To assess the extent to which current LLMs can meet these demands, we introduce a suitable benchmark for translating classical Chinese poetry into English. This task requires not only adequa… ▽ More Large language models (LLMs) have shown remarkable performance in general translation tasks. However, the increasing demand for high-quality translations that are not only adequate but also fluent and elegant. To assess the extent to which current LLMs can meet these demands, we introduce a suitable benchmark for translating classical Chinese poetry into English. This task requires not only adequacy in translating culturally and historically significant content but also a strict adherence to linguistic fluency and poetic elegance. Our study reveals that existing LLMs fall short of this task. To address these issues, we propose RAT, a \textbf{R}etrieval-\textbf{A}ugmented machine \textbf{T}ranslation method that enhances the translation process by incorporating knowledge related to classical poetry. Additionally, we propose an automatic evaluation metric based on GPT-4, which better assesses translation quality in terms of adequacy, fluency, and elegance, overcoming the limitations of traditional metrics. Our dataset and code will be made available. △ Less

Submitted 19 August, 2024; originally announced August 2024.

Comments: Work in progress

MR Optimized Reconstruction of Simultaneous Multi-Slice Imaging Using Diffusion Model

Authors: Ting Zhao, Zhuoxu Cui, Sen Jia, Qingyong Zhu, Congcong Liu, Yihang Zhou, Yanjie Zhu, Dong Liang, Haifeng Wang

Abstract: Diffusion model has been successfully applied to MRI reconstruction, including single and multi-coil acquisition of MRI data. Simultaneous multi-slice imaging (SMS), as a method for accelerating MR acquisition, can significantly reduce scanning time, but further optimization of reconstruction results is still possible. In order to optimize the reconstruction of SMS, we proposed a method to use dif… ▽ More Diffusion model has been successfully applied to MRI reconstruction, including single and multi-coil acquisition of MRI data. Simultaneous multi-slice imaging (SMS), as a method for accelerating MR acquisition, can significantly reduce scanning time, but further optimization of reconstruction results is still possible. In order to optimize the reconstruction of SMS, we proposed a method to use diffusion model based on slice-GRAPPA and SPIRiT method. approach: Specifically, our method characterizes the prior distribution of SMS data by score matching and characterizes the k-space redundant prior between coils and slices based on self-consistency. With the utilization of diffusion model, we achieved better reconstruction results.The application of diffusion model can further reduce the scanning time of MRI without compromising image quality, making it more advantageous for clinical application △ Less

Submitted 21 August, 2024; v1 submitted 4 August, 2024; originally announced August 2024.

Comments: Accepted as ISMRM 2024 Digital Poster 4024

Journal ref: ISMRM 2024 Digital poster 4024

Imaginary Poynting momentum driven particle rotation by cylindrically polarized Gaussian beams

Authors: Xue Yun, Yansheng Liang, Linquan Guo, Minru He, Tianyu Zhao, Shaowei Wang, Ming Lei

Abstract: Imaginary Poynting momentum (IPM) provides a new degree of freedom for particle manipulation. However, the application of IPM in experiments has been largely unexplored. Here, we demonstrate the IPM driven particle rotation by cylindrically polarized Gaussian beams with no spin or orbital angular momentum. Theoretical analysis and experimental measurements demonstrate that gold microparticles will… ▽ More Imaginary Poynting momentum (IPM) provides a new degree of freedom for particle manipulation. However, the application of IPM in experiments has been largely unexplored. Here, we demonstrate the IPM driven particle rotation by cylindrically polarized Gaussian beams with no spin or orbital angular momentum. Theoretical analysis and experimental measurements demonstrate that gold microparticles will be rotated in the azimuthal direction while confined in the radial direction. We achieved controllable rotation of the particle by tuning the cylindrical polarization state. Interestingly, the transfer of IPM to a gold particle is demonstrated to be competitive with that of spin angular momentum. These findings hold promising in light-matter interactions and particle manipulations. △ Less

Submitted 14 August, 2024; originally announced August 2024.

Comments: 10 pages, 6 figures

MSC Class: 78A10 Physical optics

Fabrication and characterization of low-loss Al/Si/Al parallel plate capacitors for superconducting quantum information applications

Authors: Anthony McFadden, Aranya Goswami, Tongyu Zhao, Teun van Schijndel, Trevyn F. Q. Larson, Sudhir Sahu, Stephen Gill, Florent Lecocq, Raymond Simmonds, Chris Palmstrøm

Abstract: Increasing the density of superconducting circuits requires compact components, however, superconductor-based capacitors typically perform worse as dimensions are reduced due to loss at surfaces and interfaces. Here, parallel plate capacitors composed of aluminum-contacted, crystalline silicon fins are shown to be a promising technology for use in superconducting circuits by evaluating the perform… ▽ More Increasing the density of superconducting circuits requires compact components, however, superconductor-based capacitors typically perform worse as dimensions are reduced due to loss at surfaces and interfaces. Here, parallel plate capacitors composed of aluminum-contacted, crystalline silicon fins are shown to be a promising technology for use in superconducting circuits by evaluating the performance of lumped element resonators and transmon qubits. High aspect ratio Si-fin capacitors having widths below $300nm$ with an approximate total height of 3$μ$m are fabricated using anisotropic wet etching of Si(110) substrates followed by aluminum metallization. The single-crystal Si capacitors are incorporated in lumped element resonators and transmons by shunting them with lithographically patterned aluminum inductors and conventional $Al/AlO_x/Al$ Josephson junctions respectively. Microwave characterization of these devices suggests state-of-the-art performance for superconducting parallel plate capacitors with low power internal quality factor of lumped element resonators greater than 500k and qubit $T_1$ times greater than 25$μ$s. These results suggest that Si-Fins are a promising technology for applications that require low loss, compact, superconductor-based capacitors with minimal stray capacitance. △ Less

Submitted 23 August, 2024; v1 submitted 2 August, 2024; originally announced August 2024.

Prospects for rank-reduced CCSD(T) in the context of high-accuracy thermochemistry

Authors: Tingting Zhao, James H. Thorpe, Devin A. Matthews

Abstract: Obtaining sub-chemical accuracy (1 kJ mol${}^{-1}$) for reaction energies of medium-sized gas-phase molecules is a longstanding challenge in the field of thermochemical modeling. The perturbative triples correction to CCSD, CCSD(T), constitutes an important component of all high-accuracy composite model chemistries that obtain this accuracy, but can be a roadblock in the calculation of medium to l… ▽ More Obtaining sub-chemical accuracy (1 kJ mol${}^{-1}$) for reaction energies of medium-sized gas-phase molecules is a longstanding challenge in the field of thermochemical modeling. The perturbative triples correction to CCSD, CCSD(T), constitutes an important component of all high-accuracy composite model chemistries that obtain this accuracy, but can be a roadblock in the calculation of medium to large systems due to its $\mathcal{O}(N^7)$ scaling, particularly in HEAT-like model chemistries that eschew separation of core and valance correlation. This study extends the work of Lesiuk [J. Chem. Phys. 156, 064103 (2022)] with new approximate methods and assesses the accuracy of five different approximations of (T) in the context of a subset of molecules selected from the W4-17 dataset. It is demonstrated that all of these approximate methods can achieve sub-0.1 kJ mol${}^{-1}$ accuracy with respect to canonical, density-fitted (T) contributions with a modest number of projectors. The approximation labeled $\tilde{Z}T$ appears to offer the best trade-off between cost and accuracy and shows significant promise in an order-of-magnitude reduction in the computational cost of the CCSD(T) component of high-accuracy model chemistries. △ Less

Submitted 26 July, 2024; originally announced July 2024.

HC-GST: Heterophily-aware Distribution Consistency based Graph Self-training

Authors: Fali Wang, Tianxiang Zhao, Junjie Xu, Suhang Wang

Abstract: Graph self-training (GST), which selects and assigns pseudo-labels to unlabeled nodes, is popular for tackling label sparsity in graphs. However, recent study on homophily graphs show that GST methods could introduce and amplify distribution shift between training and test nodes as they tend to assign pseudo-labels to nodes they are good at. As GNNs typically perform better on homophilic nodes, th… ▽ More Graph self-training (GST), which selects and assigns pseudo-labels to unlabeled nodes, is popular for tackling label sparsity in graphs. However, recent study on homophily graphs show that GST methods could introduce and amplify distribution shift between training and test nodes as they tend to assign pseudo-labels to nodes they are good at. As GNNs typically perform better on homophilic nodes, there could be potential shifts towards homophilic pseudo-nodes, which is underexplored. Our preliminary experiments on heterophilic graphs verify that these methods can cause shifts in homophily ratio distributions, leading to \textit{training bias} that improves performance on homophilic nodes while degrading it on heterophilic ones. Therefore, we study a novel problem of reducing homophily ratio distribution shifts during self-training on heterophilic graphs. A key challenge is the accurate calculation of homophily ratios and their distributions without extensive labeled data. To tackle them, we propose a novel Heterophily-aware Distribution Consistency-based Graph Self-Training (HC-GST) framework, which estimates homophily ratios using soft labels and optimizes a selection vector to align pseudo-nodes with the global homophily ratio distribution. Extensive experiments on both homophilic and heterophilic graphs show that HC-GST effectively reduces training bias and enhances self-training performance. △ Less

Submitted 25 July, 2024; originally announced July 2024.

Comments: accepted by CIKM 2024

Some new properties of the beta function and Ramanujan R-function

Authors: Zhen-Hang Yang, Miao-Kun Wang, Tie-Hong Zhao

Abstract: In this paper, the power series and hypergeometric series representations of the beta and Ramanujan functions \begin{equation*} \mathcal{B}\left( x\right) =\frac{Γ\left( x\right)^{2}}{Γ\left( 2x\right) }\text{ and }\mathcal{R}\left( x\right) =-2ψ\left( x\right) -2γ\end{equation*} are presented, which yield higher order monotonicity results related to $ \mathcal{B}(x)$ and $\mathcal{R}(x)$; the dec… ▽ More In this paper, the power series and hypergeometric series representations of the beta and Ramanujan functions \begin{equation*} \mathcal{B}\left( x\right) =\frac{Γ\left( x\right)^{2}}{Γ\left( 2x\right) }\text{ and }\mathcal{R}\left( x\right) =-2ψ\left( x\right) -2γ\end{equation*} are presented, which yield higher order monotonicity results related to $ \mathcal{B}(x)$ and $\mathcal{R}(x)$; the decreasing property of the functions $\mathcal{R}\left( x\right) /\mathcal{B}\left( x\right) $ and $[ \mathcal{B}(x) -\mathcal{R}(x)] /x^{2}$ on $\left( 0,\infty \right)$ are proved. Moreover, a conjecture put forward by Qiu et al. in [17] is proved to be true. As applications, several inequalities and identities are deduced. These results obtained in this paper may be helpful for the study of certain special functions. Finally, an interesting infinite series similar to Riemann zeta functions is observed initially. △ Less

Submitted 22 July, 2024; originally announced July 2024.

Comments: 18 pages

MSC Class: 33B15; 33C05; 11M06; 30B10; 26A48

Enhancing Data-Limited Graph Neural Networks by Actively Distilling Knowledge from Large Language Models

Authors: Quan Li, Tianxiang Zhao, Lingwei Chen, Junjie Xu, Suhang Wang

Abstract: Graphs are pervasive in the real-world, such as social network analysis, bioinformatics, and knowledge graphs. Graph neural networks (GNNs) have great ability in node classification, a fundamental task on graphs. Unfortunately, conventional GNNs still face challenges in scenarios with few labeled nodes, despite the prevalence of few-shot node classification tasks in real-world applications. To add… ▽ More Graphs are pervasive in the real-world, such as social network analysis, bioinformatics, and knowledge graphs. Graph neural networks (GNNs) have great ability in node classification, a fundamental task on graphs. Unfortunately, conventional GNNs still face challenges in scenarios with few labeled nodes, despite the prevalence of few-shot node classification tasks in real-world applications. To address this challenge, various approaches have been proposed, including graph meta-learning, transfer learning, and methods based on Large Language Models (LLMs). However, traditional meta-learning and transfer learning methods often require prior knowledge from base classes or fail to exploit the potential advantages of unlabeled nodes. Meanwhile, LLM-based methods may overlook the zero-shot capabilities of LLMs and rely heavily on the quality of generated contexts. In this paper, we propose a novel approach that integrates LLMs and GNNs, leveraging the zero-shot inference and reasoning capabilities of LLMs and employing a Graph-LLM-based active learning paradigm to enhance GNNs' performance. Extensive experiments demonstrate the effectiveness of our model in improving node classification accuracy with considerably limited labeled data, surpassing state-of-the-art baselines by significant margins. △ Less

Submitted 28 August, 2024; v1 submitted 18 July, 2024; originally announced July 2024.

Comments: 10 pages, 3 Figures

Surgical Robot Transformer (SRT): Imitation Learning for Surgical Tasks

Authors: Ji Woong Kim, Tony Z. Zhao, Samuel Schmidgall, Anton Deguet, Marin Kobilarov, Chelsea Finn, Axel Krieger

Abstract: We explore whether surgical manipulation tasks can be learned on the da Vinci robot via imitation learning. However, the da Vinci system presents unique challenges which hinder straight-forward implementation of imitation learning. Notably, its forward kinematics is inconsistent due to imprecise joint measurements, and naively training a policy using such approximate kinematics data often leads to… ▽ More We explore whether surgical manipulation tasks can be learned on the da Vinci robot via imitation learning. However, the da Vinci system presents unique challenges which hinder straight-forward implementation of imitation learning. Notably, its forward kinematics is inconsistent due to imprecise joint measurements, and naively training a policy using such approximate kinematics data often leads to task failure. To overcome this limitation, we introduce a relative action formulation which enables successful policy training and deployment using its approximate kinematics data. A promising outcome of this approach is that the large repository of clinical data, which contains approximate kinematics, may be directly utilized for robot learning without further corrections. We demonstrate our findings through successful execution of three fundamental surgical tasks, including tissue manipulation, needle handling, and knot-tying. △ Less

Submitted 17 July, 2024; originally announced July 2024.

Comments: 8 pages

Data Collection and Labeling Techniques for Machine Learning

Authors: Qianyu Huang, Tongfang Zhao

Abstract: Data collection and labeling are critical bottlenecks in the deployment of machine learning applications. With the increasing complexity and diversity of applications, the need for efficient and scalable data collection and labeling techniques has become paramount. This paper provides a review of the state-of-the-art methods in data collection, data labeling, and the improvement of existing data a… ▽ More Data collection and labeling are critical bottlenecks in the deployment of machine learning applications. With the increasing complexity and diversity of applications, the need for efficient and scalable data collection and labeling techniques has become paramount. This paper provides a review of the state-of-the-art methods in data collection, data labeling, and the improvement of existing data and models. By integrating perspectives from both the machine learning and data management communities, we aim to provide a holistic view of the current landscape and identify future research directions. △ Less

Submitted 19 June, 2024; originally announced July 2024.

Separation of Sodium Signals Between Mono- and Bi-Exponential T2 Decays via Multi-TE Single-Quantum Sodium (23Na) MRI

Authors: Yongxian Qian, Ying-Chia Lin, Xingye Chen, Tiejun Zhao, Karthik Lakshmanan, Yulin Ge, Yvonne W. Lui, Fernando E. Boada

Abstract: Purpose. It is a long standing pursuit in sodium (23Na) MRI to separate signals between mono and bi exponential T2 decays in the human brain, due to lack of clinically translational solutions under the restriction of intrinsically low signal to noise ratio (SNR). Here we propose a new technique called multi TE single quantum (MSQ) sodium MRI to address the challenge. Methods. We exploit an intrins… ▽ More Purpose. It is a long standing pursuit in sodium (23Na) MRI to separate signals between mono and bi exponential T2 decays in the human brain, due to lack of clinically translational solutions under the restriction of intrinsically low signal to noise ratio (SNR). Here we propose a new technique called multi TE single quantum (MSQ) sodium MRI to address the challenge. Methods. We exploit an intrinsic difference in T2 decay between mono and bi exponential sodium signals by acquiring SQ images at multiple TEs and performing voxel based matrix inversions on these SQ images. The MSQ method was then investigated on numerical models, agar phantoms, and human brains for the feasibility on clinical scanners at 3T. Results. The whole brain T2* spectrum of FID signals from the study subjects showed sparse peaks (2 to 4 peaks), suggesting a global set of T2* values (T2*fr, T2*bs, T2*bl) applicable to the separation. The simulations indicated a small impact (3.9 to 5.6 percent) of T2* variation on accuracy of the separation, and the phantom experiments showed a high accuracy of the separation, 95.8 percent for mono T2 sodium and 72.5 to 80.4 percent for biT2 sodium. The human studies demonstrated feasibility of the separation and potentials of highlighting abnormal brain regions in the biT2 sodium images. Conclusion. The MSQ technique has been shown, via the numerical simulations, phantom experiments, and human brain studies, to be able to separate mono and bi T2 sodium signals using a two TE sampling scheme and a global set of T2* values. However, MSQ has limitations and requires cautions in practice. Keywords. sodium MRI, single quantum MRI, triple quantum MRI, neuroimaging, neurodegeneration △ Less

Submitted 13 July, 2024; originally announced July 2024.

Comments: 37 pages and 14 figures

RBMD: A molecular dynamics package enabling to simulate 10 million all-atom particles in a single graphics processing unit

Authors: Weihang Gao, Teng Zhao, Yongfa Guo, Jiuyang Liang, Huan Liu, Maoying Luo, Zedong Luo, Wei Qin, Yichao Wang, Qi Zhou, Shi Jin, Zhenli Xu

Abstract: This paper introduces a random-batch molecular dynamics (RBMD) package for fast simulations of particle systems at the nano/micro scale. Different from existing packages, the RBMD uses random batch methods for nonbonded interactions of particle systems. The long-range part of Coulomb interactions is calculated in Fourier space by the random batch Ewald algorithm, which achieves linear complexity a… ▽ More This paper introduces a random-batch molecular dynamics (RBMD) package for fast simulations of particle systems at the nano/micro scale. Different from existing packages, the RBMD uses random batch methods for nonbonded interactions of particle systems. The long-range part of Coulomb interactions is calculated in Fourier space by the random batch Ewald algorithm, which achieves linear complexity and superscalability, surpassing classical lattice-based Ewald methods. For the short-range part, the random batch list algorithm is used to construct neighbor lists, significantly reducing both computational and memory costs. The RBMD is implemented on GPU-CPU heterogeneous architectures, with classical force fields for all-atom systems. Benchmark systems are used to validate accuracy and performance of the package. Comparison with the particle-particle particle-mesh method and the Verlet list method in the LAMMPS package is performed on three different NVIDIA GPUs, demonstrating high efficiency of the RBMD on heterogeneous architectures. Our results also show that the RBMD enables simulations on a single GPU with a CPU core up to 10 million particles. Typically, for systems of one million particles, the RBMD allows simulating all-atom systems with a high efficiency of 8.20 ms per step, demonstrating the attractive feature for running large-scale simulations of practical applications on a desktop machine. △ Less

Submitted 22 August, 2024; v1 submitted 12 July, 2024; originally announced July 2024.

Comments: 26 pages, 8 figures

OmChat: A Recipe to Train Multimodal Language Models with Strong Long Context and Video Understanding

Authors: Tiancheng Zhao, Qianqian Zhang, Kyusong Lee, Peng Liu, Lu Zhang, Chunxin Fang, Jiajia Liao, Kelei Jiang, Yibo Ma, Ruochen Xu

Abstract: We introduce OmChat, a model designed to excel in handling long contexts and video understanding tasks. OmChat's new architecture standardizes how different visual inputs are processed, making it more efficient and adaptable. It uses a dynamic vision encoding process to effectively handle images of various resolutions, capturing fine details across a range of image qualities. OmChat utilizes an ac… ▽ More We introduce OmChat, a model designed to excel in handling long contexts and video understanding tasks. OmChat's new architecture standardizes how different visual inputs are processed, making it more efficient and adaptable. It uses a dynamic vision encoding process to effectively handle images of various resolutions, capturing fine details across a range of image qualities. OmChat utilizes an active progressive multimodal pretraining strategy, which gradually increases the model's capacity for long contexts and enhances its overall abilities. By selecting high-quality data during training, OmChat learns from the most relevant and informative data points. With support for a context length of up to 512K, OmChat demonstrates promising performance in tasks involving multiple images and videos, outperforming most open-source models in these benchmarks. Additionally, OmChat proposes a prompting strategy for unifying complex multimodal inputs including single image text, multi-image text and videos, and achieving competitive performance on single-image benchmarks. To further evaluate the model's capabilities, we proposed a benchmark dataset named Temporal Visual Needle in a Haystack. This dataset assesses OmChat's ability to comprehend temporal visual details within long videos. Our analysis highlights several key factors contributing to OmChat's success: support for any-aspect high image resolution, the active progressive pretraining strategy, and high-quality supervised fine-tuning datasets. This report provides a detailed overview of OmChat's capabilities and the strategies that enhance its performance in visual understanding. △ Less

Submitted 5 July, 2024; originally announced July 2024.

Comments: 14 pages

Similarity Distance-Based Label Assignment for Tiny Object Detection

Authors: Shuohao Shi, Qiang Fang, Tong Zhao, Xin Xu

Abstract: Tiny object detection is becoming one of the most challenging tasks in computer vision because of the limited object size and lack of information. The label assignment strategy is a key factor affecting the accuracy of object detection. Although there are some effective label assignment strategies for tiny objects, most of them focus on reducing the sensitivity to the bounding boxes to increase th… ▽ More Tiny object detection is becoming one of the most challenging tasks in computer vision because of the limited object size and lack of information. The label assignment strategy is a key factor affecting the accuracy of object detection. Although there are some effective label assignment strategies for tiny objects, most of them focus on reducing the sensitivity to the bounding boxes to increase the number of positive samples and have some fixed hyperparameters need to set. However, more positive samples may not necessarily lead to better detection results, in fact, excessive positive samples may lead to more false positives. In this paper, we introduce a simple but effective strategy named the Similarity Distance (SimD) to evaluate the similarity between bounding boxes. This proposed strategy not only considers both location and shape similarity but also learns hyperparameters adaptively, ensuring that it can adapt to different datasets and various object sizes in a dataset. Our approach can be simply applied in common anchor-based detectors in place of the IoU for label assignment and Non Maximum Suppression (NMS). Extensive experiments on four mainstream tiny object detection datasets demonstrate superior performance of our method, especially, 1.8 AP points and 4.1 AP points of very tiny higher than the state-of-the-art competitors on AI-TOD. Code is available at: \url{https://github.com/cszzshi/SimD}. △ Less

Submitted 26 July, 2024; v1 submitted 2 July, 2024; originally announced July 2024.

Comments: 8 pages, 4 figures, this paper has been accepted by IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2024)

GMT: A Robust Global Association Model for Multi-Target Multi-Camera Tracking

Authors: Huijie Fan, Tinghui Zhao, Qiang Wang, Baojie Fan, Yandong Tang, LianQing Liu

Abstract: In the task of multi-target multi-camera (MTMC) tracking of pedestrians, the data association problem is a key issue and main challenge, especially with complications arising from camera movements, lighting variations, and obstructions. However, most MTMC models adopt two-step approaches, thus heavily depending on the results of the first-step tracking in practical applications. Moreover, the same… ▽ More In the task of multi-target multi-camera (MTMC) tracking of pedestrians, the data association problem is a key issue and main challenge, especially with complications arising from camera movements, lighting variations, and obstructions. However, most MTMC models adopt two-step approaches, thus heavily depending on the results of the first-step tracking in practical applications. Moreover, the same targets crossing different cameras may exhibit significant appearance variations, which further increases the difficulty of cross-camera matching. To address the aforementioned issues, we propose a global online MTMC tracking model that addresses the dependency on the first tracking stage in two-step methods and enhances cross-camera matching. Specifically, we propose a transformer-based global MTMC association module to explore target associations across different cameras and frames, generating global trajectories directly. Additionally, to integrate the appearance and spatio-temporal features of targets, we propose a feature extraction and fusion module for MTMC tracking. This module enhances feature representation and establishes correlations between the features of targets across multiple cameras. To accommodate high scene diversity and complex lighting condition variations, we have established the VisionTrack dataset, which enables the development of models that are more generalized and robust to various environments. Our model demonstrates significant improvements over comparison methods on the VisionTrack dataset and others. △ Less

Submitted 1 July, 2024; originally announced July 2024.

JungleGPT: Designing and Optimizing Compound AI Systems for E-Commerce

Authors: Sherry Ruan, Tian Zhao

Abstract: LLMs have significantly advanced the e-commerce industry by powering applications such as personalized recommendations and customer service. However, most current efforts focus solely on monolithic LLMs and fall short in addressing the complexity and scale of real-world e-commerce scenarios. In this work, we present JungleGPT, the first compound AI system tailored for real-world e-commerce applica… ▽ More LLMs have significantly advanced the e-commerce industry by powering applications such as personalized recommendations and customer service. However, most current efforts focus solely on monolithic LLMs and fall short in addressing the complexity and scale of real-world e-commerce scenarios. In this work, we present JungleGPT, the first compound AI system tailored for real-world e-commerce applications. We outline the system's design and the techniques used to optimize its performance for practical use cases, which have proven to reduce inference costs to less than 1% of what they would be with a powerful, monolithic LLM. △ Less

Submitted 28 May, 2024; originally announced July 2024.

Conformalized Link Prediction on Graph Neural Networks

Authors: Tianyi Zhao, Jian Kang, Lu Cheng

Abstract: Graph Neural Networks (GNNs) excel in diverse tasks, yet their applications in high-stakes domains are often hampered by unreliable predictions. Although numerous uncertainty quantification methods have been proposed to address this limitation, they often lack \textit{rigorous} uncertainty estimates. This work makes the first attempt to introduce a distribution-free and model-agnostic uncertainty… ▽ More Graph Neural Networks (GNNs) excel in diverse tasks, yet their applications in high-stakes domains are often hampered by unreliable predictions. Although numerous uncertainty quantification methods have been proposed to address this limitation, they often lack \textit{rigorous} uncertainty estimates. This work makes the first attempt to introduce a distribution-free and model-agnostic uncertainty quantification approach to construct a predictive interval with a statistical guarantee for GNN-based link prediction. We term it as \textit{conformalized link prediction.} Our approach builds upon conformal prediction (CP), a framework that promises to construct statistically robust prediction sets or intervals. We first theoretically and empirically establish a permutation invariance condition for the application of CP in link prediction tasks, along with an exact test-time coverage. Leveraging the important structural information in graphs, we then identify a novel and crucial connection between a graph's adherence to the power law distribution and the efficiency of CP. This insight leads to the development of a simple yet effective sampling-based method to align the graph structure with a power law distribution prior to the standard CP procedure. Extensive experiments demonstrate that for conformalized link prediction, our approach achieves the desired marginal coverage while significantly improving the efficiency of CP compared to baseline methods. △ Less

Submitted 18 July, 2024; v1 submitted 26 June, 2024; originally announced June 2024.

OmAgent: A Multi-modal Agent Framework for Complex Video Understanding with Task Divide-and-Conquer

Authors: Lu Zhang, Tiancheng Zhao, Heting Ying, Yibo Ma, Kyusong Lee

Abstract: Recent advancements in Large Language Models (LLMs) have expanded their capabilities to multimodal contexts, including comprehensive video understanding. However, processing extensive videos such as 24-hour CCTV footage or full-length films presents significant challenges due to the vast data and processing demands. Traditional methods, like extracting key frames or converting frames to text, ofte… ▽ More Recent advancements in Large Language Models (LLMs) have expanded their capabilities to multimodal contexts, including comprehensive video understanding. However, processing extensive videos such as 24-hour CCTV footage or full-length films presents significant challenges due to the vast data and processing demands. Traditional methods, like extracting key frames or converting frames to text, often result in substantial information loss. To address these shortcomings, we develop OmAgent, efficiently stores and retrieves relevant video frames for specific queries, preserving the detailed content of videos. Additionally, it features an Divide-and-Conquer Loop capable of autonomous reasoning, dynamically invoking APIs and tools to enhance query processing and accuracy. This approach ensures robust video understanding, significantly reducing information loss. Experimental results affirm OmAgent's efficacy in handling various types of videos and complex tasks. Moreover, we have endowed it with greater autonomy and a robust tool-calling system, enabling it to accomplish even more intricate tasks. △ Less

Submitted 24 June, 2024; v1 submitted 24 June, 2024; originally announced June 2024.

Multimodal Graph Benchmark

Authors: Jing Zhu, Yuhang Zhou, Shengyi Qian, Zhongmou He, Tong Zhao, Neil Shah, Danai Koutra

Abstract: Associating unstructured data with structured information is crucial for real-world tasks that require relevance search. However, existing graph learning benchmarks often overlook the rich semantic information associate with each node. To bridge such gap, we introduce the Multimodal Graph Benchmark (MM-GRAPH), the first comprehensive multi-modal graph benchmark that incorporates both textual and v… ▽ More Associating unstructured data with structured information is crucial for real-world tasks that require relevance search. However, existing graph learning benchmarks often overlook the rich semantic information associate with each node. To bridge such gap, we introduce the Multimodal Graph Benchmark (MM-GRAPH), the first comprehensive multi-modal graph benchmark that incorporates both textual and visual information. MM-GRAPH surpasses previous efforts, which have primarily focused on text-attributed graphs with various connectivity patterns. MM-GRAPH consists of five graph learning datasets of various scales that are appropriate for different learning tasks. Their multimodal node features, enabling a more comprehensive evaluation of graph learning algorithms in real-world scenarios. To facilitate research on multimodal graph learning, we further provide an extensive study on the performance of various graph neural networks in the presence of features from various modalities. MM-GRAPH aims to foster research on multimodal graph learning and drive the development of more advanced and robust graph learning algorithms. By providing a diverse set of datasets and benchmarks, MM-GRAPH enables researchers to evaluate and compare their models in realistic settings, ultimately leading to improved performance on real-world applications that rely on multimodal graph data. △ Less

Submitted 24 June, 2024; originally announced June 2024.

Comments: https://mm-graph-benchmark.github.io/

Robust Reinforcement Learning from Corrupted Human Feedback

Authors: Alexander Bukharin, Ilgee Hong, Haoming Jiang, Zichong Li, Qingru Zhang, Zixuan Zhang, Tuo Zhao

Abstract: Reinforcement learning from human feedback (RLHF) provides a principled framework for aligning AI systems with human preference data. For various reasons, e.g., personal bias, context ambiguity, lack of training, etc, human annotators may give incorrect or inconsistent preference labels. To tackle this challenge, we propose a robust RLHF approach -- $R^3M$, which models the potentially corrupted p… ▽ More Reinforcement learning from human feedback (RLHF) provides a principled framework for aligning AI systems with human preference data. For various reasons, e.g., personal bias, context ambiguity, lack of training, etc, human annotators may give incorrect or inconsistent preference labels. To tackle this challenge, we propose a robust RLHF approach -- $R^3M$, which models the potentially corrupted preference label as sparse outliers. Accordingly, we formulate the robust reward learning as an $\ell_1$-regularized maximum likelihood estimation problem. Computationally, we develop an efficient alternating optimization algorithm, which only incurs negligible computational overhead compared with the standard RLHF approach. Theoretically, we prove that under proper regularity conditions, $R^3M$ can consistently learn the underlying reward and identify outliers, provided that the number of outlier labels scales sublinearly with the preference sample size. Furthermore, we remark that $R^3M$ is versatile and can be extended to various preference optimization methods, including direct preference optimization (DPO). Our experiments on robotic control and natural language generation with large language models (LLMs) show that $R^3M$ improves robustness of the reward against several types of perturbations to the preference data. △ Less

Submitted 9 July, 2024; v1 submitted 21 June, 2024; originally announced June 2024.

Comments: 22 pages, 7 figures

Enhancing Travel Choice Modeling with Large Language Models: A Prompt-Learning Approach

Authors: Xuehao Zhai, Hanlin Tian, Lintong Li, Tianyu Zhao

Abstract: Travel choice analysis is crucial for understanding individual travel behavior to develop appropriate transport policies and recommendation systems in Intelligent Transportation Systems (ITS). Despite extensive research, this domain faces two critical challenges: a) modeling with limited survey data, and b) simultaneously achieving high model explainability and accuracy. In this paper, we introduc… ▽ More Travel choice analysis is crucial for understanding individual travel behavior to develop appropriate transport policies and recommendation systems in Intelligent Transportation Systems (ITS). Despite extensive research, this domain faces two critical challenges: a) modeling with limited survey data, and b) simultaneously achieving high model explainability and accuracy. In this paper, we introduce a novel prompt-learning-based Large Language Model(LLM) framework that significantly improves prediction accuracy and provides explicit explanations for individual predictions. This framework involves three main steps: transforming input variables into textual form; building of demonstrations similar to the object, and applying these to a well-trained LLM. We tested the framework's efficacy using two widely used choice datasets: London Passenger Mode Choice (LPMC) and Optima-Mode collected in Switzerland. The results indicate that the LLM significantly outperforms state-of-the-art deep learning methods and discrete choice models in predicting people's choices. Additionally, we present a case of explanation illustrating how the LLM framework generates understandable and explicit explanations at the individual level. △ Less

Submitted 22 June, 2024; v1 submitted 19 June, 2024; originally announced June 2024.

Comments: We currently do not have a replacement version available. We request withdrawal due to a significant methodological error affecting the paper's validity, specifically a miscalculation in data preprocessing. We are working on corrections, but this will take time. We believe an interim withdrawal is necessary to prevent the dissemination of incorrect information.

A data-centric approach for assessing progress of Graph Neural Networks

Authors: Tianqi Zhao, Ngan Thi Dong, Alan Hanjalic, Megha Khosla

Abstract: Graph Neural Networks (GNNs) have achieved state-of-the-art results in node classification tasks. However, most improvements are in multi-class classification, with less focus on the cases where each node could have multiple labels. The first challenge in studying multi-label node classification is the scarcity of publicly available datasets. To address this, we collected and released three real-w… ▽ More Graph Neural Networks (GNNs) have achieved state-of-the-art results in node classification tasks. However, most improvements are in multi-class classification, with less focus on the cases where each node could have multiple labels. The first challenge in studying multi-label node classification is the scarcity of publicly available datasets. To address this, we collected and released three real-world biological datasets and developed a multi-label graph generator with tunable properties. We also argue that traditional notions of homophily and heterophily do not apply well to multi-label scenarios. Therefore, we define homophily and Cross-Class Neighborhood Similarity for multi-label classification and investigate $9$ collected multi-label datasets. Lastly, we conducted a large-scale comparative study with $8$ methods across nine datasets to evaluate current progress in multi-label node classification. We release our code at \url{https://github.com/Tianqi-py/MLGNC}. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Journal ref: Published in Data-centric Machine Learning Research Worshop @ ICML 2024

Preserving Knowledge in Large Language Model with Model-Agnostic Self-Decompression

Authors: Zilun Zhang, Yutao Sun, Tiancheng Zhao, Leigang Sha, Ruochen Xu, Kyusong Lee, Jianwei Yin

Abstract: Humans can retain old knowledge while learning new information, but Large Language Models (LLMs) often suffer from catastrophic forgetting when post-pretrained or supervised fine-tuned (SFT) on domain-specific data. Moreover, for Multimodal Large Language Models (MLLMs) which are composed of the LLM base and visual projector (e.g. LLaVA), a significant decline in performance on language benchmarks… ▽ More Humans can retain old knowledge while learning new information, but Large Language Models (LLMs) often suffer from catastrophic forgetting when post-pretrained or supervised fine-tuned (SFT) on domain-specific data. Moreover, for Multimodal Large Language Models (MLLMs) which are composed of the LLM base and visual projector (e.g. LLaVA), a significant decline in performance on language benchmarks was observed compared to their single-modality counterparts. To address these challenges, we introduce a novel model-agnostic self-decompression method, Tree Generation (TG), that decompresses knowledge within LLMs into the training corpus. This paper focuses on TG-SFT, which can synthetically generate SFT data for the instruction tuning steps. By incorporating the dumped corpus during SFT for MLLMs, we significantly reduce the forgetting problem. △ Less

Submitted 19 June, 2024; v1 submitted 17 June, 2024; originally announced June 2024.

Resolving Geometric Excitations of Fractional Quantum Hall States

Authors: Yang Liu, Tongzhou Zhao, T. Xiang

Abstract: The quantum dynamics of the intrinsic metric profoundly influence the neutral excitations in the fractional quantum Hall system, as established by Haldane in 2011 \cite{Haldane2011}, and further evidenced by a recent two-photon experiment \cite{Liang2024}. Despite these advancements, a comprehensive understanding of the dynamic properties of these excitations, especially at long wavelengths, conti… ▽ More The quantum dynamics of the intrinsic metric profoundly influence the neutral excitations in the fractional quantum Hall system, as established by Haldane in 2011 \cite{Haldane2011}, and further evidenced by a recent two-photon experiment \cite{Liang2024}. Despite these advancements, a comprehensive understanding of the dynamic properties of these excitations, especially at long wavelengths, continues to elude interest. In this study, we employ tensor-network methods to investigate the neutral excitations of the Laughlin and Moore-Read states on an infinite cylinder. This investigation deepens our understanding of the excitation spectrum in regions where traditional methods do not work effectively. The spectral functions for both states reveal the presence of $S=-2$ geometric excitations. For the first time, we unveil the complex spectra of both neutral fermion and bosonic Girvin-MacDonald-Platzman modes within the excitation continuum by calculating the three-particle density response function for the Moore-Read state. Our findings support the hypothesis of emergent supersymmetry and highlight the potential for detecting neutral fermions in future experiments. △ Less

Submitted 1 July, 2024; v1 submitted 17 June, 2024; originally announced June 2024.

A Survey on Human Preference Learning for Large Language Models

Authors: Ruili Jiang, Kehai Chen, Xuefeng Bai, Zhixuan He, Juntao Li, Muyun Yang, Tiejun Zhao, Liqiang Nie, Min Zhang

Abstract: The recent surge of versatile large language models (LLMs) largely depends on aligning increasingly capable foundation models with human intentions by preference learning, enhancing LLMs with excellent applicability and effectiveness in a wide range of contexts. Despite the numerous related studies conducted, a perspective on how human preferences are introduced into LLMs remains limited, which ma… ▽ More The recent surge of versatile large language models (LLMs) largely depends on aligning increasingly capable foundation models with human intentions by preference learning, enhancing LLMs with excellent applicability and effectiveness in a wide range of contexts. Despite the numerous related studies conducted, a perspective on how human preferences are introduced into LLMs remains limited, which may prevent a deeper comprehension of the relationships between human preferences and LLMs as well as the realization of their limitations. In this survey, we review the progress in exploring human preference learning for LLMs from a preference-centered perspective, covering the sources and formats of preference feedback, the modeling and usage of preference signals, as well as the evaluation of the aligned LLMs. We first categorize the human feedback according to data sources and formats. We then summarize techniques for human preferences modeling and compare the advantages and disadvantages of different schools of models. Moreover, we present various preference usage methods sorted by the objectives to utilize human preference signals. Finally, we summarize some prevailing approaches to evaluate LLMs in terms of alignment with human intentions and discuss our outlooks on the human intention alignment for LLMs. △ Less

Submitted 18 June, 2024; v1 submitted 16 June, 2024; originally announced June 2024.

Comments: IEEE copyright statement added (also applied to the former version)

STAR: Scale-wise Text-to-image generation via Auto-Regressive representations

Authors: Xiaoxiao Ma, Mohan Zhou, Tao Liang, Yalong Bai, Tiejun Zhao, Huaian Chen, Yi Jin

Abstract: We present STAR, a text-to-image model that employs scale-wise auto-regressive paradigm. Unlike VAR, which is limited to class-conditioned synthesis within a fixed set of predetermined categories, our STAR enables text-driven open-set generation through three key designs: To boost diversity and generalizability with unseen combinations of objects and concepts, we introduce a pre-trained text encod… ▽ More We present STAR, a text-to-image model that employs scale-wise auto-regressive paradigm. Unlike VAR, which is limited to class-conditioned synthesis within a fixed set of predetermined categories, our STAR enables text-driven open-set generation through three key designs: To boost diversity and generalizability with unseen combinations of objects and concepts, we introduce a pre-trained text encoder to extract representations for textual constraints, which we then use as guidance. To improve the interactions between generated images and fine-grained textual guidance, making results more controllable, additional cross-attention layers are incorporated at each scale. Given the natural structure correlation across different scales, we leverage 2D Rotary Positional Encoding (RoPE) and tweak it into a normalized version. This ensures consistent interpretation of relative positions across token maps at different scales and stabilizes the training process. Extensive experiments demonstrate that STAR surpasses existing benchmarks in terms of fidelity,image text consistency, and aesthetic quality. Our findings emphasize the potential of auto-regressive methods in the field of high-quality image synthesis, offering promising new directions for the T2I field currently dominated by diffusion methods. △ Less

Submitted 15 June, 2024; originally announced June 2024.

Comments: 12 pages, 6 figures

RoseLoRA: Row and Column-wise Sparse Low-rank Adaptation of Pre-trained Language Model for Knowledge Editing and Fine-tuning

Authors: Haoyu Wang, Tianci Liu, Ruirui Li, Monica Cheng, Tuo Zhao, Jing Gao

Abstract: Pre-trained language models, trained on large-scale corpora, demonstrate strong generalizability across various NLP tasks. Fine-tuning these models for specific tasks typically involves updating all parameters, which is resource-intensive. Parameter-efficient fine-tuning (PEFT) methods, such as the popular LoRA family, introduce low-rank matrices to learn only a few parameters efficiently. However… ▽ More Pre-trained language models, trained on large-scale corpora, demonstrate strong generalizability across various NLP tasks. Fine-tuning these models for specific tasks typically involves updating all parameters, which is resource-intensive. Parameter-efficient fine-tuning (PEFT) methods, such as the popular LoRA family, introduce low-rank matrices to learn only a few parameters efficiently. However, during inference, the product of these matrices updates all pre-trained parameters, complicating tasks like knowledge editing that require selective updates. We propose a novel PEFT method, which conducts \textbf{r}ow and c\textbf{o}lumn-wise spar\textbf{se} \textbf{lo}w-\textbf{r}ank \textbf{a}daptation (RoseLoRA), to address this challenge. RoseLoRA identifies and updates only the most important parameters for a specific task, maintaining efficiency while preserving other model knowledge. By adding a sparsity constraint on the product of low-rank matrices and converting it to row and column-wise sparsity, we ensure efficient and precise model updates. Our theoretical analysis guarantees the lower bound of the sparsity with respective to the matrix product. Extensive experiments on five benchmarks across twenty datasets demonstrate that RoseLoRA outperforms baselines in both general fine-tuning and knowledge editing tasks. △ Less

Submitted 30 July, 2024; v1 submitted 15 June, 2024; originally announced June 2024.

QDA-SQL: Questions Enhanced Dialogue Augmentation for Multi-Turn Text-to-SQL

Authors: Yinggang Sun, Ziming Guo, Haining Yu, Chuanyi Liu, Xiang Li, Bingxuan Wang, Xiangzhan Yu, Tiancheng Zhao

Abstract: Fine-tuning large language models (LLMs) for specific domain tasks has achieved great success in Text-to-SQL tasks. However, these fine-tuned models often face challenges with multi-turn Text-to-SQL tasks caused by ambiguous or unanswerable questions. It is desired to enhance LLMs to handle multiple types of questions in multi-turn Text-to-SQL tasks. To address this, we propose a novel data augmen… ▽ More Fine-tuning large language models (LLMs) for specific domain tasks has achieved great success in Text-to-SQL tasks. However, these fine-tuned models often face challenges with multi-turn Text-to-SQL tasks caused by ambiguous or unanswerable questions. It is desired to enhance LLMs to handle multiple types of questions in multi-turn Text-to-SQL tasks. To address this, we propose a novel data augmentation method, called QDA-SQL, which generates multiple types of multi-turn Q\&A pairs by using LLMs. In QDA-SQL, we introduce a novel data augmentation method incorporating validation and correction mechanisms to handle complex multi-turn Text-to-SQL tasks. Experimental results demonstrate that QDA-SQL enables fine-tuned models to exhibit higher performance on SQL statement accuracy and enhances their ability to handle complex, unanswerable questions in multi-turn Text-to-SQL tasks. The generation script and test set are released at https://github.com/mcxiaoxiao/QDA-SQL. △ Less

Submitted 15 June, 2024; originally announced June 2024.

Comments: 13 pages, 7 figures

Multi-source Unsupervised Domain Adaptation on Graphs with Transferability Modeling

Authors: Tianxiang Zhao, Dongsheng Luo, Xiang Zhang, Suhang Wang

Abstract: In this paper, we tackle a new problem of \textit{multi-source unsupervised domain adaptation (MSUDA) for graphs}, where models trained on annotated source domains need to be transferred to the unsupervised target graph for node classification. Due to the discrepancy in distribution across domains, the key challenge is how to select good source instances and how to adapt the model. Diverse graph s… ▽ More In this paper, we tackle a new problem of \textit{multi-source unsupervised domain adaptation (MSUDA) for graphs}, where models trained on annotated source domains need to be transferred to the unsupervised target graph for node classification. Due to the discrepancy in distribution across domains, the key challenge is how to select good source instances and how to adapt the model. Diverse graph structures further complicate this problem, rendering previous MSUDA approaches less effective. In this work, we present the framework Selective Multi-source Adaptation for Graph ({\method}), with a graph-modeling-based domain selector, a sub-graph node selector, and a bi-level alignment objective for the adaptation. Concretely, to facilitate the identification of informative source data, the similarity across graphs is disentangled and measured with the transferability of a graph-modeling task set, and we use it as evidence for source domain selection. A node selector is further incorporated to capture the variation in transferability of nodes within the same source domain. To learn invariant features for adaptation, we align the target domain to selected source data both at the embedding space by minimizing the optimal transport distance and at the classification level by distilling the label function. Modules are explicitly learned to select informative source data and conduct the alignment in virtual training splits with a meta-learning strategy. Experimental results on five graph datasets show the effectiveness of the proposed method. △ Less

Submitted 22 June, 2024; v1 submitted 14 June, 2024; originally announced June 2024.

Journal ref: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '24), August 25--29, 2024, Barcelona, Spain

DiTFastAttn: Attention Compression for Diffusion Transformer Models

Authors: Zhihang Yuan, Pu Lu, Hanling Zhang, Xuefei Ning, Linfeng Zhang, Tianchen Zhao, Shengen Yan, Guohao Dai, Yu Wang

Abstract: Diffusion Transformers (DiT) excel at image and video generation but face computational challenges due to self-attention's quadratic complexity. We propose DiTFastAttn, a novel post-training compression method to alleviate DiT's computational bottleneck. We identify three key redundancies in the attention computation during DiT inference: 1. spatial redundancy, where many attention heads focus on… ▽ More Diffusion Transformers (DiT) excel at image and video generation but face computational challenges due to self-attention's quadratic complexity. We propose DiTFastAttn, a novel post-training compression method to alleviate DiT's computational bottleneck. We identify three key redundancies in the attention computation during DiT inference: 1. spatial redundancy, where many attention heads focus on local information; 2. temporal redundancy, with high similarity between neighboring steps' attention outputs; 3. conditional redundancy, where conditional and unconditional inferences exhibit significant similarity. To tackle these redundancies, we propose three techniques: 1. Window Attention with Residual Caching to reduce spatial redundancy; 2. Temporal Similarity Reduction to exploit the similarity between steps; 3. Conditional Redundancy Elimination to skip redundant computations during conditional generation. To demonstrate the effectiveness of DiTFastAttn, we apply it to DiT, PixArt-Sigma for image generation tasks, and OpenSora for video generation tasks. Evaluation results show that for image generation, our method reduces up to 88\% of the FLOPs and achieves up to 1.6x speedup at high resolution generation. △ Less

Submitted 12 June, 2024; originally announced June 2024.

DUAL-REFLECT: Enhancing Large Language Models for Reflective Translation through Dual Learning Feedback Mechanisms

Authors: Andong Chen, Lianzhang Lou, Kehai Chen, Xuefeng Bai, Yang Xiang, Muyun Yang, Tiejun Zhao, Min Zhang

Abstract: Recently, large language models (LLMs) enhanced by self-reflection have achieved promising performance on machine translation. The key idea is guiding LLMs to generate translation with human-like feedback. However, existing self-reflection methods lack effective feedback information, limiting the translation performance. To address this, we introduce a DUAL-REFLECT framework, leveraging the dual l… ▽ More Recently, large language models (LLMs) enhanced by self-reflection have achieved promising performance on machine translation. The key idea is guiding LLMs to generate translation with human-like feedback. However, existing self-reflection methods lack effective feedback information, limiting the translation performance. To address this, we introduce a DUAL-REFLECT framework, leveraging the dual learning of translation tasks to provide effective feedback, thereby enhancing the models' self-reflective abilities and improving translation performance. The application of this method across various translation tasks has proven its effectiveness in improving translation accuracy and eliminating ambiguities, especially in translation tasks with low-resource language pairs. △ Less

Submitted 21 June, 2024; v1 submitted 11 June, 2024; originally announced June 2024.

Comments: Accepted to ACL 2024 main conference

HORAE: A Domain-Agnostic Modeling Language for Automating Multimodal Service Regulation

Authors: Yutao Sun, Mingshuai Chen, Tiancheng Zhao, Kangjia Zhao, He Li, Jintao Chen, Liqiang Lu, Xinkui Zhao, Shuiguang Deng, Jianwei Yin

Abstract: Artificial intelligence is rapidly encroaching on the field of service regulation. This work presents the design principles behind HORAE, a unified specification language to model multimodal regulation rules across a diverse set of domains. We show how HORAE facilitates an intelligent service regulation pipeline by further exploiting a fine-tuned large language model named HORAE that automates the… ▽ More Artificial intelligence is rapidly encroaching on the field of service regulation. This work presents the design principles behind HORAE, a unified specification language to model multimodal regulation rules across a diverse set of domains. We show how HORAE facilitates an intelligent service regulation pipeline by further exploiting a fine-tuned large language model named HORAE that automates the HORAE modeling process, thereby yielding an end-to-end framework for fully automated intelligent service regulation. △ Less

Submitted 18 July, 2024; v1 submitted 6 June, 2024; originally announced June 2024.

Phase estimation via number-conserving operation inside the SU(1,1) interferometer

Authors: Qingqian Kang, Zekun Zhao, Teng Zhao, Cunjin Liu, Liyun Hu

Abstract: Utilizing nonlinear elements, SU(1,1) interferometers demonstrate superior phase sensitivity compared to passive interferometers. However, the precision is significantly impacted by photon losses, particularly internal losses. We propose a theoretical scheme to improve the precision of phase measurement using homodyne detection by implementing number-conserving operations (PA-then-PS and PS-then-P… ▽ More Utilizing nonlinear elements, SU(1,1) interferometers demonstrate superior phase sensitivity compared to passive interferometers. However, the precision is significantly impacted by photon losses, particularly internal losses. We propose a theoretical scheme to improve the precision of phase measurement using homodyne detection by implementing number-conserving operations (PA-then-PS and PS-then-PA) within the SU(1,1) interferometer, with the coherent state and the vacuum state as the input states. We analyze the effects of number-conserving operations on the phase sensitivity, the quantum Fisher information, and the quantum Cramer-Rao bound under both ideal and photon losses scenarios. Our findings reveal that the internal non-Gaussian operations can enhance the phase sensitivity and the quantum Fisher information, and effectively improve the robustness of the SU(1,1) interferometer against internal photon losses. Notably, the PS-then-PA scheme exhibits superior improvement in both ideal and photon losses cases in terms of phase sensitivity. Moreover, in the ideal case, PA-then-PS scheme slightly outperforms PS-then-PA scheme in terms of the quantum Fisher information and the Quantum Cramer-Rao. However, in the presence of photon losses, PS-then-PA scheme demonstrates a greater advantage. △ Less

Submitted 20 July, 2024; v1 submitted 29 March, 2024; originally announced June 2024.

Comments: arXiv admin note: text overlap with arXiv:2311.14612

GCtx-UNet: Efficient Network for Medical Image Segmentation

Authors: Khaled Alrfou, Tian Zhao

Abstract: Medical image segmentation is crucial for disease diagnosis and monitoring. Though effective, the current segmentation networks such as UNet struggle with capturing long-range features. More accurate models such as TransUNet, Swin-UNet, and CS-UNet have higher computation complexity. To address this problem, we propose GCtx-UNet, a lightweight segmentation architecture that can capture global and… ▽ More Medical image segmentation is crucial for disease diagnosis and monitoring. Though effective, the current segmentation networks such as UNet struggle with capturing long-range features. More accurate models such as TransUNet, Swin-UNet, and CS-UNet have higher computation complexity. To address this problem, we propose GCtx-UNet, a lightweight segmentation architecture that can capture global and local image features with accuracy better or comparable to the state-of-the-art approaches. GCtx-UNet uses vision transformer that leverages global context self-attention modules joined with local self-attention to model long and short range spatial dependencies. GCtx-UNet is evaluated on the Synapse multi-organ abdominal CT dataset, the ACDC cardiac MRI dataset, and several polyp segmentation datasets. In terms of Dice Similarity Coefficient (DSC) and Hausdorff Distance (HD) metrics, GCtx-UNet outperformed CNN-based and Transformer-based approaches, with notable gains in the segmentation of complex and small anatomical structures. Moreover, GCtx-UNet is much more efficient than the state-of-the-art approaches with smaller model size, lower computation workload, and faster training and inference speed, making it a practical choice for clinical applications. △ Less

Submitted 9 June, 2024; originally announced June 2024.

Comments: 13 pages, 7 figures, 7 tables

Analytic gradients for equation-of-motion coupled cluster with single, double, and perturbative triple excitations

Authors: Tingting Zhao, Devin A. Matthews

Abstract: Understanding the process of molecular photoexcitation is crucial in various fields, including drug development, materials science, photovoltaics, and more. The electronic vertical excitation energy is a critical property, for example in determining the singlet-triplet gap of chromophores. However, a full understanding of excited-state processes requires additional explorations of the excited-stat… ▽ More Understanding the process of molecular photoexcitation is crucial in various fields, including drug development, materials science, photovoltaics, and more. The electronic vertical excitation energy is a critical property, for example in determining the singlet-triplet gap of chromophores. However, a full understanding of excited-state processes requires additional explorations of the excited-state potential energy surface and electronic properties, which is greatly aided by the availability of analytic energy gradients. Owing to its robust high accuracy over a wide range of chemical problems, equation-of-motion coupled-cluster with single and double excitations (EOM-CCSD) is a powerful method for predicting excited state properties, and the implementation of analytic gradients of many EOM-CCSD (excitation energies, ionization potentials, electron attachment energies, etc.) along with numerous successful applications highlights the flexibility of the method. In specific cases where a higher level of accuracy is needed or in more complex electronic structures, the inclusion of triple excitations becomes essential, for example, in the EOM-CCSD* approach of Saeh and Stanton. In this work, we derive and implement for the first time the analytic gradients of EOMEE-CCSD*, which also provides a template for analytic gradients of related excited state methods with perturbative triple excitations. The capabilities of analytic EOMEE-CCSD* gradients are illustrated by several representative examples. △ Less

Submitted 8 June, 2024; originally announced June 2024.

Practical Explicit-time Stabilization of a Proportional Control System

Authors: Wen Yan, Tao Zhao

Abstract: Proportional control can be realized directly through the amplification of analog signals, and it also has the advantage of easy tuning parameters in digital signal control. However, it is difficult for the proportional control to preset the upper bound of settling time. To address this problem, a novel practical explicit-time control method is proposed. In bounded initial condition, this method m… ▽ More Proportional control can be realized directly through the amplification of analog signals, and it also has the advantage of easy tuning parameters in digital signal control. However, it is difficult for the proportional control to preset the upper bound of settling time. To address this problem, a novel practical explicit-time control method is proposed. In bounded initial condition, this method makes this system error converge to a predefined neighborhood of zero within an explicit time. More specifically, the initial condition set and conditionally stable set are solved by practical explicit-time stabilization theorem. Based on that, a proportional feedback control is founded to achieve practical conditional fixed-time stability. △ Less

Submitted 8 June, 2024; originally announced June 2024.

Characterizing the nuclear models informed by PREX and CREX: a view from Bayesian inference

Authors: Tianqi Zhao, Zidu Lin, Bharat Kumar, Andrew W. Steiner, Madappa Prakash

Abstract: New measurements of the weak charge density distributions of $^{48}$Ca and $^{208}$Pb challenge existing nuclear models. In the post-PREX-CREX era, it is unclear if current models can simultaneously describe weak charge distributions along with accurate measurements of binding energy and charge radii. In this letter, we explore the parameter space of relativistic and non-relativistic models to stu… ▽ More New measurements of the weak charge density distributions of $^{48}$Ca and $^{208}$Pb challenge existing nuclear models. In the post-PREX-CREX era, it is unclear if current models can simultaneously describe weak charge distributions along with accurate measurements of binding energy and charge radii. In this letter, we explore the parameter space of relativistic and non-relativistic models to study the differences between the form factors of the electric and weak charge distributions, $ΔF=F_{ch}-F_{w}$, in $^{48}$Ca and $^{208}$Pb. We show, for the first time, the parts of the mean-field models that are the most important in determining the relative magnitude of the neutron skin in lead and calcium nuclei. We carefully disentangle the tension between the PREX/CREX constraints and the ability of the RMF and Skyrme models to accurately describe binding energies and charge radii. We find that the nuclear symmetry energy coefficient $S_\mathrm{V}$ and the isovector spin-orbit coefficient $b'_4$ play different roles in determining $ΔF$ of $^{48}$Ca and $^{208}$Pb. Consequently, adjusting $S_\mathrm{V}$ or $b'_4$ shifts predicted $ΔF$ values toward or away from PREX/CREX measurements. Additionally, $S_\mathrm{V}$ and the slope L are marginally correlated given the constraints of our Bayesian inference, allowing us to infer them separately from PREX and CREX data. △ Less

Submitted 7 June, 2024; originally announced June 2024.

Comments: 5 pages, 3 figures in the letter, 11 pages, 8 figures in the supplemental material

Principles of Designing Robust Remote Face Anti-Spoofing Systems

Authors: Xiang Xu, Tianchen Zhao, Zheng Zhang, Zhihua Li, Jon Wu, Alessandro Achille, Mani Srivastava

Abstract: Protecting digital identities of human face from various attack vectors is paramount, and face anti-spoofing plays a crucial role in this endeavor. Current approaches primarily focus on detecting spoofing attempts within individual frames to detect presentation attacks. However, the emergence of hyper-realistic generative models capable of real-time operation has heightened the risk of digitally g… ▽ More Protecting digital identities of human face from various attack vectors is paramount, and face anti-spoofing plays a crucial role in this endeavor. Current approaches primarily focus on detecting spoofing attempts within individual frames to detect presentation attacks. However, the emergence of hyper-realistic generative models capable of real-time operation has heightened the risk of digitally generated attacks. In light of these evolving threats, this paper aims to address two key aspects. First, it sheds light on the vulnerabilities of state-of-the-art face anti-spoofing methods against digital attacks. Second, it presents a comprehensive taxonomy of common threats encountered in face anti-spoofing systems. Through a series of experiments, we demonstrate the limitations of current face anti-spoofing detection techniques and their failure to generalize to novel digital attack scenarios. Notably, the existing models struggle with digital injection attacks including adversarial noise, realistic deepfake attacks, and digital replay attacks. To aid in the design and implementation of robust face anti-spoofing systems resilient to these emerging vulnerabilities, the paper proposes key design principles from model accuracy and robustness to pipeline robustness and even platform robustness. Especially, we suggest to implement the proactive face anti-spoofing system using active sensors to significant reduce the risks for unseen attack vectors and improve the user experience. △ Less

Submitted 5 June, 2024; originally announced June 2024.

Comments: Under review

Adaptive Preference Scaling for Reinforcement Learning with Human Feedback

Authors: Ilgee Hong, Zichong Li, Alexander Bukharin, Yixiao Li, Haoming Jiang, Tianbao Yang, Tuo Zhao

Abstract: Reinforcement learning from human feedback (RLHF) is a prevalent approach to align AI systems with human values by learning rewards from human preference data. Due to various reasons, however, such data typically takes the form of rankings over pairs of trajectory segments, which fails to capture the varying strengths of preferences across different pairs. In this paper, we propose a novel adaptiv… ▽ More Reinforcement learning from human feedback (RLHF) is a prevalent approach to align AI systems with human values by learning rewards from human preference data. Due to various reasons, however, such data typically takes the form of rankings over pairs of trajectory segments, which fails to capture the varying strengths of preferences across different pairs. In this paper, we propose a novel adaptive preference loss, underpinned by distributionally robust optimization (DRO), designed to address this uncertainty in preference strength. By incorporating an adaptive scaling parameter into the loss for each pair, our method increases the flexibility of the reward function. Specifically, it assigns small scaling parameters to pairs with ambiguous preferences, leading to more comparable rewards, and large scaling parameters to those with clear preferences for more distinct rewards. Computationally, our proposed loss function is strictly convex and univariate with respect to each scaling parameter, enabling its efficient optimization through a simple second-order algorithm. Our method is versatile and can be readily adapted to various preference optimization frameworks, including direct preference optimization (DPO). Our experiments with robotic control and natural language generation with large language models (LLMs) show that our method not only improves policy performance but also aligns reward function selection more closely with policy optimization, simplifying the hyperparameter tuning process. △ Less

Submitted 4 June, 2024; originally announced June 2024.

ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation

Authors: Tianchen Zhao, Tongcheng Fang, Enshu Liu, Rui Wan, Widyadewi Soedarmadji, Shiyao Li, Zinan Lin, Guohao Dai, Shengen Yan, Huazhong Yang, Xuefei Ning, Yu Wang

Abstract: Diffusion transformers (DiTs) have exhibited remarkable performance in visual generation tasks, such as generating realistic images or videos based on textual instructions. However, larger model sizes and multi-frame processing for video generation lead to increased computational and memory costs, posing challenges for practical deployment on edge devices. Post-Training Quantization (PTQ) is an ef… ▽ More Diffusion transformers (DiTs) have exhibited remarkable performance in visual generation tasks, such as generating realistic images or videos based on textual instructions. However, larger model sizes and multi-frame processing for video generation lead to increased computational and memory costs, posing challenges for practical deployment on edge devices. Post-Training Quantization (PTQ) is an effective method for reducing memory costs and computational complexity. When quantizing diffusion transformers, we find that applying existing diffusion quantization methods designed for U-Net faces challenges in preserving quality. After analyzing the major challenges for quantizing diffusion transformers, we design an improved quantization scheme: "ViDiT-Q": Video and Image Diffusion Transformer Quantization) to address these issues. Furthermore, we identify highly sensitive layers and timesteps hinder quantization for lower bit-widths. To tackle this, we improve ViDiT-Q with a novel metric-decoupled mixed-precision quantization method (ViDiT-Q-MP). We validate the effectiveness of ViDiT-Q across a variety of text-to-image and video models. While baseline quantization methods fail at W8A8 and produce unreadable content at W4A8, ViDiT-Q achieves lossless W8A8 quantization. ViDiTQ-MP achieves W4A8 with negligible visual quality degradation, resulting in a 2.5x memory optimization and a 1.5x latency speedup. △ Less

Submitted 30 June, 2024; v1 submitted 4 June, 2024; originally announced June 2024.

Comments: Project Page: https://a-suozhang.xyz/viditq.github.io/

AGALE: A Graph-Aware Continual Learning Evaluation Framework

Authors: Tianqi Zhao, Alan Hanjalic, Megha Khosla

Abstract: In recent years, continual learning (CL) techniques have made significant progress in learning from streaming data while preserving knowledge across sequential tasks, particularly in the realm of euclidean data. To foster fair evaluation and recognize challenges in CL settings, several evaluation frameworks have been proposed, focusing mainly on the single- and multi-label classification task on e… ▽ More In recent years, continual learning (CL) techniques have made significant progress in learning from streaming data while preserving knowledge across sequential tasks, particularly in the realm of euclidean data. To foster fair evaluation and recognize challenges in CL settings, several evaluation frameworks have been proposed, focusing mainly on the single- and multi-label classification task on euclidean data. However, these evaluation frameworks are not trivially applicable when the input data is graph-structured, as they do not consider the topological structure inherent in graphs. Existing continual graph learning (CGL) evaluation frameworks have predominantly focussed on single-label scenarios in the node classification (NC) task. This focus has overlooked the complexities of multi-label scenarios, where nodes may exhibit affiliations with multiple labels, simultaneously participating in multiple tasks. We develop a graph-aware evaluation (\agale) framework that accommodates both single-labeled and multi-labeled nodes, addressing the limitations of previous evaluation frameworks. In particular, we define new incremental settings and devise data partitioning algorithms tailored to CGL datasets. We perform extensive experiments comparing methods from the domains of continual learning, continual graph learning, and dynamic graph learning (DGL). We theoretically analyze \agale and provide new insights about the role of homophily in the performance of compared methods. We release our framework at https://github.com/Tianqi-py/AGALE. △ Less

Submitted 7 June, 2024; v1 submitted 3 June, 2024; originally announced June 2024.

Leveraging Large Language Models for Entity Matching

Authors: Qianyu Huang, Tongfang Zhao

Abstract: Entity matching (EM) is a critical task in data integration, aiming to identify records across different datasets that refer to the same real-world entities. Traditional methods often rely on manually engineered features and rule-based systems, which struggle with diverse and unstructured data. The emergence of Large Language Models (LLMs) such as GPT-4 offers transformative potential for EM, leve… ▽ More Entity matching (EM) is a critical task in data integration, aiming to identify records across different datasets that refer to the same real-world entities. Traditional methods often rely on manually engineered features and rule-based systems, which struggle with diverse and unstructured data. The emergence of Large Language Models (LLMs) such as GPT-4 offers transformative potential for EM, leveraging their advanced semantic understanding and contextual capabilities. This vision paper explores the application of LLMs to EM, discussing their advantages, challenges, and future research directions. Additionally, we review related work on applying weak supervision and unsupervised approaches to EM, highlighting how LLMs can enhance these methods. △ Less

Submitted 31 May, 2024; originally announced May 2024.

Absolutely monotonic functions related to the asymptotic formula for the complete elliptic integral of the first kind

Authors: Tiehong Zhao, Zhen-Hang Yang

Abstract: Let $\mathcal{K}\left( x\right) $ be the complete elliptic integral of the first kind and \begin{equation*} \mathcal{G}_{p}\left( x\right) =e^{\mathcal{K}\left( \sqrt{x} \right) }-\frac{p}{\sqrt{1-x}} \end{equation*} for $p\in \mathbb{R}$ and $x\in \left( 0,1\right) $. In this paper we find the necessary and sufficient conditions for the functions… ▽ More Let $\mathcal{K}\left( x\right) $ be the complete elliptic integral of the first kind and \begin{equation*} \mathcal{G}_{p}\left( x\right) =e^{\mathcal{K}\left( \sqrt{x} \right) }-\frac{p}{\sqrt{1-x}} \end{equation*} for $p\in \mathbb{R}$ and $x\in \left( 0,1\right) $. In this paper we find the necessary and sufficient conditions for the functions $\pm \mathcal{G} _{p}^{\left( k\right) }\left( x\right) $ ($k\in \mathbb{N\cup }\left\{ 0\right\} $) to be absolutely monotonic on $\left( 0,1\right) $, which extend previous known results and yield several new functional inequalities involving the complete elliptic integral of the first kind. More importantly, we provide a new method to deal with those absolute monotonicity problem by proving the monotonicity of a sequence generated by the coefficients of the power series of $\mathcal{G}_{p}\left( x\right) $. △ Less

Submitted 29 May, 2024; originally announced May 2024.

Comments: 14 pages

MSC Class: Primary 33E05; 26A48 Secondary 40A05; 41A10

PathReasoner: Modeling Reasoning Path with Equivalent Extension for Logical Question Answering

Authors: Fangzhi Xu, Qika Lin, Tianzhe Zhao, Jiawei Han, Jun Liu

Abstract: Logical reasoning task has attracted great interest since it was proposed. Faced with such a task, current competitive models, even large language models (e.g., ChatGPT and PaLM 2), still perform badly. Previous promising LMs struggle in logical consistency modeling and logical structure perception. To this end, we model the logical reasoning task by transforming each logical sample into reasoning… ▽ More Logical reasoning task has attracted great interest since it was proposed. Faced with such a task, current competitive models, even large language models (e.g., ChatGPT and PaLM 2), still perform badly. Previous promising LMs struggle in logical consistency modeling and logical structure perception. To this end, we model the logical reasoning task by transforming each logical sample into reasoning paths and propose an architecture \textbf{PathReasoner}. It addresses the task from the views of both data and model. To expand the diversity of the logical samples, we propose an atom extension strategy supported by equivalent logical formulas, to form new reasoning paths. From the model perspective, we design a stack of transformer-style blocks. In particular, we propose a path-attention module to joint model in-atom and cross-atom relations with the high-order diffusion strategy. Experiments show that PathReasoner achieves competitive performances on two logical reasoning benchmarks and great generalization abilities. △ Less

Submitted 29 May, 2024; originally announced May 2024.

Comments: Accepted by ACL 2024

MixDQ: Memory-Efficient Few-Step Text-to-Image Diffusion Models with Metric-Decoupled Mixed Precision Quantization

Authors: Tianchen Zhao, Xuefei Ning, Tongcheng Fang, Enshu Liu, Guyue Huang, Zinan Lin, Shengen Yan, Guohao Dai, Yu Wang

Abstract: Diffusion models have achieved significant visual generation quality. However, their significant computational and memory costs pose challenge for their application on resource-constrained mobile devices or even desktop GPUs. Recent few-step diffusion models reduces the inference time by reducing the denoising steps. However, their memory consumptions are still excessive. The Post Training Quantiz… ▽ More Diffusion models have achieved significant visual generation quality. However, their significant computational and memory costs pose challenge for their application on resource-constrained mobile devices or even desktop GPUs. Recent few-step diffusion models reduces the inference time by reducing the denoising steps. However, their memory consumptions are still excessive. The Post Training Quantization (PTQ) replaces high bit-width FP representation with low-bit integer values (INT4/8) , which is an effective and efficient technique to reduce the memory cost. However, when applying to few-step diffusion models, existing quantization methods face challenges in preserving both the image quality and text alignment. To address this issue, we propose an mixed-precision quantization framework - MixDQ. Firstly, We design specialized BOS-aware quantization method for highly sensitive text embedding quantization. Then, we conduct metric-decoupled sensitivity analysis to measure the sensitivity of each layer. Finally, we develop an integer-programming-based method to conduct bit-width allocation. While existing quantization methods fall short at W8A8, MixDQ could achieve W8A8 without performance loss, and W4A8 with negligible visual degradation. Compared with FP16, we achieve 3-4x reduction in model size and memory cost, and 1.45x latency speedup. △ Less

Submitted 29 May, 2024; v1 submitted 28 May, 2024; originally announced May 2024.

Comments: Project Page: https://a-suozhang.xyz/mixdq.github.io/

JUNO Sensitivity to Invisible Decay Modes of Neutrons

Authors: JUNO Collaboration, Angel Abusleme, Thomas Adam, Kai Adamowicz, Shakeel Ahmad, Rizwan Ahmed, Sebastiano Aiello, Fengpeng An, Qi An, Giuseppe Andronico, Nikolay Anfimov, Vito Antonelli, Tatiana Antoshkina, João Pedro Athayde Marcondes de André, Didier Auguste, Weidong Bai, Nikita Balashov, Wander Baldini, Andrea Barresi, Davide Basilico, Eric Baussan, Marco Bellato, Marco Beretta, Antonio Bergnoli, Daniel Bick , et al. (635 additional authors not shown)

Abstract: We explore the bound neutrons decay into invisible particles (e.g., $n\rightarrow 3 ν$ or $nn \rightarrow 2 ν$) in the JUNO liquid scintillator detector. The invisible decay includes two decay modes: $ n \rightarrow { inv} $ and $ nn \rightarrow { inv} $. The invisible decays of $s$-shell neutrons in $^{12}{\rm C}$ will leave a highly excited residual nucleus. Subsequently, some de-excitation mode… ▽ More We explore the bound neutrons decay into invisible particles (e.g., $n\rightarrow 3 ν$ or $nn \rightarrow 2 ν$) in the JUNO liquid scintillator detector. The invisible decay includes two decay modes: $ n \rightarrow { inv} $ and $ nn \rightarrow { inv} $. The invisible decays of $s$-shell neutrons in $^{12}{\rm C}$ will leave a highly excited residual nucleus. Subsequently, some de-excitation modes of the excited residual nuclei can produce a time- and space-correlated triple coincidence signal in the JUNO detector. Based on a full Monte Carlo simulation informed with the latest available data, we estimate all backgrounds, including inverse beta decay events of the reactor antineutrino $\barν_e$, natural radioactivity, cosmogenic isotopes and neutral current interactions of atmospheric neutrinos. Pulse shape discrimination and multivariate analysis techniques are employed to further suppress backgrounds. With two years of exposure, JUNO is expected to give an order of magnitude improvement compared to the current best limits. After 10 years of data taking, the JUNO expected sensitivities at a 90% confidence level are $τ/B( n \rightarrow { inv} ) > 5.0 \times 10^{31} \, {\rm yr}$ and $τ/B( nn \rightarrow { inv} ) > 1.4 \times 10^{32} \, {\rm yr}$. △ Less

Submitted 27 May, 2024; originally announced May 2024.

Comments: 28 pages, 7 figures, 4 tables

SIAVC: Semi-Supervised Framework for Industrial Accident Video Classification

Authors: Zuoyong Li, Qinghua Lin, Haoyi Fan, Tiesong Zhao, David Zhang

Abstract: Semi-supervised learning suffers from the imbalance of labeled and unlabeled training data in the video surveillance scenario. In this paper, we propose a new semi-supervised learning method called SIAVC for industrial accident video classification. Specifically, we design a video augmentation module called the Super Augmentation Block (SAB). SAB adds Gaussian noise and randomly masks video frames… ▽ More Semi-supervised learning suffers from the imbalance of labeled and unlabeled training data in the video surveillance scenario. In this paper, we propose a new semi-supervised learning method called SIAVC for industrial accident video classification. Specifically, we design a video augmentation module called the Super Augmentation Block (SAB). SAB adds Gaussian noise and randomly masks video frames according to historical loss on the unlabeled data for model optimization. Then, we propose a Video Cross-set Augmentation Module (VCAM) to generate diverse pseudo-label samples from the high-confidence unlabeled samples, which alleviates the mismatch of sampling experience and provides high-quality training data. Additionally, we construct a new industrial accident surveillance video dataset with frame-level annotation, namely ECA9, to evaluate our proposed method. Compared with the state-of-the-art semi-supervised learning based methods, SIAVC demonstrates outstanding video classification performance, achieving 88.76\% and 89.13\% accuracy on ECA9 and Fire Detection datasets, respectively. The source code and the constructed dataset ECA9 will be released in \url{https://github.com/AlchemyEmperor/SIAVC}. △ Less

Submitted 23 May, 2024; originally announced May 2024.

BiomedParse: a biomedical foundation model for image parsing of everything everywhere all at once

Authors: Theodore Zhao, Yu Gu, Jianwei Yang, Naoto Usuyama, Ho Hin Lee, Tristan Naumann, Jianfeng Gao, Angela Crabtree, Jacob Abel, Christine Moung-Wen, Brian Piening, Carlo Bifulco, Mu Wei, Hoifung Poon, Sheng Wang

Abstract: Biomedical image analysis is fundamental for biomedical discovery in cell biology, pathology, radiology, and many other biomedical domains. Holistic image analysis comprises interdependent subtasks such as segmentation, detection, and recognition of relevant objects. Here, we propose BiomedParse, a biomedical foundation model for imaging parsing that can jointly conduct segmentation, detection, an… ▽ More Biomedical image analysis is fundamental for biomedical discovery in cell biology, pathology, radiology, and many other biomedical domains. Holistic image analysis comprises interdependent subtasks such as segmentation, detection, and recognition of relevant objects. Here, we propose BiomedParse, a biomedical foundation model for imaging parsing that can jointly conduct segmentation, detection, and recognition for 82 object types across 9 imaging modalities. Through joint learning, we can improve accuracy for individual tasks and enable novel applications such as segmenting all relevant objects in an image through a text prompt, rather than requiring users to laboriously specify the bounding box for each object. We leveraged readily available natural-language labels or descriptions accompanying those datasets and use GPT-4 to harmonize the noisy, unstructured text information with established biomedical object ontologies. We created a large dataset comprising over six million triples of image, segmentation mask, and textual description. On image segmentation, we showed that BiomedParse is broadly applicable, outperforming state-of-the-art methods on 102,855 test image-mask-label triples across 9 imaging modalities (everything). On object detection, which aims to locate a specific object of interest, BiomedParse again attained state-of-the-art performance, especially on objects with irregular shapes (everywhere). On object recognition, which aims to identify all objects in a given image along with their semantic types, we showed that BiomedParse can simultaneously segment and label all biomedical objects in an image (all at once). In summary, BiomedParse is an all-in-one tool for biomedical image analysis by jointly solving segmentation, detection, and recognition for all major biomedical image modalities, paving the path for efficient and accurate image-based biomedical discovery. △ Less

Submitted 4 June, 2024; v1 submitted 21 May, 2024; originally announced May 2024.

Comments: Project page: https://aka.ms/biomedparse-project