Showing 201–250 of 3,570 results for author: Ma, X

Improved measurement of the branching fraction of $h_{c}\rightarrowγη^\prime/η$ and search for $h_{c}\rightarrowγπ^0$

Authors: BESIII Collaboration, M. Ablikim, M. N. Achasov, P. Adlarson, O. Afedulidis, X. C. Ai, R. Aliberti, A. Amoroso, Q. An, Y. Bai, O. Bakina, I. Balossino, Y. Ban, H. -R. Bao, V. Batozskaya, K. Begzsuren, N. Berger, M. Berlowski, M. Bertani, D. Bettoni, F. Bianchi, E. Bianco, A. Bortone, I. Boyko, R. A. Briere , et al. (645 additional authors not shown)

Abstract: The processes $h_c\toγP(P = η^\prime,~η,~π^0)$ are studied with a sample of $(27.12\pm0.14)\times10^{8}$ $ψ(3686)$ events collected by the BESIII detector at the BEPCII collider. The decay $h_{c}\rightarrowγη$ is observed for the first time with the significance of $9.0\,σ$, and the branching fraction is determined to be $(3.77\pm0.55\pm0.13\pm0.26)\times10^{-4}$, while… ▽ More The processes $h_c\toγP(P = η^\prime,~η,~π^0)$ are studied with a sample of $(27.12\pm0.14)\times10^{8}$ $ψ(3686)$ events collected by the BESIII detector at the BEPCII collider. The decay $h_{c}\rightarrowγη$ is observed for the first time with the significance of $9.0\,σ$, and the branching fraction is determined to be $(3.77\pm0.55\pm0.13\pm0.26)\times10^{-4}$, while $\mathscr{B}(h_{c}\rightarrowγη^\prime)$ is measured to be $(1.40\pm0.11\pm0.04\pm0.10)\times10^{-3}$, where the first uncertainties are statistical, the second systematic, and the third from the branching fraction of $ψ(3686)\rightarrowπ^{0}h_c$. The combination of these results allows for a precise determination of $R_{h_c}=\frac{\mathscr{B}(h_c\rightarrowγη)}{\mathscr{B}(h_c\rightarrowγη^\prime)}$, which is calculated to be $(27.0\pm4.4\pm1.0)\%$. The results are valuable for gaining a deeper understanding of $η-η^\prime$ mixing, and its manifestation within quantum chromodynamics. No significant signal is found for the decay $h_c\rightarrowγπ^{0}$, and an upper limit is placed on its branching fraction of $\mathscr{B}(h_c\rightarrowγπ^{0})<5.0\times10^{-5}$, at the 90$\%$ confidence level. △ Less

Submitted 26 July, 2024; v1 submitted 19 May, 2024; originally announced May 2024.

Journal ref: J. High Energ. Phys. 08, 180 (2024)

Unifying 3D Vision-Language Understanding via Promptable Queries

Authors: Ziyu Zhu, Zhuofan Zhang, Xiaojian Ma, Xuesong Niu, Yixin Chen, Baoxiong Jia, Zhidong Deng, Siyuan Huang, Qing Li

Abstract: A unified model for 3D vision-language (3D-VL) understanding is expected to take various scene representations and perform a wide range of tasks in a 3D scene. However, a considerable gap exists between existing methods and such a unified model, due to the independent application of representation and insufficient exploration of 3D multi-task training. In this paper, we introduce PQ3D, a unified m… ▽ More A unified model for 3D vision-language (3D-VL) understanding is expected to take various scene representations and perform a wide range of tasks in a 3D scene. However, a considerable gap exists between existing methods and such a unified model, due to the independent application of representation and insufficient exploration of 3D multi-task training. In this paper, we introduce PQ3D, a unified model capable of using Promptable Queries to tackle a wide range of 3D-VL tasks, from low-level instance segmentation to high-level reasoning and planning. This is achieved through three key innovations: (1) unifying various 3D scene representations (i.e., voxels, point clouds, multi-view images) into a shared 3D coordinate space by segment-level grouping, (2) an attention-based query decoder for task-specific information retrieval guided by prompts, and (3) universal output heads for different tasks to support multi-task training. Tested across ten diverse 3D-VL datasets, PQ3D demonstrates impressive performance on these tasks, setting new records on most benchmarks. Particularly, PQ3D improves the state-of-the-art on ScanNet200 by 4.9% (AP25), ScanRefer by 5.4% ([email protected]), Multi3DRefer by 11.7% ([email protected]), and Scan2Cap by 13.4% ([email protected]). Moreover, PQ3D supports flexible inference with individual or combined forms of available 3D representations, e.g., solely voxel input. △ Less

Submitted 24 July, 2024; v1 submitted 19 May, 2024; originally announced May 2024.

Comments: ECCV 2024. Project page: https://pq3d.github.io

Domain Generalization for Zero-calibration BCIs with Knowledge Distillation-based Phase Invariant Feature Extraction

Authors: Zilin Liang, Zheng Zheng, Weihai Chen, Xinzhi Ma, Zhongcai Pei, Xiantao Sun

Abstract: The distribution shift of electroencephalography (EEG) data causes poor generalization of braincomputer interfaces (BCIs) in unseen domains. Some methods try to tackle this challenge by collecting a portion of user data for calibration. However, it is time-consuming, mentally fatiguing, and user-unfriendly. To achieve zerocalibration BCIs, most studies employ domain generalization (DG) techniques… ▽ More The distribution shift of electroencephalography (EEG) data causes poor generalization of braincomputer interfaces (BCIs) in unseen domains. Some methods try to tackle this challenge by collecting a portion of user data for calibration. However, it is time-consuming, mentally fatiguing, and user-unfriendly. To achieve zerocalibration BCIs, most studies employ domain generalization (DG) techniques to learn invariant features across different domains in the training set. However, they fail to fully explore invariant features within the same domain, leading to limited performance. In this paper, we present an novel method to learn domain-invariant features from both interdomain and intra-domain perspectives. For intra-domain invariant features, we propose a knowledge distillation framework to extract EEG phase-invariant features within one domain. As for inter-domain invariant features, correlation alignment is used to bridge distribution gaps across multiple domains. Experimental results on three public datasets validate the effectiveness of our method, showcasing stateof-the-art performance. To the best of our knowledge, this is the first domain generalization study that exploit Fourier phase information as an intra-domain invariant feature to facilitate EEG generalization. More importantly, the zerocalibration BCI based on inter- and intra-domain invariant features has significant potential to advance the practical applications of BCIs in real world. △ Less

Submitted 17 May, 2024; originally announced May 2024.

Hi-GMAE: Hierarchical Graph Masked Autoencoders

Authors: Chuang Liu, Zelin Yao, Yibing Zhan, Xueqi Ma, Dapeng Tao, Jia Wu, Wenbin Hu, Shirui Pan, Bo Du

Abstract: Graph Masked Autoencoders (GMAEs) have emerged as a notable self-supervised learning approach for graph-structured data. Existing GMAE models primarily focus on reconstructing node-level information, categorizing them as single-scale GMAEs. This methodology, while effective in certain contexts, tends to overlook the complex hierarchical structures inherent in many real-world graphs. For instance,… ▽ More Graph Masked Autoencoders (GMAEs) have emerged as a notable self-supervised learning approach for graph-structured data. Existing GMAE models primarily focus on reconstructing node-level information, categorizing them as single-scale GMAEs. This methodology, while effective in certain contexts, tends to overlook the complex hierarchical structures inherent in many real-world graphs. For instance, molecular graphs exhibit a clear hierarchical organization in the form of the atoms-functional groups-molecules structure. Hence, the inability of single-scale GMAE models to incorporate these hierarchical relationships often leads to their inadequate capture of crucial high-level graph information, resulting in a noticeable decline in performance. To address this limitation, we propose Hierarchical Graph Masked AutoEncoders (Hi-GMAE), a novel multi-scale GMAE framework designed to handle the hierarchical structures within graphs. First, Hi-GMAE constructs a multi-scale graph hierarchy through graph pooling, enabling the exploration of graph structures across different granularity levels. To ensure masking uniformity of subgraphs across these scales, we propose a novel coarse-to-fine strategy that initiates masking at the coarsest scale and progressively back-projects the mask to the finer scales. Furthermore, we integrate a gradual recovery strategy with the masking process to mitigate the learning challenges posed by completely masked subgraphs. Diverging from the standard graph neural network (GNN) used in GMAE models, Hi-GMAE modifies its encoder and decoder into hierarchical structures. This entails using GNN at the finer scales for detailed local graph analysis and employing a graph transformer at coarser scales to capture global information. Our experiments on 15 graph datasets consistently demonstrate that Hi-GMAE outperforms 17 state-of-the-art self-supervised competitors. △ Less

Submitted 17 May, 2024; originally announced May 2024.

Comments: 10 pages, 6 figures, 3 tables

When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models

Authors: Xianzheng Ma, Yash Bhalgat, Brandon Smart, Shuai Chen, Xinghui Li, Jian Ding, Jindong Gu, Dave Zhenyu Chen, Songyou Peng, Jia-Wang Bian, Philip H Torr, Marc Pollefeys, Matthias Nießner, Ian D Reid, Angel X. Chang, Iro Laina, Victor Adrian Prisacariu

Abstract: As large language models (LLMs) evolve, their integration with 3D spatial data (3D-LLMs) has seen rapid progress, offering unprecedented capabilities for understanding and interacting with physical spaces. This survey provides a comprehensive overview of the methodologies enabling LLMs to process, understand, and generate 3D data. Highlighting the unique advantages of LLMs, such as in-context lear… ▽ More As large language models (LLMs) evolve, their integration with 3D spatial data (3D-LLMs) has seen rapid progress, offering unprecedented capabilities for understanding and interacting with physical spaces. This survey provides a comprehensive overview of the methodologies enabling LLMs to process, understand, and generate 3D data. Highlighting the unique advantages of LLMs, such as in-context learning, step-by-step reasoning, open-vocabulary capabilities, and extensive world knowledge, we underscore their potential to significantly advance spatial comprehension and interaction within embodied Artificial Intelligence (AI) systems. Our investigation spans various 3D data representations, from point clouds to Neural Radiance Fields (NeRFs). It examines their integration with LLMs for tasks such as 3D scene understanding, captioning, question-answering, and dialogue, as well as LLM-based agents for spatial reasoning, planning, and navigation. The paper also includes a brief review of other methods that integrate 3D and language. The meta-analysis presented in this paper reveals significant progress yet underscores the necessity for novel approaches to harness the full potential of 3D-LLMs. Hence, with this paper, we aim to chart a course for future research that explores and expands the capabilities of 3D-LLMs in understanding and interacting with the complex 3D world. To support this survey, we have established a project page where papers related to our topic are organized and listed: https://github.com/ActiveVisionLab/Awesome-LLM-3D. △ Less

Submitted 16 May, 2024; originally announced May 2024.

Adaptive Ensemble Control for Stochastic Systems with Mixed Asymmetric Laplace Noises

Authors: Yajie Yu, Xuehui Ma, Shiliang Zhang, Zhuzhu Wang, Xubing Shi, Yushuai Li, Tingwen Huang

Abstract: This paper presents an adaptive ensemble control for stochastic systems subject to asymmetric noises and outliers. Asymmetric noises skew system observations, and outliers with large amplitude deteriorate the observations even further. Such disturbances induce poor system estimation and degraded stochastic system control. In this work, we model the asymmetric noises and outliers by mixed asymmetri… ▽ More This paper presents an adaptive ensemble control for stochastic systems subject to asymmetric noises and outliers. Asymmetric noises skew system observations, and outliers with large amplitude deteriorate the observations even further. Such disturbances induce poor system estimation and degraded stochastic system control. In this work, we model the asymmetric noises and outliers by mixed asymmetric Laplace distributions (ALDs), and propose an optimal control for stochastic systems with mixed ALD noises. Particularly, we segregate the system disturbed by mixed ALD noises into subsystems, each of which is subject to a specific ALD noise. For each subsystem, we design an iterative quantile filter (IQF) to estimate the system parameters using system observations. With the estimated parameters by IQF, we derive the certainty equivalence (CE) control law for each subsystem. Then we use the Bayesian approach to ensemble the subsystem CE controllers, with each of the controllers weighted by their posterior probability. We finalize our control law as the weighted sum of the control signals by the sub-system CE controllers. To demonstrate our approach, we conduct numerical simulations and Monte Carlo analyses. The results show improved tracking performance by our approach for skew noises and its robustness to outliers, compared with single ALD based and RLS-based control policy. △ Less

Submitted 16 May, 2024; originally announced May 2024.

ODFormer: Semantic Fundus Image Segmentation Using Transformer for Optic Nerve Head Detection

Authors: Jiayi Wang, Yi-An Mao, Xiaoyu Ma, Sicen Guo, Yuting Shao, Xiao Lv, Wenting Han, Mark Christopher, Linda M. Zangwill, Yanlong Bi, Rui Fan

Abstract: Optic nerve head (ONH) detection has been a crucial area of study in ophthalmology for years. However, the significant discrepancy between fundus image datasets, each generated using a single type of fundus camera, poses challenges to the generalizability of ONH detection approaches developed based on semantic segmentation networks. Despite the numerous recent advancements in general-purpose seman… ▽ More Optic nerve head (ONH) detection has been a crucial area of study in ophthalmology for years. However, the significant discrepancy between fundus image datasets, each generated using a single type of fundus camera, poses challenges to the generalizability of ONH detection approaches developed based on semantic segmentation networks. Despite the numerous recent advancements in general-purpose semantic segmentation methods using convolutional neural networks (CNNs) and Transformers, there is currently a lack of benchmarks for these state-of-the-art (SoTA) networks specifically trained for ONH detection. Therefore, in this article, we make contributions from three key aspects: network design, the publication of a dataset, and the establishment of a comprehensive benchmark. Our newly developed ONH detection network, referred to as ODFormer, is based upon the Swin Transformer architecture and incorporates two novel components: a multi-scale context aggregator and a lightweight bidirectional feature recalibrator. Our published large-scale dataset, known as TongjiU-DROD, provides multi-resolution fundus images for each participant, captured using two distinct types of cameras. Our established benchmark involves three datasets: DRIONS-DB, DRISHTI-GS1, and TongjiU-DROD, created by researchers from different countries and containing fundus images captured from participants of diverse races and ages. Extensive experimental results demonstrate that our proposed ODFormer outperforms other state-of-the-art (SoTA) networks in terms of performance and generalizability. Our dataset and source code are publicly available at mias.group/ODFormer. △ Less

Submitted 2 June, 2024; v1 submitted 15 April, 2024; originally announced May 2024.

Search for the leptonic decays $D^{*+}\to e^+ν_e$ and $D^{*+}\to μ^+ν_μ$

Authors: BESIII Collaboration, M. Ablikim, M. N. Achasov, P. Adlarson, M. Albrecht, R. Aliberti, A. Amoroso, M. R. An, Q. An, Y. Bai, O. Bakina, R. Baldini Ferroli, I. Balossino, Y. Ban, V. Batozskaya, D. Becker, K. Begzsuren, N. Berger, M. Bertani, D. Bettoni, F. Bianchi, E. Bianco, J. Bloms, A. Bortone, I. Boyko , et al. (559 additional authors not shown)

Abstract: We present the first search for the leptonic decays $D^{*+}\to e^+ν_e$ and $D^{*+}\to μ^+ν_μ$ by analyzing a data sample of electron-positron collisions recorded with the BESIII detector at center-of-mass energies between 4.178 and 4.226 GeV, corresponding to an integrated luminosity of 6.32~fb$^{-1}$. No significant signal is observed. The upper limits on the branching fractions for… ▽ More We present the first search for the leptonic decays $D^{*+}\to e^+ν_e$ and $D^{*+}\to μ^+ν_μ$ by analyzing a data sample of electron-positron collisions recorded with the BESIII detector at center-of-mass energies between 4.178 and 4.226 GeV, corresponding to an integrated luminosity of 6.32~fb$^{-1}$. No significant signal is observed. The upper limits on the branching fractions for $D^{*+}\to e^+ν_e$ and $D^{*+}\to μ^+ν_μ$ are set to be $1.1 \times 10^{-5}$ and $4.3 \times 10^{-6}$ at 90\% confidence level, respectively. △ Less

Submitted 14 May, 2024; originally announced May 2024.

Comments: 14 pages, 7 figures

An Efficient Phase-field Framework for Contact Dynamics between Deformable Solids in Fluid Flow

Authors: Biswajeet Rath, Xiaoyu Mao, Rajeev K. Jaiman

Abstract: Elastic contact in hydrodynamic environments is a complex multiphysics phenomenon and can be found in applications ranging from engineering to biological systems. Understanding the intricacies of this problem requires the development of a generalized framework capable of handling topological changes and transitioning implicitly from fluid-structure interaction (FSI) conditions to solid-solid conta… ▽ More Elastic contact in hydrodynamic environments is a complex multiphysics phenomenon and can be found in applications ranging from engineering to biological systems. Understanding the intricacies of this problem requires the development of a generalized framework capable of handling topological changes and transitioning implicitly from fluid-structure interaction (FSI) conditions to solid-solid contact conditions. We propose a mono-field interface advancing method for handling multibody contact simulations in a submerged environment. Given the physical demands of the problem, we adopt a phase-field based fully Eulerian approach to resolve the multiphase and multibody interactions in the system. We employ a stabilized finite element formulation and a partitioned iterative procedure to solve the unified momentum equation comprising solid and fluid dynamics coupled with the Allen-Cahn phase-field equation. We introduce a contact force approach to handle smooth elastic-elastic and elastic-rigid contact based on the overlap of the diffused interfaces of two colliding bodies. The diffuse interface representation ameliorates the issue of discontinuous contact forces generally found in conventional approaches. The implemented model is verified for smooth dry contact and FSI contact scenarios. We obtain a spatial convergence rate of around $1.8$ for the Hertzian contact case involving a deformable cylinder on an elastic plane. Relative volume error is found to be within $5\%$ for the FSI contact case. The developed approach eliminates the need for solving multiple phase-field equations or reconstructing the solid boundaries at every time step. This reduces the overall computational time by nearly $10\%$ compared to an explicit interface extraction method. Using the proposed framework, we demonstrate the collision dynamics between multiple bodies submerged in an open tank. △ Less

Submitted 9 May, 2024; originally announced May 2024.

Comments: 32 pages

Search for the radiative transition $χ_{c1}(3872)\toγψ_2(3823)$

Authors: BESIII Collaboration, M. Ablikim, M. N. Achasov, P. Adlarson, O. Afedulidis, X. C. Ai, R. Aliberti, A. Amoroso, M. R. An, Q. An, Y. Bai, O. Bakina, I. Balossino, Y. Ban, H. -R. Bao, V. Batozskaya, K. Begzsuren, N. Berger, M. Berlowski, M. Bertani, D. Bettoni, F. Bianchi, E. Bianco, A. Bortone, I. Boyko , et al. (635 additional authors not shown)

Abstract: Using 9.0 $\rm fb^{-1}$ of $e^+e^-$ collision data collected at center-of-mass energies from 4.178 to 4.278 GeV with the BESIII detector at the BEPCII collider, we perform the first search for the radiative transition $χ_{c1}(3872)\toγψ_2(3823)$. No $χ_{c1}(3872)\toγψ_2(3823)$ signal is observed. The upper limit on the ratio of branching fractions… ▽ More Using 9.0 $\rm fb^{-1}$ of $e^+e^-$ collision data collected at center-of-mass energies from 4.178 to 4.278 GeV with the BESIII detector at the BEPCII collider, we perform the first search for the radiative transition $χ_{c1}(3872)\toγψ_2(3823)$. No $χ_{c1}(3872)\toγψ_2(3823)$ signal is observed. The upper limit on the ratio of branching fractions $\mathcal{B}(χ_{c1}(3872)\toγψ_2(3823), ψ_2(3823)\toγχ_{c1})/\mathcal{B}(χ_{c1}(3872)\toπ^+π^- J/ψ)$ is set as 0.075 at the 90\% confidence level. Our result contradicts theoretical predictions under the assumption that the $χ_{c1}(3872)$ is the pure charmonium state $χ_{c1}(2P)$. △ Less

Submitted 13 May, 2024; originally announced May 2024.

Comments: 8 pages, 2 figures

Discovery of Very-high-energy Gamma-ray Emissions from the Low Luminosity AGN NGC 4278 by LHAASO

Authors: Zhen Cao, F. Aharonian, Q. An, Axikegu, Y. X. Bai, Y. W. Bao, D. Bastieri, X. J. Bi, Y. J. Bi, J. T. Cai, Q. Cao, W. Y. Cao, Zhe Cao, J. Chang, J. F. Chang, A. M. Chen, E. S. Chen, Liang Chen, Lin Chen, Long Chen, M. J. Chen, M. L. Chen, Q. H. Chen, S. H. Chen, S. Z. Chen , et al. (255 additional authors not shown)

Abstract: The first source catalog of Large High Altitude Air Shower Observatory reported the detection of a very-high-energy gamma ray source, 1LHAASO J1219+2915. In this paper a further detailed study of the spectral and temporal behavior of this point-like source have been carried. The best-fit position of the TeV source ($\rm{RA}=185.05^{\circ}\pm0.04^{\circ}$, $\rm{Dec}=29.25^{\circ}\pm0.03^{\circ}$) i… ▽ More The first source catalog of Large High Altitude Air Shower Observatory reported the detection of a very-high-energy gamma ray source, 1LHAASO J1219+2915. In this paper a further detailed study of the spectral and temporal behavior of this point-like source have been carried. The best-fit position of the TeV source ($\rm{RA}=185.05^{\circ}\pm0.04^{\circ}$, $\rm{Dec}=29.25^{\circ}\pm0.03^{\circ}$) is compatible with NGC 4278 within $\sim0.03$ degree. Variation analysis shows an indication of the variability at a few months level in the TeV band, which is consistent with low frequency observations. Based on these observations, we report the detection of TeV $γ$-ray emissions from this low-luminosity AGN NGC 4278. The observations by LHAASO-WCDA during active period has a significance level of 8.8\,$σ$ with best-fit photon spectral index $\varGamma=2.56\pm0.14$ and a flux $f_{1-10\,\rm{TeV}}=(7.0\pm1.1_{\rm{sta}}\pm0.35_{\rm{syst}})\times10^{-13}\,\rm{photons\,cm^{-2}\,s^{-1}}$, or approximately $5\%$ of the Crab Nebula. The discovery of VHE from NGC 4278 indicates that the compact, weak radio jet can efficiently accelerate particles and emit TeV photons. △ Less

Submitted 13 May, 2024; originally announced May 2024.

Comments: 11 pages, 5 figures

China's plug-in hybrid electric vehicle transition: an operational carbon perspective

Authors: Yanqiao Deng, Minda Ma, Nan Zhou, Zhili Ma, Ran Yan, Xin Ma

Abstract: Assessing the emissions of plug-in hybrid electric vehicle (PHEV) operations is crucial for accelerating the carbon-neutral transition in the passenger car sector. This study is the first to adopt a bottom-up model to measure the real-world energy use and carbon dioxide emissions of China's top twenty selling PHEV models across different regions from 2020 to 2022. The results indicate that (1) the… ▽ More Assessing the emissions of plug-in hybrid electric vehicle (PHEV) operations is crucial for accelerating the carbon-neutral transition in the passenger car sector. This study is the first to adopt a bottom-up model to measure the real-world energy use and carbon dioxide emissions of China's top twenty selling PHEV models across different regions from 2020 to 2022. The results indicate that (1) the actual electricity intensity of the best-selling PHEV models (20.2-38.2 kWh/100 km) was 30-40% higher than the New European Driving Cycle values, and the actual gasoline intensity (4.7-23.5 L/100 km) was 3-6 times greater than the New European Driving Cycle values. (2) The overall energy use of the best-selling models varied among different regions, and the energy use from 2020 to 2022 in Southern China was double that Northern China and the Yangtze River Middle Reach. (3) The top-selling models emitted 4.7 megatons of carbon dioxide nationwide from 2020 to 2022, with 1.9 megatons released by electricity consumption and 2.8 megatons released by gasoline combustion. Furthermore, targeted policy implications for expediting the carbon-neutral transition within the passenger car sector are proposed. In essence, this study explores and compares benchmark data at both the national and regional levels, along with performance metrics associated with PHEV operations. The main objective is to aid nationwide decarbonization efforts, focusing on carbon reduction and promoting the rapid transition of road transportation toward a net-zero carbon future. △ Less

Submitted 19 August, 2024; v1 submitted 12 May, 2024; originally announced May 2024.

Comments: 42 pages, 8 figures

Large-dimensional Robust Factor Analysis with Group Structure

Authors: Yong He, Xiaoyang Ma, Xingheng Wang, Yalin Wang

Abstract: In this paper, we focus on exploiting the group structure for large-dimensional factor models, which captures the homogeneous effects of common factors on individuals within the same group. In view of the fact that datasets in macroeconomics and finance are typically heavy-tailed, we propose to identify the unknown group structure using the agglomerative hierarchical clustering algorithm and an in… ▽ More In this paper, we focus on exploiting the group structure for large-dimensional factor models, which captures the homogeneous effects of common factors on individuals within the same group. In view of the fact that datasets in macroeconomics and finance are typically heavy-tailed, we propose to identify the unknown group structure using the agglomerative hierarchical clustering algorithm and an information criterion with the robust two-step (RTS) estimates as initial values. The loadings and factors are then re-estimated conditional on the identified groups. Theoretically, we demonstrate the consistency of the estimators for both group membership and the number of groups determined by the information criterion. Under finite second moment condition, we provide the convergence rate for the newly estimated factor loadings with group information, which are shown to achieve efficiency gains compared to those obtained without group structure information. Numerical simulations and real data analysis demonstrate the nice finite sample performance of our proposed approach in the presence of both group structure and heavy-tailedness. △ Less

Submitted 11 May, 2024; originally announced May 2024.

Semantic and Spatial Adaptive Pixel-level Classifier for Semantic Segmentation

Authors: Xiaowen Ma, Zhenliang Ni, Xinghao Chen

Abstract: Vanilla pixel-level classifiers for semantic segmentation are based on a certain paradigm, involving the inner product of fixed prototypes obtained from the training set and pixel features in the test image. This approach, however, encounters significant limitations, i.e., feature deviation in the semantic domain and information loss in the spatial domain. The former struggles with large intra-cla… ▽ More Vanilla pixel-level classifiers for semantic segmentation are based on a certain paradigm, involving the inner product of fixed prototypes obtained from the training set and pixel features in the test image. This approach, however, encounters significant limitations, i.e., feature deviation in the semantic domain and information loss in the spatial domain. The former struggles with large intra-class variance among pixel features from different images, while the latter fails to utilize the structured information of semantic objects effectively. This leads to blurred mask boundaries as well as a deficiency of fine-grained recognition capability. In this paper, we propose a novel Semantic and Spatial Adaptive (SSA) classifier to address the above challenges. Specifically, we employ the coarse masks obtained from the fixed prototypes as a guide to adjust the fixed prototype towards the center of the semantic and spatial domains in the test image. The adapted prototypes in semantic and spatial domains are then simultaneously considered to accomplish classification decisions. In addition, we propose an online multi-domain distillation learning strategy to improve the adaption process. Experimental results on three publicly available benchmarks show that the proposed SSA significantly improves the segmentation performance of the baseline models with only a minimal increase in computational cost. Code is available at https://github.com/xwmaxwma/SSA. △ Less

Submitted 10 May, 2024; originally announced May 2024.

Measurement of the ${e}^{+}{e}^{-}\to p \bar{p}π^{0}$ cross section at $\sqrt{s}=2.1000-3.0800$ GeV

Authors: BESIII Collaboration, M. Ablikim, M. N. Achasov, P. Adlarson, O. Afedulidis, X. C. Ai, R. Aliberti, A. Amoroso, Q. An, Y. Bai, O. Bakina, I. Balossino, Y. Ban, H. -R. Bao, V. Batozskaya, K. Begzsuren, N. Berger, M. Berlowski, M. Bertani, D. Bettoni, F. Bianchi, E. Bianco, A. Bortone, I. Boyko, R. A. Briere , et al. (639 additional authors not shown)

Abstract: The process $e^{+}e^{-}\to p\bar{p}π^{0}$ is studied at 20 center-of-mass energies ranging from 2.1000 to 3.0800 GeV using 636.8 pb$^{-1}$ of data collected with the BESIII detector operating at the BEPCII collider. The Born cross sections for $e^{+}e^{-}\to p\bar{p}π^{0}$ are measured with high precision. Since the lowest center-of-mass energy, 2.1000 GeV, is less than 90 MeV above the… ▽ More The process $e^{+}e^{-}\to p\bar{p}π^{0}$ is studied at 20 center-of-mass energies ranging from 2.1000 to 3.0800 GeV using 636.8 pb$^{-1}$ of data collected with the BESIII detector operating at the BEPCII collider. The Born cross sections for $e^{+}e^{-}\to p\bar{p}π^{0}$ are measured with high precision. Since the lowest center-of-mass energy, 2.1000 GeV, is less than 90 MeV above the $p\bar{p}π^0$ energy threshold, we can probe the threshold behavior for this reaction. However, no anomalous threshold enhancement is found in the cross sections for $e^{+}e^{-}\to p\bar{p}π^{0}$. △ Less

Submitted 10 May, 2024; originally announced May 2024.

A note on the $8π$ problem of Jäger-Luckhaus system

Authors: Xuan Mao, Yuxiang Li

Abstract: It is proven that for any radially symmetric, nonnegative and continuous initial data with critical mass $8π$, Jäger-Luckhuas system, posed on the unit disk, admits a global classical solution, which is uniform-in-time bounded. It is proven that for any radially symmetric, nonnegative and continuous initial data with critical mass $8π$, Jäger-Luckhuas system, posed on the unit disk, admits a global classical solution, which is uniform-in-time bounded. △ Less

Submitted 10 May, 2024; originally announced May 2024.

Special Characters Attack: Toward Scalable Training Data Extraction From Large Language Models

Authors: Yang Bai, Ge Pei, Jindong Gu, Yong Yang, Xingjun Ma

Abstract: Large language models (LLMs) have achieved remarkable performance on a wide range of tasks. However, recent studies have shown that LLMs can memorize training data and simple repeated tokens can trick the model to leak the data. In this paper, we take a step further and show that certain special characters or their combinations with English letters are stronger memory triggers, leading to more sev… ▽ More Large language models (LLMs) have achieved remarkable performance on a wide range of tasks. However, recent studies have shown that LLMs can memorize training data and simple repeated tokens can trick the model to leak the data. In this paper, we take a step further and show that certain special characters or their combinations with English letters are stronger memory triggers, leading to more severe data leakage. The intuition is that, since LLMs are trained with massive data that contains a substantial amount of special characters (e.g. structural symbols {, } of JSON files, and @, # in emails and online posts), the model may memorize the co-occurrence between these special characters and the raw texts. This motivates us to propose a simple but effective Special Characters Attack (SCA) to induce training data leakage. Our experiments verify the high effectiveness of SCA against state-of-the-art LLMs: they can leak diverse training data, such as code corpus, web pages, and personally identifiable information, and sometimes generate non-stop outputs as a byproduct. We further show that the composition of the training data corpus can be revealed by inspecting the leaked data -- one crucial piece of information for pre-training high-performance LLMs. Our work can help understand the sensitivity of LLMs to special characters and identify potential areas for improvement. △ Less

Submitted 20 May, 2024; v1 submitted 8 May, 2024; originally announced May 2024.

Achieving millisecond coherence fluxonium through overlap Josephson junctions

Authors: Fei Wang, Kannan Lu, Huijuan Zhan, Lu Ma, Feng Wu, Hantao Sun, Hao Deng, Yang Bai, Feng Bao, Xu Chang, Ran Gao, Xun Gao, Guicheng Gong, Lijuan Hu, Ruizi Hu, Honghong Ji, Xizheng Ma, Liyong Mao, Zhijun Song, Chengchun Tang, Hongcheng Wang, Tenghui Wang, Ziang Wang, Tian Xia, Hongxin Xu , et al. (10 additional authors not shown)

Abstract: Fluxonium qubits are recognized for their high coherence times and high operation fidelities, attributed to their unique design incorporating over 100 Josephson junctions per superconducting loop. However, this complexity poses significant fabrication challenges, particularly in achieving high yield and junction uniformity with traditional methods. Here, we introduce an overlap process for Josephs… ▽ More Fluxonium qubits are recognized for their high coherence times and high operation fidelities, attributed to their unique design incorporating over 100 Josephson junctions per superconducting loop. However, this complexity poses significant fabrication challenges, particularly in achieving high yield and junction uniformity with traditional methods. Here, we introduce an overlap process for Josephson junction fabrication that achieves nearly 100% yield and maintains uniformity across a 2-inch wafer with less than 5% variation for the phase slip junction and less than 2% for the junction array. Our compact junction array design facilitates fluxonium qubits with energy relaxation times exceeding 1 millisecond at the flux frustration point, demonstrating consistency with state-of-the-art dielectric loss tangents and flux noise across multiple devices. This work suggests the scalability of high coherence fluxonium processors using CMOS-compatible processes, marking a significant step towards practical quantum computing. △ Less

Submitted 8 May, 2024; originally announced May 2024.

Revisiting general dark matter-bound-electron interactions

Authors: Jin-Han Liang, Yi Liao, Xiao-Dong Ma, Hao-Lin Wang

Abstract: In this letter we revisit general dark matter (DM)-bound-electron interactions studied previously in the influential work of [Catena et al., Phys. Rev. Res. 2, 033195 (2020)]. We derive the DM-electron response functions and find a crucial minus sign was missed for the second atomic response function $W_2$ defined in that work. The minus sign has significant phenomenological consequences when expl… ▽ More In this letter we revisit general dark matter (DM)-bound-electron interactions studied previously in the influential work of [Catena et al., Phys. Rev. Res. 2, 033195 (2020)]. We derive the DM-electron response functions and find a crucial minus sign was missed for the second atomic response function $W_2$ defined in that work. The minus sign has significant phenomenological consequences when explaining experimental bounds on specific DM scenarios. Furthermore, for the most general DM-electron nonrelativistic or relativistic interactions for DM with spin up to one, we find there are three DM response functions ($a_{0,1,2}$) whose corresponding atomic response functions ($\widetilde W_{0,1,2}$) are linear combinations of the four response functions ($W_{1,2,3,4}$) given in that work, $$ \widetilde W_0 = W_1, \, \widetilde W_2 = |\mathbf{v}_0^\perp|^2 W_1+ W_3 - 2 {m_e\, \mathbf{q}\cdot \mathbf{v}_0^\perp \over \mathbf{q}^2} W_2,\, \widetilde W_3 = { (\mathbf{q}\cdot \mathbf{v}_0^\perp)^2 \over \mathbf{q}^2} W_1 + {m_e^2 \over \mathbf{q}^2}W_4 - 2 {m_e\, \mathbf{q}\cdot \mathbf{v}_0^\perp \over \mathbf{q}^2} W_2. $$ Due to the minus sign correction for $W_2$, there can be significant cancellations between the $W_2$ and $W_{3,4}$ terms, so that $\widetilde W_{2,3}$ are dominated by the usual response function $W_1$ in some cases. Ignoring the sign could thus result in misinterpretation of the experimental data in some DM scenarios. As an example, we show that the recent XENON1T constraint on the fermionic DM anapole moment is weakened by a factor of 2 or so. Many DM scenarios involving DM or electron axial-vector current can yield $W_2$ and thus are potentially affected by the sign. △ Less

Submitted 8 May, 2024; originally announced May 2024.

Comments: 6+2 pages, 4 figures

Machine learning aided parameter analysis in Perovskite X-ray Detector

Authors: Bobo Zhang, Endai Huang, Xinyi Du, Xiaokang Ma, Lu Zhang, Jiaxue You, Alex K. Y. Jen, Shengzhong, Liu

Abstract: Many factors in perovskite X-ray detectors, such as crystal lattice and carrier dynamics, determine the final device performance (e.g., sensitivity and detection limit). However, the relationship between these factors remains unknown due to the complexity of the material. In this study, we employ machine learning to reveal the relationship between 15 intrinsic properties of halide perovskite mater… ▽ More Many factors in perovskite X-ray detectors, such as crystal lattice and carrier dynamics, determine the final device performance (e.g., sensitivity and detection limit). However, the relationship between these factors remains unknown due to the complexity of the material. In this study, we employ machine learning to reveal the relationship between 15 intrinsic properties of halide perovskite materials and their device performance. We construct a database of X-ray detectors for the training of machine learning. The results show that the band gap is mainly influenced by the atomic number of the B-site metal, and the lattice length parameter b has the greatest impact on the carrier mobility-lifetime product (μτ). An X-ray detector (m-F-PEA)2PbI4 were generated in the experiment and it further verified the accuracy of our ML models. We suggest further study on random forest regression for X-ray detector applications. △ Less

Submitted 7 May, 2024; originally announced May 2024.

Comments: 20 pages

Cross-Language Dependencies: An Empirical Study of Kotlin-Java

Authors: Qiong Feng, Huan Ji, Xiaotian Ma, Peng Liang

Abstract: Background: Since Google introduced Kotlin as an official programming language for developing Android apps in 2017, Kotlin has gained widespread adoption in Android development. The inter-operability of Java and Kotlin's design nature allows them to coexist and interact with each other smoothly within a project. Aims: However, there is limited research on how Java and Kotlin interact with each oth… ▽ More Background: Since Google introduced Kotlin as an official programming language for developing Android apps in 2017, Kotlin has gained widespread adoption in Android development. The inter-operability of Java and Kotlin's design nature allows them to coexist and interact with each other smoothly within a project. Aims: However, there is limited research on how Java and Kotlin interact with each other in real-world projects and what challenges are faced during these interactions. The answers to these questions are key to understanding these kinds of cross-language software systems. Methods: In this paper, we implemented a tool named DependEx-tractor, which can extract 11 kinds of Kotlin-Java dependencies, and conducted an empirical study of 23 Kotlin-Java real-world projects with 3,227 Java and 8,630 Kotlin source files. Results: Our findings revealed that Java and Kotlin frequently interact with each other in these cross-language projects, with access and call dependency types being the most dominant. Compared to files interacting with other files in the same language, Java/Kotlin source files, which participate in the cross-language interactions, undergo more commits. Additionally, among all Kotlin-Java problematic interactions, we identified seven common mistakes, along with their fixing strategies. Conclusions: The findings of this study can help developers understand and address the challenges in Kotlin-Java projects. △ Less

Submitted 26 July, 2024; v1 submitted 7 May, 2024; originally announced May 2024.

Comments: The 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)

On the Influence of Data Resampling for Deep Learning-Based Log Anomaly Detection: Insights and Recommendations

Authors: Xiaoxue Ma, Huiqi Zou, Jacky Keung, Pinjia He, Yishu Li, Xiao Yu, Federica Sarro

Abstract: Numerous DL-based approaches have garnered considerable attention in the field of software Log Anomaly Detection. However, a practical challenge persists: the class imbalance in the public data commonly used to train the DL models. This imbalance is characterized by a substantial disparity in the number of abnormal log sequences compared to normal ones, for example, anomalies represent less than 1… ▽ More Numerous DL-based approaches have garnered considerable attention in the field of software Log Anomaly Detection. However, a practical challenge persists: the class imbalance in the public data commonly used to train the DL models. This imbalance is characterized by a substantial disparity in the number of abnormal log sequences compared to normal ones, for example, anomalies represent less than 1% of one of the most popular datasets. Previous research has indicated that existing DLLAD approaches may exhibit unsatisfactory performance, particularly when confronted with datasets featuring severe class imbalances. Mitigating class imbalance through data resampling has proven effective for other software engineering tasks, however, it has been unexplored for LAD thus far. This study aims to fill this gap by providing an in-depth analysis of the impact of diverse data resampling methods on existing DLLAD approaches from two distinct perspectives. Firstly, we assess the performance of these DLLAD approaches across three datasets and explore the impact of resampling ratios of normal to abnormal data on ten data resampling methods. Secondly, we evaluate the effectiveness of the data resampling methods when utilizing optimal resampling ratios of normal to abnormal data. Our findings indicate that oversampling methods generally outperform undersampling and hybrid methods. Data resampling on raw data yields superior results compared to data resampling in the feature space. In most cases, certain undersampling and hybrid methods show limited effectiveness. Additionally, by exploring the resampling ratio of normal to abnormal data, we suggest generating more data for minority classes through oversampling while removing less data from majority classes through undersampling. In conclusion, our study provides valuable insights into the intricate relationship between data resampling methods and DLLAD. △ Less

Submitted 6 May, 2024; originally announced May 2024.

Comments: 15 pages, 2 figures

DexSkills: Skill Segmentation Using Haptic Data for Learning Autonomous Long-Horizon Robotic Manipulation Tasks

Authors: Xiaofeng Mao, Gabriele Giudici, Claudio Coppola, Kaspar Althoefer, Ildar Farkhatdinov, Zhibin Li, Lorenzo Jamone

Abstract: Effective execution of long-horizon tasks with dexterous robotic hands remains a significant challenge in real-world problems. While learning from human demonstrations have shown encouraging results, they require extensive data collection for training. Hence, decomposing long-horizon tasks into reusable primitive skills is a more efficient approach. To achieve so, we developed DexSkills, a novel s… ▽ More Effective execution of long-horizon tasks with dexterous robotic hands remains a significant challenge in real-world problems. While learning from human demonstrations have shown encouraging results, they require extensive data collection for training. Hence, decomposing long-horizon tasks into reusable primitive skills is a more efficient approach. To achieve so, we developed DexSkills, a novel supervised learning framework that addresses long-horizon dexterous manipulation tasks using primitive skills. DexSkills is trained to recognize and replicate a select set of skills using human demonstration data, which can then segment a demonstrated long-horizon dexterous manipulation task into a sequence of primitive skills to achieve one-shot execution by the robot directly. Significantly, DexSkills operates solely on proprioceptive and tactile data, i.e., haptic data. Our real-world robotic experiments show that DexSkills can accurately segment skills, thereby enabling autonomous robot execution of a diverse range of tasks. △ Less

Submitted 6 May, 2024; originally announced May 2024.

A Universal List Decoding Algorithm with Application to Decoding of Polar Codes

Authors: Xiangping Zheng, Xiao Ma

Abstract: This paper is concerned with a guessing codeword decoding (GCD) of linear block codes. Compared with the guessing noise decoding (GND), which is only efficient for high-rate codes, the GCD is efficient for not only high-rate codes but also low-rate codes. We prove that the GCD typically requires a fewer number of queries than the GND. Compared with the ordered statistics decoding (OSD), the GCD do… ▽ More This paper is concerned with a guessing codeword decoding (GCD) of linear block codes. Compared with the guessing noise decoding (GND), which is only efficient for high-rate codes, the GCD is efficient for not only high-rate codes but also low-rate codes. We prove that the GCD typically requires a fewer number of queries than the GND. Compared with the ordered statistics decoding (OSD), the GCD does not require the online Gaussian elimination (GE). In addition to limiting the maximum number of searches, we suggest limiting the radius of searches in terms of soft weights or tolerated performance loss to further reduce the decoding complexity, resulting in the so-called truncated GCD. The performance gap between the truncated GCD and the optimal decoding can be upper bounded approximately by the saddlepoint approach or other numerical approaches. The derived upper bound captures the relationship between the performance and the decoding parameters, enabling us to balance the performance and the complexity by optimizing the decoding parameters of the truncated GCD. We also introduce a parallel implementation of the (truncated) GCD algorithm to reduce decoding latency without compromising performance. Another contribution of this paper is the application of the GCD to the polar codes. We propose a multiple-bit-wise decoding algorithm over a pruned tree for the polar codes, referred to as the successive-cancellation list (SCL) decoding algorithm by GCD. First, we present a strategy for pruning the conventional polar decoding tree based on the complexity analysis rather than the specific bit patterns. Then we apply the GCD algorithm in parallel aided by the early stopping criteria to the leaves of the pruned tree. Simulation results show that, without any performance loss as justified by analysis, the proposed decoding algorithm can significantly reduce the decoding latency of the polar codes. △ Less

Submitted 6 May, 2024; originally announced May 2024.

Comments: 47 pages, 24 figures

DeepMpMRI: Tensor-decomposition Regularized Learning for Fast and High-Fidelity Multi-Parametric Microstructural MR Imaging

Authors: Wenxin Fan, Jian Cheng, Cheng Li, Xinrui Ma, Jing Yang, Juan Zou, Ruoyou Wu, Zan Chen, Yuanjing Feng, Hairong Zheng, Shanshan Wang

Abstract: Deep learning has emerged as a promising approach for learning the nonlinear mapping between diffusion-weighted MR images and tissue parameters, which enables automatic and deep understanding of the brain microstructures. However, the efficiency and accuracy in the multi-parametric estimations are still limited since previous studies tend to estimate multi-parametric maps with dense sampling and i… ▽ More Deep learning has emerged as a promising approach for learning the nonlinear mapping between diffusion-weighted MR images and tissue parameters, which enables automatic and deep understanding of the brain microstructures. However, the efficiency and accuracy in the multi-parametric estimations are still limited since previous studies tend to estimate multi-parametric maps with dense sampling and isolated signal modeling. This paper proposes DeepMpMRI, a unified framework for fast and high-fidelity multi-parametric estimation from various diffusion models using sparsely sampled q-space data. DeepMpMRI is equipped with a newly designed tensor-decomposition-based regularizer to effectively capture fine details by exploiting the correlation across parameters. In addition, we introduce a Nesterov-based adaptive learning algorithm that optimizes the regularization parameter dynamically to enhance the performance. DeepMpMRI is an extendable framework capable of incorporating flexible network architecture. Experimental results demonstrate the superiority of our approach over 5 state-of-the-art methods in simultaneously estimating multi-parametric maps for various diffusion models with fine-grained details both quantitatively and qualitatively, achieving 4.5 - 22.5$\times$ acceleration compared to the dense sampling of a total of 270 diffusion gradients. △ Less

Submitted 6 May, 2024; originally announced May 2024.

I$^3$Net: Inter-Intra-slice Interpolation Network for Medical Slice Synthesis

Authors: Haofei Song, Xintian Mao, Jing Yu, Qingli Li, Yan Wang

Abstract: Medical imaging is limited by acquisition time and scanning equipment. CT and MR volumes, reconstructed with thicker slices, are anisotropic with high in-plane resolution and low through-plane resolution. We reveal an intriguing phenomenon that due to the mentioned nature of data, performing slice-wise interpolation from the axial view can yield greater benefits than performing super-resolution fr… ▽ More Medical imaging is limited by acquisition time and scanning equipment. CT and MR volumes, reconstructed with thicker slices, are anisotropic with high in-plane resolution and low through-plane resolution. We reveal an intriguing phenomenon that due to the mentioned nature of data, performing slice-wise interpolation from the axial view can yield greater benefits than performing super-resolution from other views. Based on this observation, we propose an Inter-Intra-slice Interpolation Network (I$^3$Net), which fully explores information from high in-plane resolution and compensates for low through-plane resolution. The through-plane branch supplements the limited information contained in low through-plane resolution from high in-plane resolution and enables continual and diverse feature learning. In-plane branch transforms features to the frequency domain and enforces an equal learning opportunity for all frequency bands in a global context learning paradigm. We further propose a cross-view block to take advantage of the information from all three views online. Extensive experiments on two public datasets demonstrate the effectiveness of I$^3$Net, and noticeably outperforms state-of-the-art super-resolution, video frame interpolation and slice interpolation methods by a large margin. We achieve 43.90dB in PSNR, with at least 1.14dB improvement under the upscale factor of $\times$2 on MSD dataset with faster inference. Code is available at https://github.com/DeepMed-Lab-ECNU/Medical-Image-Reconstruction. △ Less

Submitted 5 May, 2024; originally announced May 2024.

Large Language Models for Mobility in Transportation Systems: A Survey on Forecasting Tasks

Authors: Zijian Zhang, Yujie Sun, Zepu Wang, Yuqi Nie, Xiaobo Ma, Peng Sun, Ruolin Li

Abstract: Mobility analysis is a crucial element in the research area of transportation systems. Forecasting traffic information offers a viable solution to address the conflict between increasing transportation demands and the limitations of transportation infrastructure. Predicting human travel is significant in aiding various transportation and urban management tasks, such as taxi dispatch and urban plan… ▽ More Mobility analysis is a crucial element in the research area of transportation systems. Forecasting traffic information offers a viable solution to address the conflict between increasing transportation demands and the limitations of transportation infrastructure. Predicting human travel is significant in aiding various transportation and urban management tasks, such as taxi dispatch and urban planning. Machine learning and deep learning methods are favored for their flexibility and accuracy. Nowadays, with the advent of large language models (LLMs), many researchers have combined these models with previous techniques or applied LLMs to directly predict future traffic information and human travel behaviors. However, there is a lack of comprehensive studies on how LLMs can contribute to this field. This survey explores existing approaches using LLMs for mobility forecasting problems. We provide a literature review concerning the forecasting applications within transportation systems, elucidating how researchers utilize LLMs, showcasing recent state-of-the-art advancements, and identifying the challenges that must be overcome to fully leverage LLMs in this domain. △ Less

Submitted 2 May, 2024; originally announced May 2024.

Comments: 9 pages

Structural Pruning of Pre-trained Language Models via Neural Architecture Search

Authors: Aaron Klein, Jacek Golebiowski, Xingchen Ma, Valerio Perrone, Cedric Archambeau

Abstract: Pre-trained language models (PLM), for example BERT or RoBERTa, mark the state-of-the-art for natural language understanding task when fine-tuned on labeled data. However, their large size poses challenges in deploying them for inference in real-world applications, due to significant GPU memory requirements and high inference latency. This paper explores neural architecture search (NAS) for struct… ▽ More Pre-trained language models (PLM), for example BERT or RoBERTa, mark the state-of-the-art for natural language understanding task when fine-tuned on labeled data. However, their large size poses challenges in deploying them for inference in real-world applications, due to significant GPU memory requirements and high inference latency. This paper explores neural architecture search (NAS) for structural pruning to find sub-parts of the fine-tuned network that optimally trade-off efficiency, for example in terms of model size or latency, and generalization performance. We also show how we can utilize more recently developed two-stage weight-sharing NAS approaches in this setting to accelerate the search process. Unlike traditional pruning methods with fixed thresholds, we propose to adopt a multi-objective approach that identifies the Pareto optimal set of sub-networks, allowing for a more flexible and automated compression process. △ Less

Submitted 25 August, 2024; v1 submitted 3 May, 2024; originally announced May 2024.

CodeFort: Robust Training for Code Generation Models

Authors: Yuhao Zhang, Shiqi Wang, Haifeng Qian, Zijian Wang, Mingyue Shang, Linbo Liu, Sanjay Krishna Gouda, Baishakhi Ray, Murali Krishna Ramanathan, Xiaofei Ma, Anoop Deoras

Abstract: Code generation models are not robust to small perturbations, which often lead to inconsistent and incorrect generations and significantly degrade the performance of these models. Improving the robustness of code generation models is crucial to better user experience when these models are deployed in real-world applications. However, existing efforts have not addressed this issue for code generati… ▽ More Code generation models are not robust to small perturbations, which often lead to inconsistent and incorrect generations and significantly degrade the performance of these models. Improving the robustness of code generation models is crucial to better user experience when these models are deployed in real-world applications. However, existing efforts have not addressed this issue for code generation models. To fill this gap, we propose CodeFort, a framework to improve the robustness of code generation models, generalizing a large variety of code perturbations to enrich the training data and enabling various robust training strategies, mixing data augmentation, batch augmentation, adversarial logits pairing, and contrastive learning, all carefully designed to support high-throughput training. Extensive evaluations show that we improve the average robust pass rates of baseline CodeGen models from 14.79 to 21.74. Notably, the improvement in robustness against code-syntax perturbations is evidenced by a significant decrease in pass rate drop from 95.04% to 53.35% △ Less

Submitted 11 April, 2024; originally announced May 2024.

Non-overshooting sliding mode for UAV control

Authors: Xinhua Wang, Xuerui Mao

Abstract: For a class of uncertain systems, a non-overshooting sliding mode control is presented to make them globally exponentially stable and without overshoot. Even when the unknown stochastic disturbance exists, and the time-variant reference trajectory is required, the strict non-overshooting stabilization is still achieved. The control law design is based on a desired second-order sliding mode (2-slid… ▽ More For a class of uncertain systems, a non-overshooting sliding mode control is presented to make them globally exponentially stable and without overshoot. Even when the unknown stochastic disturbance exists, and the time-variant reference trajectory is required, the strict non-overshooting stabilization is still achieved. The control law design is based on a desired second-order sliding mode (2-sliding mode), which successively includes two bounded-gain subsystems. Non-overshooting stability requires that the system gains depend on the initial values of system variables. In order to obtain the global non-overshooting stability, the first subsystem with non-overshooting reachability compresses the initial values of the second subsystem to a given bounded range. By partitioning these initial values, the bounded system gains are determined to satisfy the robust non-overshooting stability. In order to reject the chattering in the controller output, a tanh-function-based sliding mode is developed for the design of smoothed non-overshooting controller. The proposed method is applied to a UAV trajectory tracking when the disturbances and uncertainties exist. The control laws are designed to implement the non-overshooting stabilization in position and attitude. Finally, the effectiveness of the proposed method is demonstrated by the flying tests. △ Less

Submitted 12 May, 2024; v1 submitted 2 May, 2024; originally announced May 2024.

Comments: 57 pages, 29 figures

Journal ref: The Aeronautical Journal, 2024

Polarization Perspectives on Hercules X-1: Further Constraining the Geometry

Authors: Qingchang Zhao, Hancheng Li, Lian Tao, Hua Feng, Shuangnan Zhang, Roland Walter, Mingyu Ge, Hao Tong, Long Ji, Liang Zhang, Jinlu Qu, Yue Huang, Xiang Ma, Shu Zhang, Qianqing Yin, Hongxing Yin, Ruican Ma, Shujie Zhao, Panping Li, Zixu Yang, Hexin Liu, Wei Yu, Yiming Huang, Zexi Li, Yajun Li , et al. (2 additional authors not shown)

Abstract: We conduct a comprehensive analysis of the accreting X-ray pulsar, Hercules X-1, utilizing data from IXPE and NuSTAR. IXPE performed five observations of Her X-1, consisting of three in the Main-on state and two in the Short-on state. Our time-resolved analysis uncovers the linear correlations between the flux and polarization degree as well as the pulse fraction and polarization degree. Geometry… ▽ More We conduct a comprehensive analysis of the accreting X-ray pulsar, Hercules X-1, utilizing data from IXPE and NuSTAR. IXPE performed five observations of Her X-1, consisting of three in the Main-on state and two in the Short-on state. Our time-resolved analysis uncovers the linear correlations between the flux and polarization degree as well as the pulse fraction and polarization degree. Geometry parameters are rigorously constrained by fitting the phase-resolved modulations of Cyclotron Resonance Scattering Feature and polarization angle with a simple dipole model and Rotating Vector Model respectively, yielding roughly consistent results. The changes of $χ_{\rm p}$ (the position angle of the pulsar's spin axis on the plane of the sky) between different Main-on observations suggest the possible forced precession of the neutron star crust. Furthermore, a linear association between the energy of Cyclotron Resonance Scattering Feature and polarization angle implies the prevalence of a dominant dipole magnetic field, and their phase-resolved modulations likely arise from viewing angle effects. △ Less

Submitted 1 May, 2024; originally announced May 2024.

Comments: Accepted for MNRAS

Hyperuniformity in phase ordering: the roles of activity, noise, and non-constant mobility

Authors: Filippo De Luca, Xiao Ma, Cesare Nardini, Michael E. Cates

Abstract: Hyperuniformity emerges generically in the coarsening regime of phase-separating fluids. Numerical studies of active and passive systems have shown that the structure factor $S(q)$ behaves as $q^ς$ for $q\to 0$, with hyperuniformity exponent $ς= 4$. For passive systems, this result was explained in 1991 by a qualitative scaling analysis of Tomita, exploiting isotropy at scales much larger than the… ▽ More Hyperuniformity emerges generically in the coarsening regime of phase-separating fluids. Numerical studies of active and passive systems have shown that the structure factor $S(q)$ behaves as $q^ς$ for $q\to 0$, with hyperuniformity exponent $ς= 4$. For passive systems, this result was explained in 1991 by a qualitative scaling analysis of Tomita, exploiting isotropy at scales much larger than the coarsening length $\ell$. Here we reconsider and extend Tomita's argument to address cases of active phase separation and of non-constant mobility, again finding $ς=4$. We further show that dynamical noise of variance $D$ creates a transient $ς= 2$ regime for $\hat q\ll \hat{q}_\ast \sim \sqrt{D} t^{[1-(d+2)ν]/2}$, crossing over to $ς= 4$ at larger $\hat{q}$. Here, $ν$ is the coarsening exponent, with $\ell\sim t^ν$, and $\hat{q} \propto q \ell$ is the rescaled wavenumber. In diffusive coarsening, $ν=1/3$, so the rescaled crossover wavevector $\hat{q}_\ast$ vanishes at large times when $d\geq 2$. The slowness of this decay suggests a natural explanation for experiments that observe a long-lived $ς= 2$ scaling in phase-separating active fluids (where noise is typically large). Conversely, in $d=1$, we demonstrate that with noise the $ς= 2$ regime survives as $t\to\infty$, with $\hat{q}_\ast\sim D^{5/6}$. (The structure factor is not then determined by the zero-temperature fixed point.) We confirm our analytical predictions by numerical simulations of active and passive continuum theories in the deterministic case and of Model B for the stochastic case. We also compare them with related findings for a system near an absorbing-state transition rather than undergoing phase separation. A central role is played throughout by the presence or absence of a conservation law for the centre of mass position of the order parameter field. △ Less

Submitted 1 May, 2024; originally announced May 2024.

Comments: 34 pages, 4 figures

Journal ref: J. Phys.: Condens. Matter 36 (2024) 405101

DiaryHelper: Exploring the Use of an Automatic Contextual Information Recording Agent for Elicitation Diary Study

Authors: Junze Li, Changyang He, Jiaxiong Hu, Boyang Jia, Alon Halevy, Xiaojuan Ma

Abstract: Elicitation diary studies, a type of qualitative, longitudinal research method, involve participants to self-report aspects of events of interest at their occurrences as memory cues for providing details and insights during post-study interviews. However, due to time constraints and lack of motivation, participants' diary entries may be vague or incomplete, impairing their later recall. To address… ▽ More Elicitation diary studies, a type of qualitative, longitudinal research method, involve participants to self-report aspects of events of interest at their occurrences as memory cues for providing details and insights during post-study interviews. However, due to time constraints and lack of motivation, participants' diary entries may be vague or incomplete, impairing their later recall. To address this challenge, we designed an automatic contextual information recording agent, DiaryHelper, based on the theory of episodic memory. DiaryHelper can predict five dimensions of contextual information and confirm with participants. We evaluated the use of DiaryHelper in both the recording period and the elicitation interview through a within-subject study (N=12) over a period of two weeks. Our results demonstrated that DiaryHelper can assist participants in capturing abundant and accurate contextual information without significant burden, leading to a more detailed recall of recorded events and providing greater insights. △ Less

Submitted 30 April, 2024; originally announced April 2024.

Comments: CHI 2024

A Minimal Set of Parameters Based Depth-Dependent Distortion Model and Its Calibration Method for Stereo Vision Systems

Authors: Xin Ma, Puchen Zhu, Xiao Li, Xiaoyin Zheng, Jianshu Zhou, Xuchen Wang, Kwok Wai Samuel Au

Abstract: Depth position highly affects lens distortion, especially in close-range photography, which limits the measurement accuracy of existing stereo vision systems. Moreover, traditional depth-dependent distortion models and their calibration methods have remained complicated. In this work, we propose a minimal set of parameters based depth-dependent distortion model (MDM), which considers the radial an… ▽ More Depth position highly affects lens distortion, especially in close-range photography, which limits the measurement accuracy of existing stereo vision systems. Moreover, traditional depth-dependent distortion models and their calibration methods have remained complicated. In this work, we propose a minimal set of parameters based depth-dependent distortion model (MDM), which considers the radial and decentering distortions of the lens to improve the accuracy of stereo vision systems and simplify their calibration process. In addition, we present an easy and flexible calibration method for the MDM of stereo vision systems with a commonly used planar pattern, which requires cameras to observe the planar pattern in different orientations. The proposed technique is easy to use and flexible compared with classical calibration techniques for depth-dependent distortion models in which the lens must be perpendicular to the planar pattern. The experimental validation of the MDM and its calibration method showed that the MDM improved the calibration accuracy by 56.55% and 74.15% compared with the Li's distortion model and traditional Brown's distortion model. Besides, an iteration-based reconstruction method is proposed to iteratively estimate the depth information in the MDM during three-dimensional reconstruction. The results showed that the accuracy of the iteration-based reconstruction method was improved by 9.08% compared with that of the non-iteration reconstruction method. △ Less

Submitted 1 May, 2024; v1 submitted 29 April, 2024; originally announced April 2024.

Comments: This paper has been accepted for publication in IEEE Transactions on Instrumentation and Measurement

PromptReps: Prompting Large Language Models to Generate Dense and Sparse Representations for Zero-Shot Document Retrieval

Authors: Shengyao Zhuang, Xueguang Ma, Bevan Koopman, Jimmy Lin, Guido Zuccon

Abstract: Utilizing large language models (LLMs) for zero-shot document ranking is done in one of two ways: 1) prompt-based re-ranking methods, which require no further training but are only feasible for re-ranking a handful of candidate documents due to computational costs; and 2) unsupervised contrastive trained dense retrieval methods, which can retrieve relevant documents from the entire corpus but requ… ▽ More Utilizing large language models (LLMs) for zero-shot document ranking is done in one of two ways: 1) prompt-based re-ranking methods, which require no further training but are only feasible for re-ranking a handful of candidate documents due to computational costs; and 2) unsupervised contrastive trained dense retrieval methods, which can retrieve relevant documents from the entire corpus but require a large amount of paired text data for contrastive training. In this paper, we propose PromptReps, which combines the advantages of both categories: no need for training and the ability to retrieve from the whole corpus. Our method only requires prompts to guide an LLM to generate query and document representations for effective document retrieval. Specifically, we prompt the LLMs to represent a given text using a single word, and then use the last token's hidden states and the corresponding logits associated with the prediction of the next token to construct a hybrid document retrieval system. The retrieval system harnesses both dense text embedding and sparse bag-of-words representations given by the LLM. We further explore variations of this core idea that consider the generation of multiple words, and representations that rely on multiple embeddings and sparse distributions. Our experimental evaluation on the MSMARCO, TREC deep learning and BEIR zero-shot document retrieval datasets illustrates that this simple prompt-based LLM retrieval method can achieve a similar or higher retrieval effectiveness than state-of-the-art LLM embedding methods that are trained with large amounts of unsupervised data, especially when using a larger LLM. △ Less

Submitted 16 June, 2024; v1 submitted 29 April, 2024; originally announced April 2024.

3AM: An Ambiguity-Aware Multi-Modal Machine Translation Dataset

Authors: Xinyu Ma, Xuebo Liu, Derek F. Wong, Jun Rao, Bei Li, Liang Ding, Lidia S. Chao, Dacheng Tao, Min Zhang

Abstract: Multimodal machine translation (MMT) is a challenging task that seeks to improve translation quality by incorporating visual information. However, recent studies have indicated that the visual information provided by existing MMT datasets is insufficient, causing models to disregard it and overestimate their capabilities. This issue presents a significant obstacle to the development of MMT researc… ▽ More Multimodal machine translation (MMT) is a challenging task that seeks to improve translation quality by incorporating visual information. However, recent studies have indicated that the visual information provided by existing MMT datasets is insufficient, causing models to disregard it and overestimate their capabilities. This issue presents a significant obstacle to the development of MMT research. This paper presents a novel solution to this issue by introducing 3AM, an ambiguity-aware MMT dataset comprising 26,000 parallel sentence pairs in English and Chinese, each with corresponding images. Our dataset is specifically designed to include more ambiguity and a greater variety of both captions and images than other MMT datasets. We utilize a word sense disambiguation model to select ambiguous data from vision-and-language datasets, resulting in a more challenging dataset. We further benchmark several state-of-the-art MMT models on our proposed dataset. Experimental results show that MMT models trained on our dataset exhibit a greater ability to exploit visual information than those trained on other MMT datasets. Our work provides a valuable resource for researchers in the field of multimodal learning and encourages further exploration in this area. The data, code and scripts are freely available at https://github.com/MaxyLee/3AM. △ Less

Submitted 29 April, 2024; originally announced April 2024.

NTIRE 2024 Quality Assessment of AI-Generated Content Challenge

Authors: Xiaohong Liu, Xiongkuo Min, Guangtao Zhai, Chunyi Li, Tengchuan Kou, Wei Sun, Haoning Wu, Yixuan Gao, Yuqin Cao, Zicheng Zhang, Xiele Wu, Radu Timofte, Fei Peng, Huiyuan Fu, Anlong Ming, Chuanming Wang, Huadong Ma, Shuai He, Zifei Dou, Shu Chen, Huacong Zhang, Haiyi Xie, Chengwei Wang, Baoying Chen, Jishen Zeng , et al. (89 additional authors not shown)

Abstract: This paper reports on the NTIRE 2024 Quality Assessment of AI-Generated Content Challenge, which will be held in conjunction with the New Trends in Image Restoration and Enhancement Workshop (NTIRE) at CVPR 2024. This challenge is to address a major challenge in the field of image and video processing, namely, Image Quality Assessment (IQA) and Video Quality Assessment (VQA) for AI-Generated Conte… ▽ More This paper reports on the NTIRE 2024 Quality Assessment of AI-Generated Content Challenge, which will be held in conjunction with the New Trends in Image Restoration and Enhancement Workshop (NTIRE) at CVPR 2024. This challenge is to address a major challenge in the field of image and video processing, namely, Image Quality Assessment (IQA) and Video Quality Assessment (VQA) for AI-Generated Content (AIGC). The challenge is divided into the image track and the video track. The image track uses the AIGIQA-20K, which contains 20,000 AI-Generated Images (AIGIs) generated by 15 popular generative models. The image track has a total of 318 registered participants. A total of 1,646 submissions are received in the development phase, and 221 submissions are received in the test phase. Finally, 16 participating teams submitted their models and fact sheets. The video track uses the T2VQA-DB, which contains 10,000 AI-Generated Videos (AIGVs) generated by 9 popular Text-to-Video (T2V) models. A total of 196 participants have registered in the video track. A total of 991 submissions are received in the development phase, and 185 submissions are received in the test phase. Finally, 12 participating teams submitted their models and fact sheets. Some methods have achieved better results than baseline methods, and the winning methods in both tracks have demonstrated superior prediction performance on AIGC. △ Less

Submitted 7 May, 2024; v1 submitted 25 April, 2024; originally announced April 2024.

Soft X-ray prompt emission from a high-redshift gamma-ray burst EP240315a

Authors: Y. Liu, H. Sun, D. Xu, D. S. Svinkin, J. Delaunay, N. R. Tanvir, H. Gao, C. Zhang, Y. Chen, X. -F. Wu, B. Zhang, W. Yuan, J. An, G. Bruni, D. D. Frederiks, G. Ghirlanda, J. -W. Hu, A. Li, C. -K. Li, J. -D. Li, D. B. Malesani, L. Piro, G. Raman, R. Ricci, E. Troja , et al. (170 additional authors not shown)

Abstract: Long gamma-ray bursts (GRBs) are believed to originate from core collapse of massive stars. High-redshift GRBs can probe the star formation and reionization history of the early universe, but their detection remains rare. Here we report the detection of a GRB triggered in the 0.5--4 keV band by the Wide-field X-ray Telescope (WXT) on board the Einstein Probe (EP) mission, designated as EP240315a,… ▽ More Long gamma-ray bursts (GRBs) are believed to originate from core collapse of massive stars. High-redshift GRBs can probe the star formation and reionization history of the early universe, but their detection remains rare. Here we report the detection of a GRB triggered in the 0.5--4 keV band by the Wide-field X-ray Telescope (WXT) on board the Einstein Probe (EP) mission, designated as EP240315a, whose bright peak was also detected by the Swift Burst Alert Telescope and Konus-Wind through off-line analyses. At a redshift of $z=4.859$, EP240315a showed a much longer and more complicated light curve in the soft X-ray band than in gamma-rays. Benefiting from a large field-of-view ($\sim$3600 deg$^2$) and a high sensitivity, EP-WXT captured the earlier engine activation and extended late engine activity through a continuous detection. With a peak X-ray flux at the faint end of previously known high-$z$ GRBs, the detection of EP240315a demonstrates the great potential for EP to study the early universe via GRBs. △ Less

Submitted 25 April, 2024; originally announced April 2024.

Comments: 41 pages, 8 figures, 7 tables

The fast X-ray transient EP240315a: a z ~ 5 gamma-ray burst in a Lyman continuum leaking galaxy

Authors: Andrew J. Levan, Peter G. Jonker, Andrea Saccardi, Daniele Bjørn Malesani, Nial R. Tanvir, Luca Izzo, Kasper E. Heintz, Daniel Mata Sánchez, Jonathan Quirola-Vásquez, Manuel A. P. Torres, Susanna D. Vergani, Steve Schulze, Andrea Rossi, Paolo D'Avanzo, Benjamin Gompertz, Antonio Martin-Carrillo, Antonio de Ugarte Postigo, Benjamin Schneider, Weimin Yuan, Zhixing Ling, Wenjie Zhang, Xuan Mao, Yuan Liu, Hui Sun, Dong Xu , et al. (51 additional authors not shown)

Abstract: The nature of the minute-to-hour long Fast X-ray Transients (FXTs) localised by telescopes such as Chandra, Swift, and XMM-Newton remains mysterious, with numerous models suggested for the events. Here, we report multi-wavelength observations of EP240315a, a 1600 s long transient detected by the Einstein Probe, showing it to have a redshift of z=4.859. We measure a low column density of neutral hy… ▽ More The nature of the minute-to-hour long Fast X-ray Transients (FXTs) localised by telescopes such as Chandra, Swift, and XMM-Newton remains mysterious, with numerous models suggested for the events. Here, we report multi-wavelength observations of EP240315a, a 1600 s long transient detected by the Einstein Probe, showing it to have a redshift of z=4.859. We measure a low column density of neutral hydrogen, indicating that the event is embedded in a low-density environment, further supported by direct detection of leaking ionising Lyman-continuum. The observed properties are consistent with EP240315a being a long-duration gamma-ray burst, and these observations support an interpretation in which a significant fraction of the FXT population are lower-luminosity examples of similar events. Such transients are detectable at high redshifts by the Einstein Probe and, in the (near) future, out to even larger distances by SVOM, THESEUS, and Athena, providing samples of events into the epoch of reionisation. △ Less

Submitted 25 April, 2024; originally announced April 2024.

Comments: 41 pages, 7 figures, submitted

New Timing Results of MSPs from NICER Observations

Authors: Shijie Zheng, Dawei Han, Heng Xu, Kejia Lee, Jianping Yuan, Haoxi Wang, Mingyu Ge, Liang Zhang, Yongye Li, Yitao Yin, Xiang Ma, Yong Chen, Shuangnan Zhang

Abstract: Millisecond pulsars (MSPs) are known for their long-term stability. Using six years of observations from the Neutron Star Interior Composition Explorer (NICER), we have conducted an in-depth analysis of the X-ray timing results for six MSPs: PSRs B1937+21, B1821$-$24, J0437$-$4715, J0030+0451, J0218+4232, and J2124$-$3358. The timing stability parameter $σ_z$ has been calculated, revealing remarka… ▽ More Millisecond pulsars (MSPs) are known for their long-term stability. Using six years of observations from the Neutron Star Interior Composition Explorer (NICER), we have conducted an in-depth analysis of the X-ray timing results for six MSPs: PSRs B1937+21, B1821$-$24, J0437$-$4715, J0030+0451, J0218+4232, and J2124$-$3358. The timing stability parameter $σ_z$ has been calculated, revealing remarkable timing precision on the order of $10^{-14}$ for PSRs B1937+21 and J0437$-$4715, and $10^{-13}$ for PSRs B1821$-$24, J0218+4232, and J0030+0451 over a timescale of 1000 days. These findings underscore the feasibility of autonomous in-orbit timekeeping using X-ray observations of MSPs. In addition, the consistency of long-term spin-down noise in the X-ray and radio bands has been investigated by comparison with IPTA radio data. △ Less

Submitted 24 April, 2024; originally announced April 2024.

Where to Mask: Structure-Guided Masking for Graph Masked Autoencoders

Authors: Chuang Liu, Yuyao Wang, Yibing Zhan, Xueqi Ma, Dapeng Tao, Jia Wu, Wenbin Hu

Abstract: Graph masked autoencoders (GMAE) have emerged as a significant advancement in self-supervised pre-training for graph-structured data. Previous GMAE models primarily utilize a straightforward random masking strategy for nodes or edges during training. However, this strategy fails to consider the varying significance of different nodes within the graph structure. In this paper, we investigate the po… ▽ More Graph masked autoencoders (GMAE) have emerged as a significant advancement in self-supervised pre-training for graph-structured data. Previous GMAE models primarily utilize a straightforward random masking strategy for nodes or edges during training. However, this strategy fails to consider the varying significance of different nodes within the graph structure. In this paper, we investigate the potential of leveraging the graph's structural composition as a fundamental and unique prior in the masked pre-training process. To this end, we introduce a novel structure-guided masking strategy (i.e., StructMAE), designed to refine the existing GMAE models. StructMAE involves two steps: 1) Structure-based Scoring: Each node is evaluated and assigned a score reflecting its structural significance. Two distinct types of scoring manners are proposed: predefined and learnable scoring. 2) Structure-guided Masking: With the obtained assessment scores, we develop an easy-to-hard masking strategy that gradually increases the structural awareness of the self-supervised reconstruction task. Specifically, the strategy begins with random masking and progresses to masking structure-informative nodes based on the assessment scores. This design gradually and effectively guides the model in learning graph structural information. Furthermore, extensive experiments consistently demonstrate that our StructMAE method outperforms existing state-of-the-art GMAE models in both unsupervised and transfer learning tasks. Codes are available at https://github.com/LiuChuang0059/StructMAE. △ Less

Submitted 24 April, 2024; originally announced April 2024.

Comments: 9 pages, 3 Figures. Accepted by IJCAI 2024

BASS: Batched Attention-optimized Speculative Sampling

Authors: Haifeng Qian, Sujan Kumar Gonugondla, Sungsoo Ha, Mingyue Shang, Sanjay Krishna Gouda, Ramesh Nallapati, Sudipta Sengupta, Xiaofei Ma, Anoop Deoras

Abstract: Speculative decoding has emerged as a powerful method to improve latency and throughput in hosting large language models. However, most existing implementations focus on generating a single sequence. Real-world generative AI applications often require multiple responses and how to perform speculative decoding in a batched setting while preserving its latency benefits poses non-trivial challenges.… ▽ More Speculative decoding has emerged as a powerful method to improve latency and throughput in hosting large language models. However, most existing implementations focus on generating a single sequence. Real-world generative AI applications often require multiple responses and how to perform speculative decoding in a batched setting while preserving its latency benefits poses non-trivial challenges. This paper describes a system of batched speculative decoding that sets a new state of the art in multi-sequence generation latency and that demonstrates superior GPU utilization as well as quality of generations within a time budget. For example, for a 7.8B-size model on a single A100 GPU and with a batch size of 8, each sequence is generated at an average speed of 5.8ms per token, the overall throughput being 1.1K tokens per second. These results represent state-of-the-art latency and a 2.15X speed-up over optimized regular decoding. Within a time budget that regular decoding does not finish, our system is able to generate sequences with HumanEval Pass@First of 43% and Pass@All of 61%, far exceeding what's feasible with single-sequence speculative decoding. Our peak GPU utilization during decoding reaches as high as 15.8%, more than 3X the highest of that of regular decoding and around 10X of single-sequence speculative decoding. △ Less

Submitted 26 June, 2024; v1 submitted 24 April, 2024; originally announced April 2024.

Gradformer: Graph Transformer with Exponential Decay

Authors: Chuang Liu, Zelin Yao, Yibing Zhan, Xueqi Ma, Shirui Pan, Wenbin Hu

Abstract: Graph Transformers (GTs) have demonstrated their advantages across a wide range of tasks. However, the self-attention mechanism in GTs overlooks the graph's inductive biases, particularly biases related to structure, which are crucial for the graph tasks. Although some methods utilize positional encoding and attention bias to model inductive biases, their effectiveness is still suboptimal analytic… ▽ More Graph Transformers (GTs) have demonstrated their advantages across a wide range of tasks. However, the self-attention mechanism in GTs overlooks the graph's inductive biases, particularly biases related to structure, which are crucial for the graph tasks. Although some methods utilize positional encoding and attention bias to model inductive biases, their effectiveness is still suboptimal analytically. Therefore, this paper presents Gradformer, a method innovatively integrating GT with the intrinsic inductive bias by applying an exponential decay mask to the attention matrix. Specifically, the values in the decay mask matrix diminish exponentially, correlating with the decreasing node proximities within the graph structure. This design enables Gradformer to retain its ability to capture information from distant nodes while focusing on the graph's local details. Furthermore, Gradformer introduces a learnable constraint into the decay mask, allowing different attention heads to learn distinct decay masks. Such an design diversifies the attention heads, enabling a more effective assimilation of diverse structural information within the graph. Extensive experiments on various benchmarks demonstrate that Gradformer consistently outperforms the Graph Neural Network and GT baseline models in various graph classification and regression tasks. Additionally, Gradformer has proven to be an effective method for training deep GT models, maintaining or even enhancing accuracy compared to shallow models as the network deepens, in contrast to the significant accuracy drop observed in other GT models.Codes are available at \url{https://github.com/LiuChuang0059/Gradformer}. △ Less

Submitted 24 April, 2024; originally announced April 2024.

Comments: 9 pages, 7 figures. Accepted by IJCAI 2024

MalleTrain: Deep Neural Network Training on Unfillable Supercomputer Nodes

Authors: Xiaolong Ma, Feng Yan, Lei Yang, Ian Foster, Michael E. Papka, Zhengchun Liu, Rajkumar Kettimuthu

Abstract: First-come first-serve scheduling can result in substantial (up to 10%) of transiently idle nodes on supercomputers. Recognizing that such unfilled nodes are well-suited for deep neural network (DNN) training, due to the flexible nature of DNN training tasks, Liu et al. proposed that the re-scaling DNN training tasks to fit gaps in schedules be formulated as a mixed-integer linear programming (MIL… ▽ More First-come first-serve scheduling can result in substantial (up to 10%) of transiently idle nodes on supercomputers. Recognizing that such unfilled nodes are well-suited for deep neural network (DNN) training, due to the flexible nature of DNN training tasks, Liu et al. proposed that the re-scaling DNN training tasks to fit gaps in schedules be formulated as a mixed-integer linear programming (MILP) problem, and demonstrated via simulation the potential benefits of the approach. Here, we introduce MalleTrain, a system that provides the first practical implementation of this approach and that furthermore generalizes it by allowing it use even for DNN training applications for which model information is unknown before runtime. Key to this latter innovation is the use of a lightweight online job profiling advisor (JPA) to collect critical scalability information for DNN jobs -- information that it then employs to optimize resource allocations dynamically, in real time. We describe the MalleTrain architecture and present the results of a detailed experimental evaluation on a supercomputer GPU cluster and several representative DNN training workloads, including neural architecture search and hyperparameter optimization. Our results not only confirm the practical feasibility of leveraging idle supercomputer nodes for DNN training but improve significantly on prior results, improving training throughput by up to 22.3\% without requiring users to provide job scalability information. △ Less

Submitted 24 April, 2024; originally announced April 2024.

Training all-mechanical neural networks for task learning through in situ backpropagation

Authors: Shuaifeng Li, Xiaoming Mao

Abstract: Recent advances unveiled physical neural networks as promising machine learning platforms, offering faster and more energy-efficient information processing. Compared with extensively-studied optical neural networks, the development of mechanical neural networks (MNNs) remains nascent and faces significant challenges, including heavy computational demands and learning with approximate gradients. He… ▽ More Recent advances unveiled physical neural networks as promising machine learning platforms, offering faster and more energy-efficient information processing. Compared with extensively-studied optical neural networks, the development of mechanical neural networks (MNNs) remains nascent and faces significant challenges, including heavy computational demands and learning with approximate gradients. Here, we introduce the mechanical analogue of in situ backpropagation to enable highly efficient training of MNNs. We demonstrate that the exact gradient can be obtained locally in MNNs, enabling learning through their immediate vicinity. With the gradient information, we showcase the successful training of MNNs for behavior learning and machine learning tasks, achieving high accuracy in regression and classification. Furthermore, we present the retrainability of MNNs involving task-switching and damage, demonstrating the resilience. Our findings, which integrate the theory for training MNNs and experimental and numerical validations, pave the way for mechanical machine learning hardware and autonomous self-learning material systems. △ Less

Submitted 23 April, 2024; originally announced April 2024.

Comments: 25 pages, 5 figures

Advantage Distillation for Quantum Key Distribution

Authors: Zhenyu Du, Guoding Liu, Xiongfeng Ma

Abstract: Enhancing the performance of quantum key distribution is crucial, driving the exploration of various key distillation techniques to increase the key rate and tolerable error rate. It is imperative to develop a comprehensive framework to encapsulate and enhance the existing methods. In this work, we propose an advantage distillation framework for quantum key distribution. Building on the entangleme… ▽ More Enhancing the performance of quantum key distribution is crucial, driving the exploration of various key distillation techniques to increase the key rate and tolerable error rate. It is imperative to develop a comprehensive framework to encapsulate and enhance the existing methods. In this work, we propose an advantage distillation framework for quantum key distribution. Building on the entanglement distillation protocol, our framework integrates all the existing key distillation methods and offers better generalization and performance. Using classical linear codes, our framework can achieve higher key rates, particularly without one-time pad encryption for postprocessing. Our approach provides insights into existing protocols and offers a systematic way for future enhancements of quantum key distribution protocols. △ Less

Submitted 23 April, 2024; originally announced April 2024.

Comments: 24 pages, 14 figures, comments are welcome

Exploring and Unleashing the Power of Large Language Models in Automated Code Translation

Authors: Zhen Yang, Fang Liu, Zhongxing Yu, Jacky Wai Keung, Jia Li, Shuo Liu, Yifan Hong, Xiaoxue Ma, Zhi Jin, Ge Li

Abstract: Code translation tools (transpilers) are developed for automatic source-to-source translation. Although learning-based transpilers have shown impressive enhancement against rule-based counterparts, owing to their task-specific pre-training on extensive monolingual corpora. Their current performance still remains unsatisfactory for practical deployment, and the associated training resources are als… ▽ More Code translation tools (transpilers) are developed for automatic source-to-source translation. Although learning-based transpilers have shown impressive enhancement against rule-based counterparts, owing to their task-specific pre-training on extensive monolingual corpora. Their current performance still remains unsatisfactory for practical deployment, and the associated training resources are also prohibitively expensive. LLMs pre-trained on huge amounts of human-written code/text have shown remarkable performance in many code intelligence tasks due to their powerful generality, even without task-specific training. Thus, LLMs can potentially circumvent the above limitations, but they have not been exhaustively explored yet. This paper investigates diverse LLMs and learning-based transpilers for automated code translation tasks, finding that: although certain LLMs have outperformed current transpilers, they still have some accuracy issues, where most of the failures are induced by a lack of comprehension of source programs, missing clear instructions on I/O types in translation, and ignoring discrepancies between source and target programs. Enlightened by the above findings, we further propose UniTrans, a Unified code Translation framework, applicable to various LLMs, for unleashing their power in this field. Specifically, UniTrans first crafts a series of test cases for target programs with the assistance of source programs. Next, it harnesses the above auto-generated test cases to augment the code translation and then evaluate their correctness via execution. Afterward, UniTrans further (iteratively) repairs incorrectly translated programs prompted by test case execution results. Extensive experiments are conducted on six settings of translation datasets between Python, Java, and C++. Three recent LLMs of diverse sizes are tested with UniTrans, and all achieve substantial improvements. △ Less

Submitted 11 May, 2024; v1 submitted 22 April, 2024; originally announced April 2024.

Comments: 23 pages, 7 figures, accepted by FSE'24 (2024 ACM International Conference on the Foundations of Software Engineering)

An Empirical Study of LLaMA3 Quantization: From LLMs to MLLMs

Authors: Wei Huang, Xingyu Zheng, Xudong Ma, Haotong Qin, Chengtao Lv, Hong Chen, Jie Luo, Xiaojuan Qi, Xianglong Liu, Michele Magno

Abstract: The LLaMA family has become one of the most powerful open-source Large Language Models (LLMs) and the popular LLM backbones of Multimodal Large Language Models (MLLMs), widely applied in Computer Vision (CV) and Natural Language Understanding (NLU) tasks. Notably, LLaMA3 models have recently been released and achieve impressive performance across various with super-large scale pre-training on over… ▽ More The LLaMA family has become one of the most powerful open-source Large Language Models (LLMs) and the popular LLM backbones of Multimodal Large Language Models (MLLMs), widely applied in Computer Vision (CV) and Natural Language Understanding (NLU) tasks. Notably, LLaMA3 models have recently been released and achieve impressive performance across various with super-large scale pre-training on over 15T tokens of data. Given the wide application of low-bit quantization for LLMs in resource-limited scenarios, we explore LLaMA3's capabilities when quantized to low bit-width. This exploration can potentially unveil new insights and challenges for low-bit quantization of LLaMA3 and other forthcoming LLMs, especially in addressing performance degradation problems that suffer in LLM compression. Specifically, we comprehensively evaluate the 10 existing post-training quantization and LoRA-finetuning methods of LLaMA3 on 1-8 bits and diverse datasets to reveal LLaMA3's low-bit quantization performance. To uncover the capabilities of low-bit quantized MLLM, we assessed the performance of the LLaMA3-based LLaVA-Next-8B model under 2-4 ultra-low bits with post-training quantization methods. Our experimental results indicate that LLaMA3 still suffers non-negligent degradation in linguistic and visual contexts, particularly under ultra-low bit widths. This highlights the significant performance gap under low bit-width that needs to be bridged in future developments. We expect that this empirical study will prove valuable in advancing future models, driving LLMs and MLLMs to achieve higher accuracy at lower bit to enhance practicality. △ Less

Submitted 19 July, 2024; v1 submitted 22 April, 2024; originally announced April 2024.

Study of $e^+e^-\toωX(3872)$ and $γX(3872)$ from 4.66 to 4.95 GeV

Authors: BESIII Collaboration, M. Ablikim, M. N. Achasov, P. Adlarson, O. Afedulidis, X. C. Ai, R. Aliberti, A. Amoroso, Q. An, Y. Bai, O. Bakina, I. Balossino, Y. Ban, H. -R. Bao, V. Batozskaya, K. Begzsuren, N. Berger, M. Berlowski, M. Bertani, D. Bettoni, F. Bianchi, E. Bianco, A. Bortone, I. Boyko, R. A. Briere , et al. (634 additional authors not shown)

Abstract: Using data samples with an integrated luminosity of $4.5~\text{fb}^{-1}$ collected by the BESIII detector at center-of-mass energies ranging from 4.66 to 4.95 GeV, we study the processes of $e^+e^-\toωX(3872)$ and $e^+e^-\toγX(3872)$. With the $e^+e^-\toωX(3872)$ process, the branching fraction ratio $R\equiv\frac{\mathcal{B}(X(3872)\toγJ/ψ)}{\mathcal{B}(X(3872)\toπ^+π^- J/ψ)}$ is measured to be… ▽ More Using data samples with an integrated luminosity of $4.5~\text{fb}^{-1}$ collected by the BESIII detector at center-of-mass energies ranging from 4.66 to 4.95 GeV, we study the processes of $e^+e^-\toωX(3872)$ and $e^+e^-\toγX(3872)$. With the $e^+e^-\toωX(3872)$ process, the branching fraction ratio $R\equiv\frac{\mathcal{B}(X(3872)\toγJ/ψ)}{\mathcal{B}(X(3872)\toπ^+π^- J/ψ)}$ is measured to be $0.38\pm0.20_\text{stat.}\pm0.01_\text{syst.}$ ($R< 0.83$ at 90\% confidence level). In addition, we measure the ratio of the average cross section of $e^+e^-\toωX(3872)$ to $e^+e^-\toωχ_{c1}(ωχ_{c2})$ to be $σ_{ωX(3872)}/σ_{ωχ_{c1}}~(σ_{ωX(3872)}/σ_{ωχ_{c2}})=5.2\pm1.0_\text{stat.}\pm1.9_\text{syst.}~ (5.5\pm1.1_\text{stat.}\pm2.4_\text{syst.})$. Finally, we search for the process of $e^+e^-\toγX(3872)$, and no obvious signal is observed. The upper limit on the ratio of the average cross section of $e^+e^-\toγX(3872)$ to $e^+e^-\toωX(3872)$ is set as $σ_{γX(3872)}/σ_{ωX(3872)}<0.23$ at 90\% confidence level. △ Less

Submitted 13 July, 2024; v1 submitted 21 April, 2024; originally announced April 2024.

Comments: 19 pages, 10 figures

Journal ref: Phys. Rev. D 110, 012006 (2024)

Heterogeneous Subgraph Transformer for Fake News Detection

Authors: Yuchen Zhang, Xiaoxiao Ma, Jia Wu, Jian Yang, Hao Fan

Abstract: Fake news is pervasive on social media, inflicting substantial harm on public discourse and societal well-being. We investigate the explicit structural information and textual features of news pieces by constructing a heterogeneous graph concerning the relations among news topics, entities, and content. Through our study, we reveal that fake news can be effectively detected in terms of the atypica… ▽ More Fake news is pervasive on social media, inflicting substantial harm on public discourse and societal well-being. We investigate the explicit structural information and textual features of news pieces by constructing a heterogeneous graph concerning the relations among news topics, entities, and content. Through our study, we reveal that fake news can be effectively detected in terms of the atypical heterogeneous subgraphs centered on them, which encapsulate the essential semantics and intricate relations between news elements. However, suffering from the heterogeneity, exploring such heterogeneous subgraphs remains an open problem. To bridge the gap, this work proposes a heterogeneous subgraph transformer (HeteroSGT) to exploit subgraphs in our constructed heterogeneous graph. In HeteroSGT, we first employ a pre-trained language model to derive both word-level and sentence-level semantics. Then the random walk with restart (RWR) is applied to extract subgraphs centered on each news, which are further fed to our proposed subgraph Transformer to quantify the authenticity. Extensive experiments on five real-world datasets demonstrate the superior performance of HeteroSGT over five baselines. Further case and ablation studies validate our motivation and demonstrate that performance improvement stems from our specially designed components. △ Less

Submitted 19 April, 2024; originally announced April 2024.