Showing 1–50 of 1,330 results for author: He, H

NeuralOOD: Improving Out-of-Distribution Generalization Performance with Brain-machine Fusion Learning Framework

Authors: Shuangchen Zhao, Changde Du, Hui Li, Huiguang He

Abstract: Deep Neural Networks (DNNs) have demonstrated exceptional recognition capabilities in traditional computer vision (CV) tasks. However, existing CV models often suffer a significant decrease in accuracy when confronted with out-of-distribution (OOD) data. In contrast to these DNN models, human can maintain a consistently low error rate when facing OOD scenes, partly attributed to the rich prior cog… ▽ More Deep Neural Networks (DNNs) have demonstrated exceptional recognition capabilities in traditional computer vision (CV) tasks. However, existing CV models often suffer a significant decrease in accuracy when confronted with out-of-distribution (OOD) data. In contrast to these DNN models, human can maintain a consistently low error rate when facing OOD scenes, partly attributed to the rich prior cognitive knowledge stored in the human brain. Previous OOD generalization researches only focus on the single modal, overlooking the advantages of multimodal learning method. In this paper, we utilize the multimodal learning method to improve the OOD generalization and propose a novel Brain-machine Fusion Learning (BMFL) framework. We adopt the cross-attention mechanism to fuse the visual knowledge from CV model and prior cognitive knowledge from the human brain. Specially, we employ a pre-trained visual neural encoding model to predict the functional Magnetic Resonance Imaging (fMRI) from visual features which eliminates the need for the fMRI data collection and pre-processing, effectively reduces the workload associated with conventional BMFL methods. Furthermore, we construct a brain transformer to facilitate the extraction of knowledge inside the fMRI data. Moreover, we introduce the Pearson correlation coefficient maximization regularization method into the training process, which improves the fusion capability with better constrains. Our model outperforms the DINOv2 and baseline models on the ImageNet-1k validation dataset as well as six curated OOD datasets, showcasing its superior performance in diverse scenarios. △ Less

Submitted 27 August, 2024; originally announced August 2024.

Multi-SIGATnet: A multimodal schizophrenia MRI classification algorithm using sparse interaction mechanisms and graph attention networks

Authors: Yuhong Jiao, Jiaqing Miao, Jinnan Gong, Hui He, Ping Liang, Cheng Luo, Ying Tan

Abstract: Schizophrenia is a serious psychiatric disorder. Its pathogenesis is not completely clear, making it difficult to treat patients precisely. Because of the complicated non-Euclidean network structure of the human brain, learning critical information from brain networks remains difficult. To effectively capture the topological information of brain neural networks, a novel multimodal graph attention… ▽ More Schizophrenia is a serious psychiatric disorder. Its pathogenesis is not completely clear, making it difficult to treat patients precisely. Because of the complicated non-Euclidean network structure of the human brain, learning critical information from brain networks remains difficult. To effectively capture the topological information of brain neural networks, a novel multimodal graph attention network based on sparse interaction mechanism (Multi-SIGATnet) was proposed for SZ classification was proposed for SZ classification. Firstly, structural and functional information were fused into multimodal data to obtain more comprehensive and abundant features for patients with SZ. Subsequently, a sparse interaction mechanism was proposed to effectively extract salient features and enhance the feature representation capability. By enhancing the strong connections and weakening the weak connections between feature information based on an asymmetric convolutional network, high-order interactive features were captured. Moreover, sparse learning strategies were designed to filter out redundant connections to improve model performance. Finally, local and global features were updated in accordance with the topological features and connection weight constraints of the higher-order brain network, the features being projected to the classification target space for disorder classification. The effectiveness of the model is verified on the Center for Biomedical Research Excellence (COBRE) and University of California Los Angeles (UCLA) datasets, achieving 81.9\% and 75.8\% average accuracy, respectively, 4.6\% and 5.5\% higher than the graph attention network (GAT) method. Experiments showed that the Multi-SIGATnet method exhibited good performance in identifying SZ. △ Less

Submitted 25 August, 2024; originally announced August 2024.

A Law of Next-Token Prediction in Large Language Models

Authors: Hangfeng He, Weijie J. Su

Abstract: Large language models (LLMs) have been widely employed across various application domains, yet their black-box nature poses significant challenges to understanding how these models process input data internally to make predictions. In this paper, we introduce a precise and quantitative law that governs the learning of contextualized token embeddings through intermediate layers in pre-trained LLMs… ▽ More Large language models (LLMs) have been widely employed across various application domains, yet their black-box nature poses significant challenges to understanding how these models process input data internally to make predictions. In this paper, we introduce a precise and quantitative law that governs the learning of contextualized token embeddings through intermediate layers in pre-trained LLMs for next-token prediction. Our findings reveal that each layer contributes equally to enhancing prediction accuracy, from the lowest to the highest layer -- a universal phenomenon observed across a diverse array of open-source LLMs, built on architectures such as Transformer, RWKV, and Mamba. We demonstrate that this law offers new perspectives and insights to inform and guide practices in LLM development and applications, including model scaling, pre-training tasks, and information flow. Overall, our law enables more fine-grained approaches to the design, training, and interpretation of LLMs through scrutinizing their internal data processing mechanisms. △ Less

Submitted 23 August, 2024; originally announced August 2024.

UV Completion of Neutral Triple Gauge Couplings

Authors: John Ellis, Hong-Jian He, Rui-Qing Xiao, Shi-Ping Zeng, Jiaming Zheng

Abstract: Neutral triple gauge couplings (nTGCs) are manifestation of new physics beyond the Standard Model (SM), as they are absent in the SM and are first generated by dimension-8 operators in the SM Effective Field Theory (SMEFT). We study the UV completion of nTGCs in a renormalizable model with vector-like heavy fermions. We compute the one-loop heavy fermion contributions to nTGC vertices by matching… ▽ More Neutral triple gauge couplings (nTGCs) are manifestation of new physics beyond the Standard Model (SM), as they are absent in the SM and are first generated by dimension-8 operators in the SM Effective Field Theory (SMEFT). We study the UV completion of nTGCs in a renormalizable model with vector-like heavy fermions. We compute the one-loop heavy fermion contributions to nTGC vertices by matching them to dimension-8 operators in the low energy limit. Such fermion loops contain either heavy fermions only or mixture of heavy fermions with light SM fermions. We find that their contributions can induce dimension-8 nTGC effective operators containing two SM Higgs-doublet fields, which are formulated with a complete set of 7 dimension-8 operators generating off-shell CP-even nTGCs. We present the results in terms of SMEFT coefficients and in terms of nTGC vertices (form factors) with two on-shell gauge bosons. In the heavy-light mixing case there appear terms that cannot be accommodated by conventional parametrizations of form factors due to extra logarithmic corrections. We further discuss the implications for probing such UV dynamics via nTGCs at the high energy colliders. △ Less

Submitted 22 August, 2024; originally announced August 2024.

Report number: KCL-PH-TH/2024-45, CERN-TH-2024-143

Adaptive Spiking Neural Networks with Hybrid Coding

Authors: Huaxu He

Abstract: The Spiking Neural Network (SNN), due to its unique spiking-driven nature, is a more energy-efficient and effective neural network compared to Artificial Neural Networks (ANNs). The encoding method directly influences the overall performance of the network, and currently, direct encoding is primarily used for directly trained SNNs. When working with static image datasets, direct encoding inputs th… ▽ More The Spiking Neural Network (SNN), due to its unique spiking-driven nature, is a more energy-efficient and effective neural network compared to Artificial Neural Networks (ANNs). The encoding method directly influences the overall performance of the network, and currently, direct encoding is primarily used for directly trained SNNs. When working with static image datasets, direct encoding inputs the same feature map at every time step, failing to fully exploit the spatiotemporal properties of SNNs. While temporal encoding converts input data into spike trains with spatiotemporal characteristics, traditional SNNs utilize the same neurons when processing input data across different time steps, limiting their ability to integrate and utilize spatiotemporal information effectively.To address this, this paper employs temporal encoding and proposes the Adaptive Spiking Neural Network (ASNN), enhancing the utilization of temporal encoding in conventional SNNs. Additionally, temporal encoding is less frequently used because short time steps can lead to significant loss of input data information, often necessitating a higher number of time steps in practical applications. However, training large SNNs with long time steps is challenging due to hardware constraints. To overcome this, this paper introduces a hybrid encoding approach that not only reduces the required time steps for training but also continues to improve the overall network performance.Notably, significant improvements in classification performance are observed on both Spikformer and Spiking ResNet architectures.our code is available at https://github.com/hhx0320/ASNN △ Less

Submitted 22 August, 2024; originally announced August 2024.

Search for neutrino signals correlated with LHAASO diffuse Galactic emission

Authors: Wenlian Li, Tian-Qi Huang, Donglian Xu, Huihai He

Abstract: The diffuse Galactic $γ$-ray emission originates from the interactions between cosmic rays and the interstellar medium or radiation fields within our Galaxy, where the production of neutrinos is also anticipated. Recently, the Large High Altitude Air Shower Observatory (LHAASO) reported measurements of diffuse $γ$-rays from the Galactic plane with energies ranging from sub-TeV to 1 PeV. Using publ… ▽ More The diffuse Galactic $γ$-ray emission originates from the interactions between cosmic rays and the interstellar medium or radiation fields within our Galaxy, where the production of neutrinos is also anticipated. Recently, the Large High Altitude Air Shower Observatory (LHAASO) reported measurements of diffuse $γ$-rays from the Galactic plane with energies ranging from sub-TeV to 1 PeV. Using publicly available 7 years of IceCube track data with the full detector, we conduct a template search using the $γ$-ray flux map observed by LHAASO-KM2A as the neutrino emission template and perform a scan search of the Galactic plane. In the template search, a mild excess of neutrinos is observed in the Galactic plane with a pretrial significance of $1.9σ$. The measured muon neutrino intensity at 25 TeV is $4.73^{+2.53}_{-2.51}\times10^{-14}\,{\rm TeV^{-1}\,cm^{-2}\,s^{-1}\,sr^{-1}}$, consistent with the expected neutrino flux assuming that all the diffuse Galactic $γ$-rays originate from hadronic interactions. In the Galactic plane scan search, the most significant location is found at $l=63.57^{\circ}$ and $b=0.93^{\circ}$ with a pretrial (posttrial) significance of $4.6σ$ ($1.8σ$). △ Less

Submitted 22 August, 2024; originally announced August 2024.

GRANDlib: A simulation pipeline for the Giant Radio Array for Neutrino Detection (GRAND)

Authors: GRAND Collaboration, Rafael Alves Batista, Aurélien Benoit-Lévy, Teresa Bister, Martina Bohacova, Mauricio Bustamante, Washington Carvalho, Yiren Chen, LingMei Cheng, Simon Chiche, Jean-Marc Colley, Pablo Correa, Nicoleta Cucu Laurenciu, Zigao Dai, Rogerio M. de Almeida, Beatriz de Errico, Sijbrand de Jong, João R. T. de Mello Neto, Krijn D. de Vries, Valentin Decoene, Peter B. Denton, Bohao Duan, Kaikai Duan, Ralph Engel, William Erba , et al. (90 additional authors not shown)

Abstract: The operation of upcoming ultra-high-energy cosmic-ray, gamma-ray, and neutrino radio-detection experiments, like the Giant Radio Array for Neutrino Detection (GRAND), poses significant computational challenges involving the production of numerous simulations of particle showers and their detection, and a high data throughput. GRANDlib is an open-source software tool designed to meet these challen… ▽ More The operation of upcoming ultra-high-energy cosmic-ray, gamma-ray, and neutrino radio-detection experiments, like the Giant Radio Array for Neutrino Detection (GRAND), poses significant computational challenges involving the production of numerous simulations of particle showers and their detection, and a high data throughput. GRANDlib is an open-source software tool designed to meet these challenges. Its primary goal is to perform end-to-end simulations of the detector operation, from the interaction of ultra-high-energy particles, through -- by interfacing with external air-shower simulations -- the ensuing particle shower development and its radio emission, to its detection by antenna arrays and its processing by data-acquisition systems. Additionally, GRANDlib manages the visualization, storage, and retrieval of experimental and simulated data. We present an overview of GRANDlib to serve as the basis of future GRAND analyses. △ Less

Submitted 20 August, 2024; originally announced August 2024.

Comments: 11 pages, 9 figures, plus appendices

Exploring New Physics with PandaX-4T Low Energy Electronic Recoil Data

Authors: PandaX Collaboration, Xinning Zeng, Zihao Bo, Wei Chen, Xun Chen, Yunhua Chen, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Zhixing Gao, Lisheng Geng, Karl Giboni, Xunan Guo, Xuyuan Guo, Zichao Guo, Chencheng Han, Ke HanChangda He, Jinrong He, Di Huang, Houqi Huang, Junting Huang, Ruquan Hou, Yu Hou, Xiangdong Ji , et al. (76 additional authors not shown)

Abstract: New particles beyond the Standard Model of particle physics, such as axions, can be effectively searched through their interactions with electrons. We use the large liquid xenon detector PandaX-4T to search for novel electronic recoil signals induced by solar axions, neutrinos with anomalous magnetic moment, axion-like particles, dark photons, and light fermionic dark matter. A detailed background… ▽ More New particles beyond the Standard Model of particle physics, such as axions, can be effectively searched through their interactions with electrons. We use the large liquid xenon detector PandaX-4T to search for novel electronic recoil signals induced by solar axions, neutrinos with anomalous magnetic moment, axion-like particles, dark photons, and light fermionic dark matter. A detailed background model is established with the latest datasets with 1.54 $\rm tonne \cdot year$ exposure. No significant excess above the background has been observed, and we have obtained competitive constraints for axion couplings, neutrino magnetic moment, and fermionic dark matter interactions. △ Less

Submitted 14 August, 2024; originally announced August 2024.

Open-Source Software Architecture for Multi-Robot Wire Arc Additive Manufacturing (WAAM)

Authors: Honglu He, Chen-lung Lu, Jinhan Ren, Joni Dhar, Glenn Saunders, John Wason, Johnson Samuel, Agung Julius, John T. Wen

Abstract: Wire Arc Additive Manufacturing (WAAM) is a metal 3D printing technology that deposits molten metal wire on a substrate to form desired geometries. Articulated robot arms are commonly used in WAAM to produce complex geometric shapes. However, they mostly rely on proprietary robot and weld control software that limits process tuning and customization, incorporation of third-party sensors, implement… ▽ More Wire Arc Additive Manufacturing (WAAM) is a metal 3D printing technology that deposits molten metal wire on a substrate to form desired geometries. Articulated robot arms are commonly used in WAAM to produce complex geometric shapes. However, they mostly rely on proprietary robot and weld control software that limits process tuning and customization, incorporation of third-party sensors, implementation on robots and weld controllers from multiple vendors, and customizable user programming. This paper presents a general open-source software architecture for WAAM that addresses these limitations. The foundation of this architecture is Robot Raconteur, an open-source control and communication framework that serves as the middleware for integrating robots and sensors from different vendors. Based on this architecture, we developed an end-to-end robotic WAAM implementation that takes a CAD file to a printed WAAM part and evaluates the accuracy of the result. The major components in the architecture include part slicing, robot motion planning, part metrology, in-process sensing, and process tuning. The current implementation is based on Motoman robots and Fronius weld controller, but the approach is applicable to other industrial robots and weld controllers. The capability of the WAAM tested is demonstrated through the printing of parts of various geometries and acquisition of in-process sensor data for motion adjustment. △ Less

Submitted 7 August, 2024; originally announced August 2024.

A First Look at License Compliance Capability of LLMs in Code Generation

Authors: Weiwei Xu, Kai Gao, Hao He, Minghui Zhou

Abstract: Recent advances in Large Language Models (LLMs) have revolutionized code generation, leading to widespread adoption of AI coding tools by developers. However, LLMs can generate license-protected code without providing the necessary license information, leading to potential intellectual property violations during software production. This paper addresses the critical, yet underexplored, issue of li… ▽ More Recent advances in Large Language Models (LLMs) have revolutionized code generation, leading to widespread adoption of AI coding tools by developers. However, LLMs can generate license-protected code without providing the necessary license information, leading to potential intellectual property violations during software production. This paper addresses the critical, yet underexplored, issue of license compliance in LLM-generated code by establishing a benchmark to evaluate the ability of LLMs to provide accurate license information for their generated code. To establish this benchmark, we conduct an empirical study to identify a reasonable standard for "striking similarity" that excludes the possibility of independent creation, indicating a copy relationship between the LLM output and certain open-source code. Based on this standard, we propose an evaluation benchmark LiCoEval, to evaluate the license compliance capabilities of LLMs. Using LiCoEval, we evaluate 14 popular LLMs, finding that even top-performing LLMs produce a non-negligible proportion (0.88% to 2.01%) of code strikingly similar to existing open-source implementations. Notably, most LLMs fail to provide accurate license information, particularly for code under copyleft licenses. These findings underscore the urgent need to enhance LLM compliance capabilities in code generation tasks. Our study provides a foundation for future research and development to improve license compliance in AI-assisted software development, contributing to both the protection of open-source software copyrights and the mitigation of legal risks for LLM users. △ Less

Submitted 5 August, 2024; originally announced August 2024.

Intensities of the hydrogen Balmer lines of solar-like stars revealed by the LAMOST spectroscopic surveys

Authors: Han He

Abstract: The intensities of the hydrogen Balmer lines of solar-like stars are investigated for stellar chromospheric activity by using the co-source spectral data of the LAMOST Low-Resolution Spectroscopic Survey (LRS) and Medium-Resolution Spectroscopic Survey (MRS). The Balmer H$α$, H$β$, H$γ$, and H$δ$ lines in the LRS data and the H$α$ line in the MRS data are analyzed. The absolute flux indexes, defin… ▽ More The intensities of the hydrogen Balmer lines of solar-like stars are investigated for stellar chromospheric activity by using the co-source spectral data of the LAMOST Low-Resolution Spectroscopic Survey (LRS) and Medium-Resolution Spectroscopic Survey (MRS). The Balmer H$α$, H$β$, H$γ$, and H$δ$ lines in the LRS data and the H$α$ line in the MRS data are analyzed. The absolute flux indexes, defined as the ratios of the absolute fluxes at the centers of the Balmer lines to the stellar bolometric flux, are employed to indicate the intensity magnitudes of the Balmer lines in response to stellar activity. The H$α$ indexes derived from the LRS data and the MRS data, respectively, are calibrated to be quantitatively consistent with each other. It is found that, as the H$α$ index increases, the H$β$, H$γ$, and H$δ$ indexes first present trend of increasing and then decreasing, and finally increase synchronously with the H$α$ index. The distributions of the Balmer line indexes also reveal the three distinct stages of stellar activity (normal stage, intense stage, and extremely intense stage), in which the extremely intense stage is characterized by the synchronous growth of the indexes of the four Balmer lines. The different behaviors of the H$β$, H$γ$, and H$δ$ lines from that of the H$α$ line can be interpreted by the different mechanisms by which the line-core intensities are formed, and the three distinct activity stages imply the very different magnetic field environments and physical conditions of solar-like stars. △ Less

Submitted 4 August, 2024; originally announced August 2024.

Comments: 29 pages, 13 figures, 2 tables, with supplementary_information; published in SciRep_2024_14_17962

Journal ref: SciRep (2024) 14 17962

POA: Pre-training Once for Models of All Sizes

Authors: Yingying Zhang, Xin Guo, Jiangwei Lao, Lei Yu, Lixiang Ru, Jian Wang, Guo Ye, Huimei He, Jingdong Chen, Ming Yang

Abstract: Large-scale self-supervised pre-training has paved the way for one foundation model to handle many different vision tasks. Most pre-training methodologies train a single model of a certain size at one time. Nevertheless, various computation or storage constraints in real-world scenarios require substantial efforts to develop a series of models with different sizes to deploy. Thus, in this study, w… ▽ More Large-scale self-supervised pre-training has paved the way for one foundation model to handle many different vision tasks. Most pre-training methodologies train a single model of a certain size at one time. Nevertheless, various computation or storage constraints in real-world scenarios require substantial efforts to develop a series of models with different sizes to deploy. Thus, in this study, we propose a novel tri-branch self-supervised training framework, termed as POA (Pre-training Once for All), to tackle this aforementioned issue. Our approach introduces an innovative elastic student branch into a modern self-distillation paradigm. At each pre-training step, we randomly sample a sub-network from the original student to form the elastic student and train all branches in a self-distilling fashion. Once pre-trained, POA allows the extraction of pre-trained models of diverse sizes for downstream tasks. Remarkably, the elastic student facilitates the simultaneous pre-training of multiple models with different sizes, which also acts as an additional ensemble of models of various sizes to enhance representation learning. Extensive experiments, including k-nearest neighbors, linear probing evaluation and assessments on multiple downstream tasks demonstrate the effectiveness and advantages of our POA. It achieves state-of-the-art performance using ViT, Swin Transformer and ResNet backbones, producing around a hundred models with different sizes through a single pre-training session. The code is available at: https://github.com/Qichuzyy/POA. △ Less

Submitted 2 August, 2024; originally announced August 2024.

Comments: Accepted by ECCV2024

MSC Class: 68T07

Large deviations for the maximum of a reducible two-type branching Brownian motion

Authors: Hui He

Abstract: We consider a two-type reducible branching Brownian motion, defined as a particle system on the real line in which particles of two types move according to independent Brownian motions and create offspring at a constant rate. Particles of type $1$ can give birth to particles of types $1$ and $2$, but particles of type $2$ only give birth to descendants of type $2$. Under some specific conditions,… ▽ More We consider a two-type reducible branching Brownian motion, defined as a particle system on the real line in which particles of two types move according to independent Brownian motions and create offspring at a constant rate. Particles of type $1$ can give birth to particles of types $1$ and $2$, but particles of type $2$ only give birth to descendants of type $2$. Under some specific conditions, Belloum and Mallein in \cite{BeMa21} showed that the maximum position $M_t$ of all particles alive at time $t$, suitably centered by a deterministic function $m_t$, converge weakly. In this work, we are interested in the decay rate of the following upper large deviation probability, as $t\rightarrow\infty$, \[ {\mathbb P}(M_t\geq θm_t),\quad θ>1. \] We shall show that the decay rate function exhibits phase transitions depending on certain relations between $θ$, the variance of the underling Brownian motion and the branching rate. △ Less

Submitted 5 August, 2024; v1 submitted 1 August, 2024; originally announced August 2024.

Comments: Draft version

Identifying the Hierarchical Emotional Areas in the Human Brain Through Information Fusion

Authors: Zhongyu Huang, Changde Du, Chaozhuo Li, Kaicheng Fu, Huiguang He

Abstract: The brain basis of emotion has consistently received widespread attention, attracting a large number of studies to explore this cutting-edge topic. However, the methods employed in these studies typically only model the pairwise relationship between two brain regions, while neglecting the interactions and information fusion among multiple brain regions$\unicode{x2014}$one of the key ideas of the p… ▽ More The brain basis of emotion has consistently received widespread attention, attracting a large number of studies to explore this cutting-edge topic. However, the methods employed in these studies typically only model the pairwise relationship between two brain regions, while neglecting the interactions and information fusion among multiple brain regions$\unicode{x2014}$one of the key ideas of the psychological constructionist hypothesis. To overcome the limitations of traditional methods, this study provides an in-depth theoretical analysis of how to maximize interactions and information fusion among brain regions. Building on the results of this analysis, we propose to identify the hierarchical emotional areas in the human brain through multi-source information fusion and graph machine learning methods. Comprehensive experiments reveal that the identified hierarchical emotional areas, from lower to higher levels, primarily facilitate the fundamental process of emotion perception, the construction of basic psychological operations, and the coordination and integration of these operations. Overall, our findings provide unique insights into the brain mechanisms underlying specific emotions based on the psychological constructionist hypothesis. △ Less

Submitted 1 August, 2024; originally announced August 2024.

TVDiag: A Task-oriented and View-invariant Failure Diagnosis Framework with Multimodal Data

Authors: Shuaiyu Xie, Jian Wang, Hanbin He, Zhihao Wang, Yuqi Zhao, Neng Zhang, Bing Li

Abstract: Microservice-based systems often suffer from reliability issues due to their intricate interactions and expanding scale. With the rapid growth of observability techniques, various methods have been proposed to achieve failure diagnosis, including root cause localization and failure type identification, by leveraging diverse monitoring data such as logs, metrics, or traces. However, traditional fai… ▽ More Microservice-based systems often suffer from reliability issues due to their intricate interactions and expanding scale. With the rapid growth of observability techniques, various methods have been proposed to achieve failure diagnosis, including root cause localization and failure type identification, by leveraging diverse monitoring data such as logs, metrics, or traces. However, traditional failure diagnosis methods that use single-modal data can hardly cover all failure scenarios due to the restricted information. Several failure diagnosis methods have been recently proposed to integrate multimodal data based on deep learning. These methods, however, tend to combine modalities indiscriminately and treat them equally in failure diagnosis, ignoring the relationship between specific modalities and different diagnostic tasks. This oversight hinders the effective utilization of the unique advantages offered by each modality. To address the limitation, we propose \textit{TVDiag}, a multimodal failure diagnosis framework for locating culprit microservice instances and identifying their failure types (e.g., Net-packets Corruption) in microservice-based systems. \textit{TVDiag} employs task-oriented learning to enhance the potential advantages of each modality and establishes cross-modal associations based on contrastive learning to extract view-invariant failure information. Furthermore, we develop a graph-level data augmentation strategy that randomly inactivates the observability of some normal microservice instances during training to mitigate the shortage of training data. Experimental results show that \textit{TVDiag} outperforms state-of-the-art methods in multimodal failure diagnosis, achieving at least a 55.94\% higher $HR@1$ accuracy and over a 4.08\% increase in F1-score across two datasets. △ Less

Submitted 23 August, 2024; v1 submitted 29 July, 2024; originally announced July 2024.

Comments: 32 pages

Obstacle-Aware Length-Matching Routing for Any-Direction Traces in Printed Circuit Board

Authors: Weijie Fang, Longkun Guo, Jiawei Lin, Silu Xiong, Huan He, Jiacen Xu, Jianli Chen

Abstract: Emerging applications in Printed Circuit Board (PCB) routing impose new challenges on automatic length matching, including adaptability for any-direction traces with their original routing preserved for interactiveness. The challenges can be addressed through two orthogonal stages: assign non-overlapping routing regions to each trace and meander the traces within their regions to reach the target… ▽ More Emerging applications in Printed Circuit Board (PCB) routing impose new challenges on automatic length matching, including adaptability for any-direction traces with their original routing preserved for interactiveness. The challenges can be addressed through two orthogonal stages: assign non-overlapping routing regions to each trace and meander the traces within their regions to reach the target length. In this paper, mainly focusing on the meandering stage, we propose an obstacle-aware detailed routing approach to optimize the utilization of available space and achieve length matching while maintaining the original routing of traces. Furthermore, our approach incorporating the proposed Multi-Scale Dynamic Time Warping (MSDTW) method can also handle differential pairs against common decoupled problems. Experimental results demonstrate that our approach has effective length-matching routing ability and compares favorably to previous approaches under more complicated constraints. △ Less

Submitted 27 July, 2024; originally announced July 2024.

Spatial analysis of tails of air pollution PDFs in Europe

Authors: Hankun He, Benjamin Schäfer, Christian Beck

Abstract: Outdoor air pollution is estimated to cause a huge number of premature deaths worldwide, it catalyses many diseases on a variety of time scales, and it has a detrimental effect on the environment. In light of these impacts it is necessary to obtain a better understanding of the dynamics and statistics of measured air pollution concentrations, including temporal fluctuations of observed concentrati… ▽ More Outdoor air pollution is estimated to cause a huge number of premature deaths worldwide, it catalyses many diseases on a variety of time scales, and it has a detrimental effect on the environment. In light of these impacts it is necessary to obtain a better understanding of the dynamics and statistics of measured air pollution concentrations, including temporal fluctuations of observed concentrations and spatial heterogeneities. Here we present an extensive analysis for measured data from Europe. The observed probability density functions (PDFs) of air pollution concentrations depend very much on the spatial location and on the pollutant substance. We analyse a large number of time series data from 3544 different European monitoring sites and show that the PDFs of nitric oxide ($NO$), nitrogen dioxide ($NO_{2}$) and particulate matter ($PM_{10}$ and $PM_{2.5}$) concentrations generically exhibit heavy tails. These are asymptotically well approximated by $q$-exponential distributions with a given entropic index $q$ and width parameter $λ$. We observe that the power-law parameter $q$ and the width parameter $λ$ vary widely for the different spatial locations. We present the results of our data analysis in the form of a map that shows which parameters $q$ and $λ$ are most relevant in a given region. A variety of interesting spatial patterns is observed that correlate to properties of the geographical region. We also present results on typical time scales associated with the dynamical behaviour. △ Less

Submitted 17 July, 2024; originally announced July 2024.

Comments: arXiv admin note: substantial text overlap with arXiv:2203.04296

Balanced Multi-Relational Graph Clustering

Authors: Zhixiang Shen, Haolan He, Zhao Kang

Abstract: Multi-relational graph clustering has demonstrated remarkable success in uncovering underlying patterns in complex networks. Representative methods manage to align different views motivated by advances in contrastive learning. Our empirical study finds the pervasive presence of imbalance in real-world graphs, which is in principle contradictory to the motivation of alignment. In this paper, we fir… ▽ More Multi-relational graph clustering has demonstrated remarkable success in uncovering underlying patterns in complex networks. Representative methods manage to align different views motivated by advances in contrastive learning. Our empirical study finds the pervasive presence of imbalance in real-world graphs, which is in principle contradictory to the motivation of alignment. In this paper, we first propose a novel metric, the Aggregation Class Distance, to empirically quantify structural disparities among different graphs. To address the challenge of view imbalance, we propose Balanced Multi-Relational Graph Clustering (BMGC), comprising unsupervised dominant view mining and dual signals guided representation learning. It dynamically mines the dominant view throughout the training process, synergistically improving clustering performance with representation learning. Theoretical analysis ensures the effectiveness of dominant view mining. Extensive experiments and in-depth analysis on real-world and synthetic datasets showcase that BMGC achieves state-of-the-art performance, underscoring its superiority in addressing the view imbalance inherent in multi-relational graphs. The source code and datasets are available at https://github.com/zxlearningdeep/BMGC. △ Less

Submitted 23 July, 2024; originally announced July 2024.

Comments: Accepted by ACM Multimedia 2024

Ground states of a coupled pseudo-relativistic Hartree system: existence and concentration behavior

Authors: Huiting He, Chungen Liu, Jiabin Zuo

Abstract: This paper is concerned with the ground states of a coupled pseudo-relativistic Hartree system in $\mathbb{R} ^{3} $ with trapping potentials, where the intraspecies and the interspecies interaction are both attractive. By investigating an associated constraint minimization problem, the existence and non-existence of ground states are classified completely. Under certain conditions on the trapping… ▽ More This paper is concerned with the ground states of a coupled pseudo-relativistic Hartree system in $\mathbb{R} ^{3} $ with trapping potentials, where the intraspecies and the interspecies interaction are both attractive. By investigating an associated constraint minimization problem, the existence and non-existence of ground states are classified completely. Under certain conditions on the trapping potentials, we present a precise analysis on the concentration behavior of the minimizers as the coupling coefficient goes to a critical value, where the minimizers blow up and the maximum point sequence concentrates at a global minima of the associated trapping potentials. We also identify an optimal blowing up rate under polynomial potentials by establishing some delicate estimates of energy functionals. △ Less

Submitted 22 July, 2024; originally announced July 2024.

Comments: 34 pages

MSC Class: 35Q40; 35B40; 35R11

Approach for composition measurement of cosmic rays using the muon-to-electron ratio observed by LHAASO-KM2A

Authors: Xishui Tian, Zhuo Li, Quanbu Gou, Hengying Zhang, Huihai He, Cunfeng Feng, Giuseppe Di Sciascio

Abstract: Composition measurement of cosmic rays (CRs) around the knee of the CR energy spectrum is crucial for studying the processes of particle acceleration and propagation of Galactic CRs. The Square Kilometer Array (KM2A) of Large High Altitude Air Shower Observatory (LHAASO) can provide precise measurement of the muonic and electromagnetic (em.) components in CR-induced extensive air showers, and henc… ▽ More Composition measurement of cosmic rays (CRs) around the knee of the CR energy spectrum is crucial for studying the processes of particle acceleration and propagation of Galactic CRs. The Square Kilometer Array (KM2A) of Large High Altitude Air Shower Observatory (LHAASO) can provide precise measurement of the muonic and electromagnetic (em.) components in CR-induced extensive air showers, and hence a good chance to disentangle the CR composition. Here we propose an approach of decomposing CR compositions with the number ratio between muons and em. particles ($N_μ$/$N_{\rm e}$) observed by LHAASO-KM2A: we reconstruct the energy spectra of individual CR compositions by fitting $N_μ$/$N_{\rm e}$ distributions in each reconstructed energy bin using the template shapes of $N_μ$/$N_{\rm e}$ distributions of individual CR compositions based on Monte Carlo (MC) simulation. We evaluate the performance of this approach with MC tests where mock data of LHAASO-KM2A are generated by MC simulation. We show that the input composition model can be well recovered in this approach, independent of the CR composition model adopted in the MC simulation for the template distributions. The uncertainties of the reconstructed spectra at < 20 PeV, mainly limited by simulation statistics, are $\le$ 7% for proton, He, and Fe groups, and $\le$ 8% and $\le$ 16% for CNO and MgAlSi groups, respectively. △ Less

Submitted 18 July, 2024; originally announced July 2024.

Comments: 12 pages, 9 figures, 1 table. Accepted by the PRD

Robust Multivariate Time Series Forecasting against Intra- and Inter-Series Transitional Shift

Authors: Hui He, Qi Zhang, Kun Yi, Xiaojun Xue, Shoujin Wang, Liang Hu, Longbing Cao

Abstract: The non-stationary nature of real-world Multivariate Time Series (MTS) data presents forecasting models with a formidable challenge of the time-variant distribution of time series, referred to as distribution shift. Existing studies on the distribution shift mostly adhere to adaptive normalization techniques for alleviating temporal mean and covariance shifts or time-variant modeling for capturing… ▽ More The non-stationary nature of real-world Multivariate Time Series (MTS) data presents forecasting models with a formidable challenge of the time-variant distribution of time series, referred to as distribution shift. Existing studies on the distribution shift mostly adhere to adaptive normalization techniques for alleviating temporal mean and covariance shifts or time-variant modeling for capturing temporal shifts. Despite improving model generalization, these normalization-based methods often assume a time-invariant transition between outputs and inputs but disregard specific intra-/inter-series correlations, while time-variant models overlook the intrinsic causes of the distribution shift. This limits model expressiveness and interpretability of tackling the distribution shift for MTS forecasting. To mitigate such a dilemma, we present a unified Probabilistic Graphical Model to Jointly capturing intra-/inter-series correlations and modeling the time-variant transitional distribution, and instantiate a neural framework called JointPGM for non-stationary MTS forecasting. Specifically, JointPGM first employs multiple Fourier basis functions to learn dynamic time factors and designs two distinct learners: intra-series and inter-series learners. The intra-series learner effectively captures temporal dynamics by utilizing temporal gates, while the inter-series learner explicitly models spatial dynamics through multi-hop propagation, incorporating Gumbel-softmax sampling. These two types of series dynamics are subsequently fused into a latent variable, which is inversely employed to infer time factors, generate final prediction, and perform reconstruction. We validate the effectiveness and efficiency of JointPGM through extensive experiments on six highly non-stationary MTS datasets, achieving state-of-the-art forecasting performance of MTS forecasting. △ Less

Submitted 18 July, 2024; originally announced July 2024.

Comments: 19 pages, 11 figures

MSC Class: 68Txx ACM Class: I.2.6

DreamStory: Open-Domain Story Visualization by LLM-Guided Multi-Subject Consistent Diffusion

Authors: Huiguo He, Huan Yang, Zixi Tuo, Yuan Zhou, Qiuyue Wang, Yuhang Zhang, Zeyu Liu, Wenhao Huang, Hongyang Chao, Jian Yin

Abstract: Story visualization aims to create visually compelling images or videos corresponding to textual narratives. Despite recent advances in diffusion models yielding promising results, existing methods still struggle to create a coherent sequence of subject-consistent frames based solely on a story. To this end, we propose DreamStory, an automatic open-domain story visualization framework by leveragin… ▽ More Story visualization aims to create visually compelling images or videos corresponding to textual narratives. Despite recent advances in diffusion models yielding promising results, existing methods still struggle to create a coherent sequence of subject-consistent frames based solely on a story. To this end, we propose DreamStory, an automatic open-domain story visualization framework by leveraging the LLMs and a novel multi-subject consistent diffusion model. DreamStory consists of (1) an LLM acting as a story director and (2) an innovative Multi-Subject consistent Diffusion model (MSD) for generating consistent multi-subject across the images. First, DreamStory employs the LLM to generate descriptive prompts for subjects and scenes aligned with the story, annotating each scene's subjects for subsequent subject-consistent generation. Second, DreamStory utilizes these detailed subject descriptions to create portraits of the subjects, with these portraits and their corresponding textual information serving as multimodal anchors (guidance). Finally, the MSD uses these multimodal anchors to generate story scenes with consistent multi-subject. Specifically, the MSD includes Masked Mutual Self-Attention (MMSA) and Masked Mutual Cross-Attention (MMCA) modules. MMSA and MMCA modules ensure appearance and semantic consistency with reference images and text, respectively. Both modules employ masking mechanisms to prevent subject blending. To validate our approach and promote progress in story visualization, we established a benchmark, DS-500, which can assess the overall performance of the story visualization framework, subject-identification accuracy, and the consistency of the generation model. Extensive experiments validate the effectiveness of DreamStory in both subjective and objective evaluations. Please visit our project homepage at https://dream-xyz.github.io/dreamstory. △ Less

Submitted 17 July, 2024; originally announced July 2024.

Voltage-Controlled Magnetoelectric Devices for Neuromorphic Diffusion Process

Authors: Yang Cheng, Qingyuan Shu, Albert Lee, Haoran He, Ivy Zhu, Haris Suhail, Minzhang Chen, Renhe Chen, Zirui Wang, Hantao Zhang, Chih-Yao Wang, Shan-Yi Yang, Yu-Chen Hsin, Cheng-Yi Shih, Hsin-Han Lee, Ran Cheng, Sudhakar Pamarti, Xufeng Kou, Kang L. Wang

Abstract: Stochastic diffusion processes are pervasive in nature, from the seemingly erratic Brownian motion to the complex interactions of synaptically-coupled spiking neurons. Recently, drawing inspiration from Langevin dynamics, neuromorphic diffusion models were proposed and have become one of the major breakthroughs in the field of generative artificial intelligence. Unlike discriminative models that h… ▽ More Stochastic diffusion processes are pervasive in nature, from the seemingly erratic Brownian motion to the complex interactions of synaptically-coupled spiking neurons. Recently, drawing inspiration from Langevin dynamics, neuromorphic diffusion models were proposed and have become one of the major breakthroughs in the field of generative artificial intelligence. Unlike discriminative models that have been well developed to tackle classification or regression tasks, diffusion models as well as other generative models such as ChatGPT aim at creating content based upon contexts learned. However, the more complex algorithms of these models result in high computational costs using today's technologies, creating a bottleneck in their efficiency, and impeding further development. Here, we develop a spintronic voltage-controlled magnetoelectric memory hardware for the neuromorphic diffusion process. The in-memory computing capability of our spintronic devices goes beyond current Von Neumann architecture, where memory and computing units are separated. Together with the non-volatility of magnetic memory, we can achieve high-speed and low-cost computing, which is desirable for the increasing scale of generative models in the current era. We experimentally demonstrate that the hardware-based true random diffusion process can be implemented for image generation and achieve comparable image quality to software-based training as measured by the Frechet inception distance (FID) score, achieving ~10^3 better energy-per-bit-per-area over traditional hardware. △ Less

Submitted 16 July, 2024; originally announced July 2024.

Learning Multi-view Anomaly Detection

Authors: Haoyang He, Jiangning Zhang, Guanzhong Tian, Chengjie Wang, Lei Xie

Abstract: This study explores the recently proposed challenging multi-view Anomaly Detection (AD) task. Single-view tasks would encounter blind spots from other perspectives, resulting in inaccuracies in sample-level prediction. Therefore, we introduce the \textbf{M}ulti-\textbf{V}iew \textbf{A}nomaly \textbf{D}etection (\textbf{MVAD}) framework, which learns and integrates features from multi-views. Specif… ▽ More This study explores the recently proposed challenging multi-view Anomaly Detection (AD) task. Single-view tasks would encounter blind spots from other perspectives, resulting in inaccuracies in sample-level prediction. Therefore, we introduce the \textbf{M}ulti-\textbf{V}iew \textbf{A}nomaly \textbf{D}etection (\textbf{MVAD}) framework, which learns and integrates features from multi-views. Specifically, we proposed a \textbf{M}ulti-\textbf{V}iew \textbf{A}daptive \textbf{S}election (\textbf{MVAS}) algorithm for feature learning and fusion across multiple views. The feature maps are divided into neighbourhood attention windows to calculate a semantic correlation matrix between single-view windows and all other views, which is a conducted attention mechanism for each single-view window and the top-K most correlated multi-view windows. Adjusting the window sizes and top-K can minimise the computational complexity to linear. Extensive experiments on the Real-IAD dataset for cross-setting (multi/single-class) validate the effectiveness of our approach, achieving state-of-the-art performance among sample \textbf{4.1\%}$\uparrow$/ image \textbf{5.6\%}$\uparrow$/pixel \textbf{6.7\%}$\uparrow$ levels with a total of ten metrics with only \textbf{18M} parameters and fewer GPU memory and training time. △ Less

Submitted 16 July, 2024; originally announced July 2024.

Comments: 10 pages

MetaUrban: A Simulation Platform for Embodied AI in Urban Spaces

Authors: Wayne Wu, Honglin He, Yiran Wang, Chenda Duan, Jack He, Zhizheng Liu, Quanyi Li, Bolei Zhou

Abstract: Public urban spaces like streetscapes and plazas serve residents and accommodate social life in all its vibrant variations. Recent advances in Robotics and Embodied AI make public urban spaces no longer exclusive to humans. Food delivery bots and electric wheelchairs have started sharing sidewalks with pedestrians, while diverse robot dogs and humanoids have recently emerged in the street. Ensurin… ▽ More Public urban spaces like streetscapes and plazas serve residents and accommodate social life in all its vibrant variations. Recent advances in Robotics and Embodied AI make public urban spaces no longer exclusive to humans. Food delivery bots and electric wheelchairs have started sharing sidewalks with pedestrians, while diverse robot dogs and humanoids have recently emerged in the street. Ensuring the generalizability and safety of these forthcoming mobile machines is crucial when navigating through the bustling streets in urban spaces. In this work, we present MetaUrban, a compositional simulation platform for Embodied AI research in urban spaces. MetaUrban can construct an infinite number of interactive urban scenes from compositional elements, covering a vast array of ground plans, object placements, pedestrians, vulnerable road users, and other mobile agents' appearances and dynamics. We design point navigation and social navigation tasks as the pilot study using MetaUrban for embodied AI research and establish various baselines of Reinforcement Learning and Imitation Learning. Experiments demonstrate that the compositional nature of the simulated environments can substantially improve the generalizability and safety of the trained mobile agents. MetaUrban will be made publicly available to provide more research opportunities and foster safe and trustworthy embodied AI in urban spaces. △ Less

Submitted 11 July, 2024; originally announced July 2024.

Comments: Technical report. Project page: https://metadriverse.github.io/metaurban/

Constraining Holographic Dark Energy and Analyzing Cosmological Tensions

Authors: Xin Tang, Yin-Zhe Ma, Wei-Ming Dai, Hong-Jian He

Abstract: We investigate cosmological constraints on the holographic dark energy (HDE) using the state-of-the-art cosmological datasets: Planck CMB angular power spectra and weak lensing power spectra, Atacama Cosmology Telescope (ACT) temperature power spectra, baryon acoustic oscillation (BAO) and redshift-space distortion (RSD) measurements from six-degree-field galaxy survey and Sloan Digital Sky Survey… ▽ More We investigate cosmological constraints on the holographic dark energy (HDE) using the state-of-the-art cosmological datasets: Planck CMB angular power spectra and weak lensing power spectra, Atacama Cosmology Telescope (ACT) temperature power spectra, baryon acoustic oscillation (BAO) and redshift-space distortion (RSD) measurements from six-degree-field galaxy survey and Sloan Digital Sky Survey (DR12 & DR16) and the Cepheids-Supernovae measurement from SH0ES team (R22). We also examine the HDE model and $Λ$CDM with and without $N_{\rm eff}$ (effective number of relativistic species) being treated as a free parameter. We find that the HDE model can relieve the tensions of $H_0$ and $S_8$ to certain degrees. With ``Planck+ACT+BAO+RSD'' datasets, the constraints are $H_0 = 69.70 \pm 1.39\ \mathrm{km\ s^{-1} Mpc^{-1}}$ and $S_8 = 0.823 \pm 0.011$ in HDE model, which brings down the Hubble tension down to $1.92σ$ confidence level (C.L.) and the $S_8$ tension to $1$-$2σ$ C.L. By adding the R22 data, their values are improved as $H_0 = 71.86 \pm 0.93 \,\mathrm{km\ s^{-1} Mpc^{-1}}$ and $S_8 = 0.813 \pm 0.010$, which further brings the Hubble tension down to $0.85σ$ C.L. and relieves the $S_{8}$ tension. We also quantify the goodness-of-fit of different models with Akaike information criterion (AIC) and Bayesian information criterion (BIC), and find that the HDE agrees with the observational data better than the $Λ$CDM and other extended models (treating $N_{\rm eff}$ as free for fitting). △ Less

Submitted 11 July, 2024; originally announced July 2024.

Comments: 11 pages, 7 figures, 5 tables

Journal ref: Physics of the Dark Universe 46 (2024) 101568

HDT: Hierarchical Document Transformer

Authors: Haoyu He, Markus Flicke, Jan Buchmann, Iryna Gurevych, Andreas Geiger

Abstract: In this paper, we propose the Hierarchical Document Transformer (HDT), a novel sparse Transformer architecture tailored for structured hierarchical documents. Such documents are extremely important in numerous domains, including science, law or medicine. However, most existing solutions are inefficient and fail to make use of the structure inherent to documents. HDT exploits document structure by… ▽ More In this paper, we propose the Hierarchical Document Transformer (HDT), a novel sparse Transformer architecture tailored for structured hierarchical documents. Such documents are extremely important in numerous domains, including science, law or medicine. However, most existing solutions are inefficient and fail to make use of the structure inherent to documents. HDT exploits document structure by introducing auxiliary anchor tokens and redesigning the attention mechanism into a sparse multi-level hierarchy. This approach facilitates information exchange between tokens at different levels while maintaining sparsity, thereby enhancing computational and memory efficiency while exploiting the document structure as an inductive bias. We address the technical challenge of implementing HDT's sample-dependent hierarchical attention pattern by developing a novel sparse attention kernel that considers the hierarchical structure of documents. As demonstrated by our experiments, utilizing structural information present in documents leads to faster convergence, higher sample efficiency and better performance on downstream tasks. △ Less

Submitted 11 July, 2024; originally announced July 2024.

Towards Bridging the Cross-modal Semantic Gap for Multi-modal Recommendation

Authors: Xinglong Wu, Anfeng Huang, Hongwei Yang, Hui He, Yu Tai, Weizhe Zhang

Abstract: Multi-modal recommendation greatly enhances the performance of recommender systems by modeling the auxiliary information from multi-modality contents. Most existing multi-modal recommendation models primarily exploit multimedia information propagation processes to enrich item representations and directly utilize modal-specific embedding vectors independently obtained from upstream pre-trained mode… ▽ More Multi-modal recommendation greatly enhances the performance of recommender systems by modeling the auxiliary information from multi-modality contents. Most existing multi-modal recommendation models primarily exploit multimedia information propagation processes to enrich item representations and directly utilize modal-specific embedding vectors independently obtained from upstream pre-trained models. However, this might be inappropriate since the abundant task-specific semantics remain unexplored, and the cross-modality semantic gap hinders the recommendation performance. Inspired by the recent progress of the cross-modal alignment model CLIP, in this paper, we propose a novel \textbf{CLIP} \textbf{E}nhanced \textbf{R}ecommender (\textbf{CLIPER}) framework to bridge the semantic gap between modalities and extract fine-grained multi-view semantic information. Specifically, we introduce a multi-view modality-alignment approach for representation extraction and measure the semantic similarity between modalities. Furthermore, we integrate the multi-view multimedia representations into downstream recommendation models. Extensive experiments conducted on three public datasets demonstrate the consistent superiority of our model over state-of-the-art multi-modal recommendation models. △ Less

Submitted 7 July, 2024; originally announced July 2024.

Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generation

Authors: Haorui He, Zengqiang Shang, Chaoren Wang, Xuyuan Li, Yicheng Gu, Hua Hua, Liwei Liu, Chen Yang, Jiaqi Li, Peiyang Shi, Yuancheng Wang, Kai Chen, Pengyuan Zhang, Zhizheng Wu

Abstract: Recently, speech generation models have made significant progress by using large-scale training data. However, the research community struggle to produce highly spontaneous and human-like speech due to the lack of large-scale, diverse, and spontaneous speech data. This paper present Emilia, the first multilingual speech generation dataset from in-the-wild speech data, and Emilia-Pipe, the first op… ▽ More Recently, speech generation models have made significant progress by using large-scale training data. However, the research community struggle to produce highly spontaneous and human-like speech due to the lack of large-scale, diverse, and spontaneous speech data. This paper present Emilia, the first multilingual speech generation dataset from in-the-wild speech data, and Emilia-Pipe, the first open-source preprocessing pipeline designed to transform in-the-wild speech data into high-quality training data with annotations for speech generation. Emilia starts with over 101k hours of speech in six languages and features diverse speech with varied speaking styles. To facilitate the scale-up of Emilia, the open-source pipeline Emilia-Pipe can process one hour of raw speech data ready for model training in a few mins, which enables the research community to collaborate on large-scale speech generation research. Experimental results validate the effectiveness of Emilia. Demos are available at: https://emilia-dataset.github.io/Emilia-Demo-Page/. △ Less

Submitted 12 July, 2024; v1 submitted 7 July, 2024; originally announced July 2024.

Comments: Fix typos

Spontaneous Reward Hacking in Iterative Self-Refinement

Authors: Jane Pan, He He, Samuel R. Bowman, Shi Feng

Abstract: Language models are capable of iteratively improving their outputs based on natural language feedback, thus enabling in-context optimization of user preference. In place of human users, a second language model can be used as an evaluator, providing feedback along with numerical ratings which the generator attempts to optimize. However, because the evaluator is an imperfect proxy of user preference… ▽ More Language models are capable of iteratively improving their outputs based on natural language feedback, thus enabling in-context optimization of user preference. In place of human users, a second language model can be used as an evaluator, providing feedback along with numerical ratings which the generator attempts to optimize. However, because the evaluator is an imperfect proxy of user preference, this optimization can lead to reward hacking, where the evaluator's ratings improve while the generation quality remains stagnant or even decreases as judged by actual user preference. The concern of reward hacking is heightened in iterative self-refinement where the generator and the evaluator use the same underlying language model, in which case the optimization pressure can drive them to exploit shared vulnerabilities. Using an essay editing task, we show that iterative self-refinement leads to deviation between the language model evaluator and human judgment, demonstrating that reward hacking can occur spontaneously in-context with the use of iterative self-refinement. In addition, we study conditions under which reward hacking occurs and observe two factors that affect reward hacking severity: model size and context sharing between the generator and the evaluator. △ Less

Submitted 5 July, 2024; originally announced July 2024.

Is Your AI-Generated Code Really Safe? Evaluating Large Language Models on Secure Code Generation with CodeSecEval

Authors: Jiexin Wang, Xitong Luo, Liuwen Cao, Hongkui He, Hailin Huang, Jiayuan Xie, Adam Jatowt, Yi Cai

Abstract: Large language models (LLMs) have brought significant advancements to code generation and code repair, benefiting both novice and experienced developers. However, their training using unsanitized data from open-source repositories, like GitHub, raises the risk of inadvertently propagating security vulnerabilities. Despite numerous studies investigating the safety of code LLMs, there remains a gap… ▽ More Large language models (LLMs) have brought significant advancements to code generation and code repair, benefiting both novice and experienced developers. However, their training using unsanitized data from open-source repositories, like GitHub, raises the risk of inadvertently propagating security vulnerabilities. Despite numerous studies investigating the safety of code LLMs, there remains a gap in comprehensively addressing their security features. In this work, we aim to present a comprehensive study aimed at precisely evaluating and enhancing the security aspects of code LLMs. To support our research, we introduce CodeSecEval, a meticulously curated dataset designed to address 44 critical vulnerability types with 180 distinct samples. CodeSecEval serves as the foundation for the automatic evaluation of code models in two crucial tasks: code generation and code repair, with a strong emphasis on security. Our experimental results reveal that current models frequently overlook security issues during both code generation and repair processes, resulting in the creation of vulnerable code. In response, we propose different strategies that leverage vulnerability-aware information and insecure code explanations to mitigate these security vulnerabilities. Furthermore, our findings highlight that certain vulnerability types particularly challenge model performance, influencing their effectiveness in real-world applications. Based on these findings, we believe our study will have a positive impact on the software engineering community, inspiring the development of improved methods for training and utilizing LLMs, thereby leading to safer and more trustworthy model deployment. △ Less

Submitted 4 July, 2024; v1 submitted 2 July, 2024; originally announced July 2024.

Comments: arXiv admin note: text overlap with arXiv:2310.16263

PHANGS-MeerKAT and MHONGOOSE HI observations of nearby spiral galaxies: physical drivers of the molecular gas fraction, $R_{\mathrm{mol}}$

Authors: Cosima Eibensteiner, Jiayi Sun, Frank Bigiel, Adam K. Leroy, Eva Schinnerer, Erik Rosolowsky, Sushma Kurapati, D. J. Pisano, W. J. G de Blok, Ashley T. Barnes, Mallory Thorp, Dario Colombo, Eric W. Koch, I-Da Chiang, Eve C. Ostriker, Eric J. Murphy, Nikki Zabel, Sebstian Laudage, Filippo M. Maccagni, Julia Healy, Srikrishna Sekhar, Dyas Utomo, Jakob den Brok, Yixian Cao, Mélanie Chevance , et al. (14 additional authors not shown)

Abstract: The molecular-to-atomic gas ratio is crucial to the evolution of the interstellar medium in galaxies. We investigate the balance between the atomic ($Σ_{\rm HI}$) and molecular gas ($Σ_{\rm H2}$) surface densities in eight nearby star-forming galaxies using new high-quality observations from MeerKAT and ALMA (for HI and CO, respectively). We define the molecular gas ratio as… ▽ More The molecular-to-atomic gas ratio is crucial to the evolution of the interstellar medium in galaxies. We investigate the balance between the atomic ($Σ_{\rm HI}$) and molecular gas ($Σ_{\rm H2}$) surface densities in eight nearby star-forming galaxies using new high-quality observations from MeerKAT and ALMA (for HI and CO, respectively). We define the molecular gas ratio as $R_{\rm mol} = Σ_{\rm H2} / Σ_{\rm HI}$ and measure how it depends on local conditions in the galaxy disks using multi-wavelength observations. We find that, depending on the galaxy, HI is detected at $>3σ$ out to 20-120 kpc in galactocentric radius ($r_{\rm gal}$). The typical radius at which $Σ_{\rm HI}$ reaches 1~$\rm M_\odot~pc^{-2}$ is $r_{\rm HI}\approx22$~kpc, which corresponds to 1-3 times the optical radius ($r_{25}$). $R_{\rm mol}$ correlates best with the dynamical equilibrium pressure, P$_{\rm DE}$, among potential drivers studied, with a median correlation coefficient of $<ρ>=0.89$. Correlations between $R_{\rm mol}$ and star formation rate, total gas and stellar surface density, metallicity, and $Σ_{\rm SFR}$/P$_{\rm DE}$ are present but somewhat weaker. Our results also show a direct correlation between P$_{\rm DE}$ and $Σ_{\rm SFR}$, supporting self-regulation models. Quantitatively, we measure similar scalings as previous works and attribute the modest differences that we find to the effect of varying resolution and sensitivity. At $r_{\rm gal} {\gtrsim}0.4~r_{25}$, atomic gas dominates over molecular gas, and at the balance of these two gas phases, we find that the baryon mass is dominated by stars, with $Σ_{*} > 5~Σ_{\rm gas}$. Our study constitutes an important step in the statistical investigation of how local galaxy properties impact the conversion from atomic to molecular gas in nearby galaxies. △ Less

Submitted 1 July, 2024; originally announced July 2024.

Comments: accepted for publication in A&A; 20 pages, 12 Figures (+4 appendix pages)

Human-like object concept representations emerge naturally in multimodal large language models

Authors: Changde Du, Kaicheng Fu, Bincheng Wen, Yi Sun, Jie Peng, Wei Wei, Ying Gao, Shengpei Wang, Chuncheng Zhang, Jinpeng Li, Shuang Qiu, Le Chang, Huiguang He

Abstract: The conceptualization and categorization of natural objects in the human mind have long intrigued cognitive scientists and neuroscientists, offering crucial insights into human perception and cognition. Recently, the rapid development of Large Language Models (LLMs) has raised the attractive question of whether these models can also develop human-like object representations through exposure to vas… ▽ More The conceptualization and categorization of natural objects in the human mind have long intrigued cognitive scientists and neuroscientists, offering crucial insights into human perception and cognition. Recently, the rapid development of Large Language Models (LLMs) has raised the attractive question of whether these models can also develop human-like object representations through exposure to vast amounts of linguistic and multimodal data. In this study, we combined behavioral and neuroimaging analysis methods to uncover how the object concept representations in LLMs correlate with those of humans. By collecting large-scale datasets of 4.7 million triplet judgments from LLM and Multimodal LLM (MLLM), we were able to derive low-dimensional embeddings that capture the underlying similarity structure of 1,854 natural objects. The resulting 66-dimensional embeddings were found to be highly stable and predictive, and exhibited semantic clustering akin to human mental representations. Interestingly, the interpretability of the dimensions underlying these embeddings suggests that LLM and MLLM have developed human-like conceptual representations of natural objects. Further analysis demonstrated strong alignment between the identified model embeddings and neural activity patterns in many functionally defined brain ROIs (e.g., EBA, PPA, RSC and FFA). This provides compelling evidence that the object representations in LLMs, while not identical to those in the human, share fundamental commonalities that reflect key schemas of human conceptual knowledge. This study advances our understanding of machine intelligence and informs the development of more human-like artificial cognitive systems. △ Less

Submitted 1 July, 2024; originally announced July 2024.

Isotropy of cosmic rays beyond $10^{20}$ eV favors their heavy mass composition

Authors: Telescope Array Collaboration, R. U. Abbasi, Y. Abe, T. Abu-Zayyad, M. Allen, Y. Arai, R. Arimura, E. Barcikowski, J. W. Belz, D. R. Bergman, S. A. Blake, I. Buckland, B. G. Cheon, M. Chikawa, T. Fujii, K. Fujisue, K. Fujita, R. Fujiwara, M. Fukushima, G. Furlich, N. Globus, R. Gonzalez, W. Hanlon, N. Hayashida, H. He , et al. (118 additional authors not shown)

Abstract: We report an estimation of the injected mass composition of ultra-high energy cosmic rays (UHECRs) at energies higher than 10 EeV. The composition is inferred from an energy-dependent sky distribution of UHECR events observed by the Telescope Array surface detector by comparing it to the Large Scale Structure of the local Universe. In the case of negligible extra-galactic magnetic fields the resul… ▽ More We report an estimation of the injected mass composition of ultra-high energy cosmic rays (UHECRs) at energies higher than 10 EeV. The composition is inferred from an energy-dependent sky distribution of UHECR events observed by the Telescope Array surface detector by comparing it to the Large Scale Structure of the local Universe. In the case of negligible extra-galactic magnetic fields the results are consistent with a relatively heavy injected composition at E ~ 10 EeV that becomes lighter up to E ~ 100 EeV, while the composition at E > 100 EeV is very heavy. The latter is true even in the presence of highest experimentally allowed extra-galactic magnetic fields, while the composition at lower energies can be light if a strong EGMF is present. The effect of the uncertainty in the galactic magnetic field on these results is subdominant. △ Less

Submitted 3 July, 2024; v1 submitted 27 June, 2024; originally announced June 2024.

Comments: 8 pages, 3 figures, accepted for publication in PRL

Mass composition of ultra-high energy cosmic rays from distribution of their arrival directions with the Telescope Array

Authors: Telescope Array Collaboration, R. U. Abbasi, Y. Abe, T. Abu-Zayyad, M. Allen, Y. Arai, R. Arimura, E. Barcikowski, J. W. Belz, D. R. Bergman, S. A. Blake, I. Buckland, B. G. Cheon, M. Chikawa, T. Fujii, K. Fujisue, K. Fujita, R. Fujiwara, M. Fukushima, G. Furlich, N. Globus, R. Gonzalez, W. Hanlon, N. Hayashida, H. He , et al. (118 additional authors not shown)

Abstract: We use a new method to estimate the injected mass composition of ultrahigh cosmic rays (UHECRs) at energies higher than 10 EeV. The method is based on comparison of the energy-dependent distribution of cosmic ray arrival directions as measured by the Telescope Array experiment (TA) with that calculated in a given putative model of UHECR under the assumption that sources trace the large-scale struc… ▽ More We use a new method to estimate the injected mass composition of ultrahigh cosmic rays (UHECRs) at energies higher than 10 EeV. The method is based on comparison of the energy-dependent distribution of cosmic ray arrival directions as measured by the Telescope Array experiment (TA) with that calculated in a given putative model of UHECR under the assumption that sources trace the large-scale structure (LSS) of the Universe. As we report in the companion letter, the TA data show large deflections with respect to the LSS which can be explained, assuming small extra-galactic magnetic fields (EGMF), by an intermediate composition changing to a heavy one (iron) in the highest energy bin. Here we show that these results are robust to uncertainties in UHECR injection spectra, the energy scale of the experiment and galactic magnetic fields (GMF). The assumption of weak EGMF, however, strongly affects this interpretation at all but the highest energies E > 100 EeV, where the remarkable isotropy of the data implies a heavy injected composition even in the case of strong EGMF. This result also holds if UHECR sources are as rare as $2 \times 10^{-5}$ Mpc$^{-3}$, that is the conservative lower limit for the source number density. △ Less

Submitted 3 July, 2024; v1 submitted 27 June, 2024; originally announced June 2024.

Comments: 18 pages, 11 figures, accepted for publication in PRD

Self-assembly behaviour of diblock copolymer-diblock copolymer under oscillating shear field

Authors: Y. Guo, H. He, X. Fu

Abstract: The self-assembly behaviour of a diblock copolymer-diblock copolymer mixture under an oscillating shear field is investigated via cell dynamics simulation. The results indicate that the macrophase separation of the composite system is accompanied by the corresponding microphase separation induced by the oscillating shear field. With an increase in the shear frequency, the AB phase changes from a t… ▽ More The self-assembly behaviour of a diblock copolymer-diblock copolymer mixture under an oscillating shear field is investigated via cell dynamics simulation. The results indicate that the macrophase separation of the composite system is accompanied by the corresponding microphase separation induced by the oscillating shear field. With an increase in the shear frequency, the AB phase changes from a tilted layered structure to a parallel layered structure, and finally to a vertical layered structure. The CD phase transforms from the initial concentric ring into a parallel layer in the ring and then into a parallel layered structure; thus, the system finally forms a layered structure of the AB phase (vertical layer) and CD phase (parallel layer) perpendicular to each other. To verify the phase transition, the dynamic evolution of the domain size at different shear frequencies is analysed. The ordered phase transition with an increase in the oscillating shear field varies when the initial composition ratio of the system is changed. This conclusion provides a valuable guidance for the formation and transformation of ordered structures in experiments. △ Less

Submitted 30 June, 2024; v1 submitted 26 June, 2024; originally announced June 2024.

Comments: 13 pages, 7 figures

Journal ref: Condensed Matter Physics, 2024, Vol. 27, No. 2, 23801

A compact model of Escherichia coli core and biosynthetic metabolism

Authors: Marco Corrao, Hai He, Wolfram Liebermeister, Elad Noor

Abstract: Metabolic models condense biochemical knowledge about organisms in a structured and standardised way. As large-scale network reconstructions are readily available for many organisms of interest, genome-scale models are being widely used among modellers and engineers. However, these large models can be difficult to analyse and visualise, and occasionally generate hard-to-interpret or even biologica… ▽ More Metabolic models condense biochemical knowledge about organisms in a structured and standardised way. As large-scale network reconstructions are readily available for many organisms of interest, genome-scale models are being widely used among modellers and engineers. However, these large models can be difficult to analyse and visualise, and occasionally generate hard-to-interpret or even biologically unrealistic predictions. Out of the thousands of enzymatic reactions in a typical bacterial metabolism, only a few hundred comprise the metabolic pathways essential to produce energy carriers and biosynthetic precursors. These pathways carry relatively high flux, are central to maintaining and reproducing the cell, and provide precursors and energy to engineered metabolic pathways. Here, focusing on these central metabolic subsystems, we present a manually-curated medium-scale model of energy and biosynthesis metabolism for the well-studied prokaryote Escherichia coli K-12 MG1655. The model is a sub-network of the most recent genome-scale reconstruction, iML1515, and comes with an updated layer of database annotations, as well as a range of metabolic maps for visualisation. We enriched the stoichiometric network with extensive biological information and quantitative data, enhancing the scope and applicability of the model. In addition, here we assess the properties of this model in relation to its genome-scale parent and demonstrate the use of the network and supporting data in various scenarios, including enzyme-constrained flux balance analysis, elementary flux mode analysis, and thermodynamic analysis. Overall, we believe this model holds the potential to become a reference medium-scale metabolic model for E. coli. △ Less

Submitted 24 June, 2024; originally announced June 2024.

Low-Complexity CSI Feedback for FDD Massive MIMO Systems via Learning to Optimize

Authors: Yifan Ma, Hengtao He, Shenghui Song, Jun Zhang, Khaled B. Letaief

Abstract: In frequency-division duplex (FDD) massive multiple-input multiple-output (MIMO) systems, the growing number of base station antennas leads to prohibitive feedback overhead for downlink channel state information (CSI). To address this challenge, state-of-the-art (SOTA) fully data-driven deep learning (DL)-based CSI feedback schemes have been proposed. However, the high computational complexity and… ▽ More In frequency-division duplex (FDD) massive multiple-input multiple-output (MIMO) systems, the growing number of base station antennas leads to prohibitive feedback overhead for downlink channel state information (CSI). To address this challenge, state-of-the-art (SOTA) fully data-driven deep learning (DL)-based CSI feedback schemes have been proposed. However, the high computational complexity and memory requirements of these methods hinder their practical deployment on resource-constrained devices like mobile phones. To solve the problem, we propose a model-driven DL-based CSI feedback approach by integrating the wisdom of compressive sensing and learning to optimize (L2O). Specifically, only a linear learnable projection is adopted at the encoder side to compress the CSI matrix, thereby significantly cutting down the user-side complexity and memory expenditure. On the other hand, the decoder incorporates two specially designed components, i.e., a learnable sparse transformation and an element-wise L2O reconstruction module. The former is developed to learn a sparse basis for CSI within the angular domain, which explores channel sparsity effectively. The latter shares the same long short term memory (LSTM) network across all elements of the optimization variable, eliminating the retraining cost when problem scale changes. Simulation results show that the proposed method achieves a comparable performance with the SOTA CSI feedback scheme but with much-reduced complexity, and enables multiple-rate feedback. △ Less

Submitted 24 June, 2024; originally announced June 2024.

Comments: submitted to IEEE for publication

Imperative Learning: A Self-supervised Neural-Symbolic Learning Framework for Robot Autonomy

Authors: Chen Wang, Kaiyi Ji, Junyi Geng, Zhongqiang Ren, Taimeng Fu, Fan Yang, Yifan Guo, Haonan He, Xiangyu Chen, Zitong Zhan, Qiwei Du, Shaoshu Su, Bowen Li, Yuheng Qiu, Yi Du, Qihang Li, Yifan Yang, Xiao Lin, Zhipeng Zhao

Abstract: Data-driven methods such as reinforcement and imitation learning have achieved remarkable success in robot autonomy. However, their data-centric nature still hinders them from generalizing well to ever-changing environments. Moreover, collecting large datasets for robotic tasks is often impractical and expensive. To overcome these challenges, we introduce a new self-supervised neural-symbolic (NeS… ▽ More Data-driven methods such as reinforcement and imitation learning have achieved remarkable success in robot autonomy. However, their data-centric nature still hinders them from generalizing well to ever-changing environments. Moreover, collecting large datasets for robotic tasks is often impractical and expensive. To overcome these challenges, we introduce a new self-supervised neural-symbolic (NeSy) computational framework, imperative learning (IL), for robot autonomy, leveraging the generalization abilities of symbolic reasoning. The framework of IL consists of three primary components: a neural module, a reasoning engine, and a memory system. We formulate IL as a special bilevel optimization (BLO), which enables reciprocal learning over the three modules. This overcomes the label-intensive obstacles associated with data-driven approaches and takes advantage of symbolic reasoning concerning logical reasoning, physical principles, geometric analysis, etc. We discuss several optimization techniques for IL and verify their effectiveness in five distinct robot autonomy tasks including path planning, rule induction, optimal control, visual odometry, and multi-robot routing. Through various experiments, we show that IL can significantly enhance robot autonomy capabilities and we anticipate that it will catalyze further research across diverse domains. △ Less

Submitted 6 August, 2024; v1 submitted 23 June, 2024; originally announced June 2024.

One Fling to Goal: Environment-aware Dynamics for Goal-conditioned Fabric Flinging

Authors: Linhan Yang, Lei Yang, Haoran Sun, Zeqing Zhang, Haibin He, Fang Wan, Chaoyang Song, Jia Pan

Abstract: Fabric manipulation dynamically is commonly seen in manufacturing and domestic settings. While dynamically manipulating a fabric piece to reach a target state is highly efficient, this task presents considerable challenges due to the varying properties of different fabrics, complex dynamics when interacting with environments, and meeting required goal conditions. To address these challenges, we pr… ▽ More Fabric manipulation dynamically is commonly seen in manufacturing and domestic settings. While dynamically manipulating a fabric piece to reach a target state is highly efficient, this task presents considerable challenges due to the varying properties of different fabrics, complex dynamics when interacting with environments, and meeting required goal conditions. To address these challenges, we present \textit{One Fling to Goal}, an algorithm capable of handling fabric pieces with diverse shapes and physical properties across various scenarios. Our method learns a graph-based dynamics model equipped with environmental awareness. With this dynamics model, we devise a real-time controller to enable high-speed fabric manipulation in one attempt, requiring less than 3 seconds to finish the goal-conditioned task. We experimentally validate our method on a goal-conditioned manipulation task in five diverse scenarios. Our method significantly improves this goal-conditioned task, achieving an average error of 13.2mm in complex scenarios. Our method can be seamlessly transferred to real-world robotic systems and generalized to unseen scenarios in a zero-shot manner. △ Less

Submitted 20 June, 2024; originally announced June 2024.

Enhancing Monotonic Modeling with Spatio-Temporal Adaptive Awareness in Diverse Marketing

Authors: Bin Li, Jiayan Pei, Feiyang Xiao, Yifan Zhao, Zhixing Zhang, Diwei Liu, HengXu He, Jia Jia

Abstract: In the mobile internet era, the Online Food Ordering Service (OFOS) emerges as an integral component of inclusive finance owing to the convenience it brings to people. OFOS platforms offer dynamic allocation incentives to users and merchants through diverse marketing campaigns to encourage payments while maintaining the platforms' budget efficiency. Despite significant progress, the marketing doma… ▽ More In the mobile internet era, the Online Food Ordering Service (OFOS) emerges as an integral component of inclusive finance owing to the convenience it brings to people. OFOS platforms offer dynamic allocation incentives to users and merchants through diverse marketing campaigns to encourage payments while maintaining the platforms' budget efficiency. Despite significant progress, the marketing domain continues to face two primary challenges: (i) how to allocate a limited budget with greater efficiency, demanding precision in predicting users' monotonic response (i.e. sensitivity) to incentives, and (ii) ensuring spatio-temporal adaptability and robustness in diverse marketing campaigns across different times and locations. To address these issues, we propose a Constrained Monotonic Adaptive Network (CoMAN) method for spatio-temporal perception within marketing pricing. Specifically, we capture spatio-temporal preferences within attribute features through two foundational spatio-temporal perception modules. To further enhance catching the user sensitivity differentials to incentives across varied times and locations, we design modules for learning spatio-temporal convexity and concavity as well as for expressing sensitivity functions. CoMAN can achieve a more efficient allocation of incentive investments during pricing, thus increasing the conversion rate and orders while maintaining budget efficiency. Extensive offline and online experimental results within our diverse marketing campaigns demonstrate the effectiveness of the proposed approach while outperforming the monotonic state-of-the-art method. △ Less

Submitted 20 June, 2024; originally announced June 2024.

Comments: 7 pages

Topological Equivalence Theorem and Double-Copy for Chern-Simons Scattering Amplitudes

Authors: Yan-Feng Hang, Hong-Jian He, Cong Shen

Abstract: We study the mechanism of topological mass-generation for 3d Chern-Simons gauge theories and propose a brand-new Topological Equivalence Theorem to connect scattering amplitudes of the physical gauge boson states to that of the transverse states under high energy expansion. We prove a general energy cancellation mechanism for $N$-point physical gauge boson amplitudes, which predicts large cancella… ▽ More We study the mechanism of topological mass-generation for 3d Chern-Simons gauge theories and propose a brand-new Topological Equivalence Theorem to connect scattering amplitudes of the physical gauge boson states to that of the transverse states under high energy expansion. We prove a general energy cancellation mechanism for $N$-point physical gauge boson amplitudes, which predicts large cancellations of $E^{4-L}\to E^{(4-L)- N}$ at any $L$-loop level ($L\geqslant 0$). We extend the double-copy approach to construct massive graviton amplitudes and study their structures. We newly uncover a series of strikingly large energy cancellations $E^{12}\to E^1$ of the tree-level four-graviton scattering amplitude under high energy expansion and establish a new correspondence between the two energy cancellations in the topologically massive Yang-Mills gauge theory and the topologically massive gravity theory. We further study the scattering amplitudes of Chern-Simons gauge bosons and gravitons in the nonrelativistic limit. △ Less

Submitted 19 June, 2024; originally announced June 2024.

Comments: 8+5 pages, to match Journal Version. This Letter paper is a companion of the systematic longer paper arXiv:2110.05399 (46pp)

Journal ref: Research 6 (2023) 0072

Low-Rank Mixture-of-Experts for Continual Medical Image Segmentation

Authors: Qian Chen, Lei Zhu, Hangzhou He, Xinliang Zhang, Shuang Zeng, Qiushi Ren, Yanye Lu

Abstract: The primary goal of continual learning (CL) task in medical image segmentation field is to solve the "catastrophic forgetting" problem, where the model totally forgets previously learned features when it is extended to new categories (class-level) or tasks (task-level). Due to the privacy protection, the historical data labels are inaccessible. Prevalent continual learning methods primarily focus… ▽ More The primary goal of continual learning (CL) task in medical image segmentation field is to solve the "catastrophic forgetting" problem, where the model totally forgets previously learned features when it is extended to new categories (class-level) or tasks (task-level). Due to the privacy protection, the historical data labels are inaccessible. Prevalent continual learning methods primarily focus on generating pseudo-labels for old datasets to force the model to memorize the learned features. However, the incorrect pseudo-labels may corrupt the learned feature and lead to a new problem that the better the model is trained on the old task, the poorer the model performs on the new tasks. To avoid this problem, we propose a network by introducing the data-specific Mixture of Experts (MoE) structure to handle the new tasks or categories, ensuring that the network parameters of previous tasks are unaffected or only minimally impacted. To further overcome the tremendous memory costs caused by introducing additional structures, we propose a Low-Rank strategy which significantly reduces memory cost. We validate our method on both class-level and task-level continual learning challenges. Extensive experiments on multiple datasets show our model outperforms all other methods. △ Less

Submitted 19 June, 2024; originally announced June 2024.

Multi-scale Restoration of Missing Data in Optical Time-series Images with Masked Spatial-Temporal Attention Network

Authors: Zaiyan Zhang, Jining Yan, Yuanqi Liang, Jiaxin Feng, Haixu He, Wei Han

Abstract: Due to factors such as thick cloud cover and sensor limitations, remote sensing images often suffer from significant missing data, resulting in incomplete time-series information. Existing methods for imputing missing values in remote sensing images do not fully exploit spatio-temporal auxiliary information, leading to limited accuracy in restoration. Therefore, this paper proposes a novel deep le… ▽ More Due to factors such as thick cloud cover and sensor limitations, remote sensing images often suffer from significant missing data, resulting in incomplete time-series information. Existing methods for imputing missing values in remote sensing images do not fully exploit spatio-temporal auxiliary information, leading to limited accuracy in restoration. Therefore, this paper proposes a novel deep learning-based approach called MS2TAN (Multi-scale Masked Spatial-Temporal Attention Network), for reconstructing time-series remote sensing images. Firstly, we introduce an efficient spatio-temporal feature extractor based on Masked Spatial-Temporal Attention (MSTA), to obtain high-quality representations of the spatio-temporal neighborhood features in the missing regions. Secondly, a Multi-scale Restoration Network consisting of the MSTA-based Feature Extractors, is employed to progressively refine the missing values by exploring spatio-temporal neighborhood features at different scales. Thirdly, we propose a ``Pixel-Structure-Perception'' Multi-Objective Joint Optimization method to enhance the visual effects of the reconstruction results from multiple perspectives and preserve more texture structures. Furthermore, the proposed method reconstructs missing values in all input temporal phases in parallel (i.e., Multi-In Multi-Out), achieving higher processing efficiency. Finally, experimental evaluations on two typical missing data restoration tasks across multiple research areas demonstrate that the proposed method outperforms state-of-the-art methods with an improvement of 0.40dB/1.17dB in mean peak signal-to-noise ratio (mPSNR) and 3.77/9.41 thousandths in mean structural similarity (mSSIM), while exhibiting stronger texture and structural consistency. △ Less

Submitted 19 June, 2024; originally announced June 2024.

Structure of Massive Gauge/Gravity Scattering Amplitudes, Equivalence Theorems, and Extended Double-Copy with Compactified Warped Space

Authors: Yanfeng Hang, Wei-Wei Zhao, Hong-Jian He, Yin-Long Qiu

Abstract: We study the structure of scattering amplitudes of massive Kaluza-Klein (KK) states in the compactified 5-dimensional warped gauge and gravity theories. We present systematic formulations of the gauge theory equivalence theorem (GAET) and the gravitational equivalence theorem (GRET) for warped KK theories in $R_ξ^{}$ gauge, where the GAET connects the scattering amplitudes of longitudinal KK gauge… ▽ More We study the structure of scattering amplitudes of massive Kaluza-Klein (KK) states in the compactified 5-dimensional warped gauge and gravity theories. We present systematic formulations of the gauge theory equivalence theorem (GAET) and the gravitational equivalence theorem (GRET) for warped KK theories in $R_ξ^{}$ gauge, where the GAET connects the scattering amplitudes of longitudinal KK gauge bosons to that of the corresponding KK Goldstone bosons and the GRET connects the scattering amplitudes of KK gravitons of helicity-zero (helicity-one) to that of the corresponding gravitational KK Goldstone bosons. We analyze the structure of 3-point and 4-point scattering amplitudes of massive KK gauge bosons and of massive KK gravitons as well as their corresponding Goldstone bosons. We first prove the GAET and GRET explicitly for the fundamental 3-point KK gauge/gravity scattering amplitudes. We then demonstrate that the validity of the GAET and GRET for 4-point gauge/gravity scattering amplitudes can be reduced to the validity of GAET and GRET for 3-point gauge/gravity scattering amplitudes at tree level. With these, we study the double-copy construction of KK scattering amplitudes in the warped gauge/gravity theories. We newly realize the double-copy for massive 3-point full gauge/gravity amplitudes at tree level under proper correspondences of color-kinematics and of gauge/gravity couplings, whereas we can construct the double-copy for 4-point KK gauge/gravity amplitudes to the leading order (LO) of high energy expansion. We further demonstrate that this LO double-copy construction can be extended to $N$-point KK scattering amplitudes with $N\geqslant 4$. △ Less

Submitted 27 June, 2024; v1 submitted 18 June, 2024; originally announced June 2024.

Comments: 91 pages. Improved version, references added

LLMs Are Prone to Fallacies in Causal Inference

Authors: Nitish Joshi, Abulhair Saparov, Yixin Wang, He He

Abstract: Recent work shows that causal facts can be effectively extracted from LLMs through prompting, facilitating the creation of causal graphs for causal inference tasks. However, it is unclear if this success is limited to explicitly-mentioned causal facts in the pretraining data which the model can memorize. Thus, this work investigates: Can LLMs infer causal relations from other relational data in te… ▽ More Recent work shows that causal facts can be effectively extracted from LLMs through prompting, facilitating the creation of causal graphs for causal inference tasks. However, it is unclear if this success is limited to explicitly-mentioned causal facts in the pretraining data which the model can memorize. Thus, this work investigates: Can LLMs infer causal relations from other relational data in text? To disentangle the role of memorized causal facts vs inferred causal relations, we finetune LLMs on synthetic data containing temporal, spatial and counterfactual relations, and measure whether the LLM can then infer causal relations. We find that: (a) LLMs are susceptible to inferring causal relations from the order of two entity mentions in text (e.g. X mentioned before Y implies X causes Y); (b) if the order is randomized, LLMs still suffer from the post hoc fallacy, i.e. X occurs before Y (temporal relation) implies X causes Y. We also find that while LLMs can correctly deduce the absence of causal relations from temporal and spatial relations, they have difficulty inferring causal relations from counterfactuals, questioning their understanding of causality. △ Less

Submitted 17 June, 2024; originally announced June 2024.

Constraints on Ultra Heavy Dark Matter Properties from Dwarf Spheroidal Galaxies with LHAASO Observations

Authors: Zhen Cao, F. Aharonian, Q. An, Axikegu, Y. X. Bai, Y. W. Bao, D. Bastieri, X. J. Bi, Y. J. Bi, J. T. Cai, Q. Cao, W. Y. Cao, Zhe Cao, J. Chang, J. F. Chang, A. M. Chen, E. S. Chen, Liang Chen, Lin Chen, Long Chen, M. J. Chen, M. L. Chen, Q. H. Chen, S. H. Chen, S. Z. Chen , et al. (255 additional authors not shown)

Abstract: In this work we try to search for signals generated by ultra-heavy dark matter at the Large High Altitude Air Shower Observatory (LHAASO) data. We look for possible gamma-ray by dark matter annihilation or decay from 16 dwarf spheroidal galaxies in the field of view of LHAASO. Dwarf spheroidal galaxies are among the most promising targets for indirect detection of dark matter which have low fluxes… ▽ More In this work we try to search for signals generated by ultra-heavy dark matter at the Large High Altitude Air Shower Observatory (LHAASO) data. We look for possible gamma-ray by dark matter annihilation or decay from 16 dwarf spheroidal galaxies in the field of view of LHAASO. Dwarf spheroidal galaxies are among the most promising targets for indirect detection of dark matter which have low fluxes of astrophysical $γ$-ray background while large amount of dark matter. By analyzing more than 700 days observational data at LHAASO, no significant dark matter signal from 1 TeV to 1 EeV is detected. Accordingly we derive the most stringent constraints on the ultra-heavy dark matter annihilation cross-section up to EeV. The constraints on the lifetime of dark matter in decay mode are also derived. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: 17 pages, 12 figures, accepted by PRL

Observation of Declination Dependence in the Cosmic Ray Energy Spectrum

Authors: The Telescope Array Collaboration, R. U. Abbasi, T. Abu-Zayyad, M. Allen, J. W. Belz, D. R. Bergman, I. Buckland, W. Campbell, B. G. Cheon, K. Endo, A. Fedynitch, T. Fujii, K. Fujisue, K. Fujita, M. Fukushima, G. Furlich, Z. Gerber, N. Globus, W. Hanlon, N. Hayashida, H. He, K. Hibino, R. Higuchi, D. Ikeda, T. Ishii , et al. (101 additional authors not shown)

Abstract: We report on an observation of the difference between northern and southern skies of the ultrahigh energy cosmic ray energy spectrum with a significance of ${\sim}8σ$. We use measurements from the two largest experiments$\unicode{x2014}$the Telescope Array observing the northern hemisphere and the Pierre Auger Observatory viewing the southern hemisphere. Since the comparison of two measurements fr… ▽ More We report on an observation of the difference between northern and southern skies of the ultrahigh energy cosmic ray energy spectrum with a significance of ${\sim}8σ$. We use measurements from the two largest experiments$\unicode{x2014}$the Telescope Array observing the northern hemisphere and the Pierre Auger Observatory viewing the southern hemisphere. Since the comparison of two measurements from different observatories introduces the issue of possible systematic differences between detectors and analyses, we validate the methodology of the comparison by examining the region of the sky where the apertures of the two observatories overlap. Although the spectra differ in this region, we find that there is only a $1.8σ$ difference between the spectrum measurements when anisotropic regions are removed and a fiducial cut in the aperture is applied. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: 8 pages, 6 figures

MAP: Low-compute Model Merging with Amortized Pareto Fronts via Quadratic Approximation

Authors: Lu Li, Tianyu Zhang, Zhiqi Bu, Suyuchen Wang, Huan He, Jie Fu, Yonghui Wu, Jiang Bian, Yong Chen, Yoshua Bengio

Abstract: Model merging has emerged as an effective approach to combine multiple single-task models, fine-tuned from the same pre-trained model, into a multitask model. This process typically involves computing a weighted average of the model parameters without any additional training. Existing model-merging methods focus on enhancing average task accuracy. However, interference and conflicts between the ob… ▽ More Model merging has emerged as an effective approach to combine multiple single-task models, fine-tuned from the same pre-trained model, into a multitask model. This process typically involves computing a weighted average of the model parameters without any additional training. Existing model-merging methods focus on enhancing average task accuracy. However, interference and conflicts between the objectives of different tasks can lead to trade-offs during model merging. In real-world applications, a set of solutions with various trade-offs can be more informative, helping practitioners make decisions based on diverse preferences. In this paper, we introduce a novel low-compute algorithm, Model Merging with Amortized Pareto Front (MAP). MAP identifies a Pareto set of scaling coefficients for merging multiple models to reflect the trade-offs. The core component of MAP is approximating the evaluation metrics of the various tasks using a quadratic approximation surrogate model derived from a pre-selected set of scaling coefficients, enabling amortized inference. Experimental results on vision and natural language processing tasks show that MAP can accurately identify the Pareto front. To further reduce the required computation of MAP, we propose (1) a Bayesian adaptive sampling algorithm and (2) a nested merging scheme with multiple stages. △ Less

Submitted 18 June, 2024; v1 submitted 11 June, 2024; originally announced June 2024.

Near-Field Channel Estimation for Extremely Large-Scale Terahertz Communications

Authors: Songjie Yang, Yizhou Peng, Wanting Lyu, Ya Li, Hongjun He, Zhongpei Zhang, Chau Yuen

Abstract: Future Terahertz communications exhibit significant potential in accommodating ultra-high-rate services. Employing extremely large-scale array antennas is a key approach to realize this potential, as they can harness substantial beamforming gains to overcome the severe path loss and leverage the electromagnetic advantages in the near field. This paper proposes novel estimation methods designed to… ▽ More Future Terahertz communications exhibit significant potential in accommodating ultra-high-rate services. Employing extremely large-scale array antennas is a key approach to realize this potential, as they can harness substantial beamforming gains to overcome the severe path loss and leverage the electromagnetic advantages in the near field. This paper proposes novel estimation methods designed to enhance efficiency in Terahertz widely-spaced multi-subarray (WSMS) systems. Initially, we introduce three sparse channel representation methods: polar-domain representation (PD-R), multi-angular-domain representation (MAD-R), and two-dimensional polar-angular-domain representation (2D-PAD-R). Each method is meticulously developed for near-field WSMS channels, capitalizing on their sparsity characteristics. Building on this, we propose four estimation frameworks using the sparse recovery theory: polar-domain estimation (PD-E), multi-angular-domain estimation (MAD-E), two-stage polar-angular-domain estimation (TS-PAD-E), and two-dimensional polar-angular-domain estimation (2D-PAD-E). Particularly, 2D-PAD-E, integrating a 2D dictionary process, and TS-PAD-E, with its sequential approach to angle and distance estimation, stand out as particularly effective for near-field angle-distance estimation, enabling decoupled calculation of these parameters. Overall, these frameworks provide versatile and efficient solutions for WSMS channel estimation, balancing low complexity with high-performance outcomes. Additionally, they represent a fresh perspective on near-field signal processing. △ Less

Submitted 8 June, 2024; originally announced June 2024.