Showing 1–50 of 715 results for author: Wei, S

OctFusion: Octree-based Diffusion Models for 3D Shape Generation

Authors: Bojun Xiong, Si-Tong Wei, Xin-Yang Zheng, Yan-Pei Cao, Zhouhui Lian, Peng-Shuai Wang

Abstract: Diffusion models have emerged as a popular method for 3D generation. However, it is still challenging for diffusion models to efficiently generate diverse and high-quality 3D shapes. In this paper, we introduce OctFusion, which can generate 3D shapes with arbitrary resolutions in 2.5 seconds on a single Nvidia 4090 GPU, and the extracted meshes are guaranteed to be continuous and manifold. The key… ▽ More Diffusion models have emerged as a popular method for 3D generation. However, it is still challenging for diffusion models to efficiently generate diverse and high-quality 3D shapes. In this paper, we introduce OctFusion, which can generate 3D shapes with arbitrary resolutions in 2.5 seconds on a single Nvidia 4090 GPU, and the extracted meshes are guaranteed to be continuous and manifold. The key components of OctFusion are the octree-based latent representation and the accompanying diffusion models. The representation combines the benefits of both implicit neural representations and explicit spatial octrees and is learned with an octree-based variational autoencoder. The proposed diffusion model is a unified multi-scale U-Net that enables weights and computation sharing across different octree levels and avoids the complexity of widely used cascaded diffusion schemes. We verify the effectiveness of OctFusion on the ShapeNet and Objaverse datasets and achieve state-of-the-art performances on shape generation tasks. We demonstrate that OctFusion is extendable and flexible by generating high-quality color fields for textured mesh generation and high-quality 3D shapes conditioned on text prompts, sketches, or category labels. Our code and pre-trained models are available at \url{https://github.com/octree-nn/octfusion}. △ Less

Submitted 26 August, 2024; originally announced August 2024.

Comments: Technical Report

Black hole thermodynamics from an ensemble-averaged theory

Authors: Peng Cheng, Yu-Xiao Liu, Shao-Wen Wei

Abstract: The path integral approach to a quantum theory of gravity is widely regarded as an indispensable strategy. However, determining what additional elements, beyond black hole or AdS spacetime, should be incorporated into the path integral remains crucial yet perplexing. We argue that the spacetime with a conical singularity in its Euclidean counterpart should be the most important ingredient to appen… ▽ More The path integral approach to a quantum theory of gravity is widely regarded as an indispensable strategy. However, determining what additional elements, beyond black hole or AdS spacetime, should be incorporated into the path integral remains crucial yet perplexing. We argue that the spacetime with a conical singularity in its Euclidean counterpart should be the most important ingredient to append to the path integral. Therefore, physical quantities should be ensemble-averaged over all geometries since they are described by the same Lorentzian metric. When the ensemble average is introduced, the Hawking-Page transition for the Schwarzschild-AdS black hole and the small-large black hole transition for the Reissner-Nordstrom-AdS black hole naturally arise as semi-classical approximations, when the size of the black hole system is much larger than the Planck length. Away from the semi-classical limit, the system is a superposition of different geometries, and the averaged quantities would deviate from the black hole thermodynamics. Expanding around the classical saddles, the subleading order of Newton's constant contributions can be derived, which are both half of the Hawking temperature for the Schwarzschild and Reissner-Nordstrom black holes. The result may imply a universal structure. The subsubleading terms and more intriguing physics that diverge from black hole thermodynamics are revealed. The ensemble-averaged theory provides a new way of studying subleading effects and extending the traditional AdS/CFT correspondence. △ Less

Submitted 18 August, 2024; originally announced August 2024.

Comments: 7 pages, 4 figures

A quantum-classical hybrid algorithm with Ising model for the learning with errors problem

Authors: Muxi Zheng, Jinfeng Zeng, Wentao Yang, Pei-Jie Chang, Bao Yan, Haoran Zhang, Min Wang, Shijie Wei, Gui-Lu Long

Abstract: The Learning-With-Errors (LWE) problem is a crucial computational challenge with significant implications for post-quantum cryptography and computational learning theory. Here we propose a quantum-classical hybrid algorithm with Ising model (HAWI) to address the LWE problem. Our approach involves transforming the LWE problem into the Shortest Vector Problem (SVP), using variable qubits to encode l… ▽ More The Learning-With-Errors (LWE) problem is a crucial computational challenge with significant implications for post-quantum cryptography and computational learning theory. Here we propose a quantum-classical hybrid algorithm with Ising model (HAWI) to address the LWE problem. Our approach involves transforming the LWE problem into the Shortest Vector Problem (SVP), using variable qubits to encode lattice vectors into an Ising Hamiltonian. We then identify the low-energy levels of the Hamiltonian to extract the solution, making it suitable for implementation on current noisy intermediate-scale quantum (NISQ) devices. We prove that the number of qubits required is less than $m(3m-1)/2$, where $m$ is the number of samples in the algorithm. Our algorithm is heuristic, and its time complexity depends on the specific quantum algorithm employed to find the Hamiltonian's low-energy levels. If the Quantum Approximate Optimization Algorithm (QAOA) is used to solve the Ising Hamiltonian problem, and the number of iterations satisfies $y < O\left(m\log m\cdot 2^{0.2972k}/pk^2\right)$, our algorithm will outperform the classical Block Korkine-Zolotarev (BKZ) algorithm, where $k$ is the block size related to problem parameters, and $p$ is the number of layers in QAOA. We demonstrate the algorithm by solving a $2$-dimensional LWE problem on a real quantum device with $5$ qubits, showing its potential for solving meaningful instances of the LWE problem in the NISQ era. △ Less

Submitted 15 August, 2024; originally announced August 2024.

Operator Feature Neural Network for Symbolic Regression

Authors: Yusong Deng, Min Wu, Lina Yu, Jingyi Liu, Shu Wei, Yanjie Li, Weijun Li

Abstract: Symbolic regression is a task aimed at identifying patterns in data and representing them through mathematical expressions, generally involving skeleton prediction and constant optimization. Many methods have achieved some success, however they treat variables and symbols merely as characters of natural language without considering their mathematical essence. This paper introduces the operator fea… ▽ More Symbolic regression is a task aimed at identifying patterns in data and representing them through mathematical expressions, generally involving skeleton prediction and constant optimization. Many methods have achieved some success, however they treat variables and symbols merely as characters of natural language without considering their mathematical essence. This paper introduces the operator feature neural network (OF-Net) which employs operator representation for expressions and proposes an implicit feature encoding method for the intrinsic mathematical operational logic of operators. By substituting operator features for numeric loss, we can predict the combination of operators of target expressions. We evaluate the model on public datasets, and the results demonstrate that the model achieves superior recovery rates and high $R^2$ scores. With the discussion of the results, we analyze the merit and demerit of OF-Net and propose optimizing schemes. △ Less

Submitted 14 August, 2024; originally announced August 2024.

Comments: 12 pages

Evolving Virtual World with Delta-Engine

Authors: Hongqiu Wu, Zekai Xu, Tianyang Xu, Shize Wei, Yan Wang, Jiale Hong, Weiqi Wu, Hai Zhao, Min Zhang, Zhezhi He

Abstract: In this paper, we focus on the \emph{virtual world}, a cyberspace where people can live in. An ideal virtual world shares great similarity with our real world. One of the crucial aspects is its evolving nature, reflected by individuals' capability to grow and thereby influence the objective world. Such dynamics is unpredictable and beyond the reach of existing systems. For this, we propose a speci… ▽ More In this paper, we focus on the \emph{virtual world}, a cyberspace where people can live in. An ideal virtual world shares great similarity with our real world. One of the crucial aspects is its evolving nature, reflected by individuals' capability to grow and thereby influence the objective world. Such dynamics is unpredictable and beyond the reach of existing systems. For this, we propose a special engine called \textbf{\emph{Delta-Engine}} to drive this virtual world. $Δ$ associates the world's evolution to the engine's scalability. It consists of a base engine and a neural proxy. The base engine programs the prototype of the virtual world; given a trigger, the neural proxy generates new snippets on the base engine through \emph{incremental prediction}. This paper presents a full-stack introduction to the delta-engine. The key feature of the delta-engine is its scalability to unknown elements within the world, Technically, it derives from the prefect co-work of the neural proxy and the base engine, and the alignment with high-quality data. We introduce an engine-oriented fine-tuning method that embeds the base engine into the proxy. We then discuss the human-LLM collaborative design to produce novel and interesting data efficiently. Eventually, we propose three evaluation principles to comprehensively assess the performance of a delta engine: naive evaluation, incremental evaluation, and adversarial evaluation. △ Less

Submitted 30 August, 2024; v1 submitted 11 August, 2024; originally announced August 2024.

Closed-Loop Magnetic Control of Medical Soft Continuum Robots for Deflection

Authors: Zhiwei Wu, Siyi Wei, Zhanxin Geng, Jinhui Zhang, Duanduan Chen

Abstract: Magnetic soft continuum robots (MSCRs) have emerged as powerful devices in endovascular interventions owing to their hyperelastic fibre matrix and enhanced magnetic manipulability. Effective closed-loop control of tethered magnetic devices contributes to the achievement of autonomous vascular robotic surgery. In this article, we employ a magnetic actuation system equipped with a single rotatable p… ▽ More Magnetic soft continuum robots (MSCRs) have emerged as powerful devices in endovascular interventions owing to their hyperelastic fibre matrix and enhanced magnetic manipulability. Effective closed-loop control of tethered magnetic devices contributes to the achievement of autonomous vascular robotic surgery. In this article, we employ a magnetic actuation system equipped with a single rotatable permanent magnet to achieve closed-loop deflection control of the MSCR. To this end, we establish a differential kinematic model of MSCRs exposed to non-uniform magnetic fields. The relationship between the existence and uniqueness of Jacobian and the geometric position between robots is deduced. The accurate control direction induced by Jacobian is demonstrated to be crucial in simulations. Then, the corresponding quasi-static control (QSC) framework integrates a linear extended state observer to estimate model uncertainties. Finally, the effectiveness of the proposed QSC framework is validated through comparative trajectory tracking experiments with the PD controller under external disturbances. The proposed control framework effectively prevents the actuator from reaching the joint limit and achieves fast and low error-tracking performance without overshooting. △ Less

Submitted 6 August, 2024; originally announced August 2024.

Holographic thermodynamics of a five-dimensional neutral Gauss-Bonnet AdS black hole

Authors: Si-Jiang Yang, Md Sabir Ali, Shao-Wen Wei, Yu-Xiao Liu

Abstract: Motivated by the recent progress on the holographic dual of the extended thermodynamics for black holes in anti-de-Sitter (AdS) space, we investiggate the hologrphic thermodynamics for the five-dimensional neutral Gauss-Bonnet AdS black hole in the context of the anti-de Sitter/conformal field theory (AdS/CFT) correspondence. Through the extended bulk thermodynamics for the five-dimensional Gauss-… ▽ More Motivated by the recent progress on the holographic dual of the extended thermodynamics for black holes in anti-de-Sitter (AdS) space, we investiggate the hologrphic thermodynamics for the five-dimensional neutral Gauss-Bonnet AdS black hole in the context of the anti-de Sitter/conformal field theory (AdS/CFT) correspondence. Through the extended bulk thermodynamics for the five-dimensional Gauss-Bonnet AdS black hole, we derive the first law of the CFT thermodynamics which is obtained by directly translating the arbitrary conformal factors in the dual CFT. In addition to the newly defined chemical potential $μ$ conjugating to the central charge $C$, we obtain other pairs of thermodynamics for the CFT, such as the temperature $\tilde{T}$ and the entropy $S$, the Gauss-Bonnet coupling constant $\tildeα$ and its conjugate variable $ \tilde{\mathcal{A}}$, the pressure $\mathcal{P}$ and its conjugate volume $\mathcal{V}$. In the fixed $C$, $\mathcal{V}$ and $\tildeα$ canonical ensemble, we obtain the canonical description of the CFT thermodynamics and observe the standard swallowtail behavior in the Helmholtz free energy vs the temperature plot. The self-intersection point of the Helmholtz free energy indicates the phase transition between the high and low entropy states of the CFT. By using Maxwell's equal area law, we get the critical point and coexistence curve for the high and low entropy phases of the CFT. Besides, we get the critical exponents for the CFT, and find that the critical point and critical exponents associated with the $\tilde{T}-S$ criticality of the CFT are the same as those of the five-dimensional Gauss-Bonnet AdS black hole. △ Less

Submitted 1 August, 2024; originally announced August 2024.

Comments: 22 pages,5 figures

BackdoorBench: A Comprehensive Benchmark and Analysis of Backdoor Learning

Authors: Baoyuan Wu, Hongrui Chen, Mingda Zhang, Zihao Zhu, Shaokui Wei, Danni Yuan, Mingli Zhu, Ruotong Wang, Li Liu, Chao Shen

Abstract: As an emerging approach to explore the vulnerability of deep neural networks (DNNs), backdoor learning has attracted increasing interest in recent years, and many seminal backdoor attack and defense algorithms are being developed successively or concurrently, in the status of a rapid arms race. However, mainly due to the diverse settings, and the difficulties of implementation and reproducibility… ▽ More As an emerging approach to explore the vulnerability of deep neural networks (DNNs), backdoor learning has attracted increasing interest in recent years, and many seminal backdoor attack and defense algorithms are being developed successively or concurrently, in the status of a rapid arms race. However, mainly due to the diverse settings, and the difficulties of implementation and reproducibility of existing works, there is a lack of a unified and standardized benchmark of backdoor learning, causing unfair comparisons or unreliable conclusions (e.g., misleading, biased or even false conclusions). Consequently, it is difficult to evaluate the current progress and design the future development roadmap of this literature. To alleviate this dilemma, we build a comprehensive benchmark of backdoor learning called BackdoorBench. Our benchmark makes three valuable contributions to the research community. 1) We provide an integrated implementation of state-of-the-art (SOTA) backdoor learning algorithms (currently including 20 attack and 32 defense algorithms), based on an extensible modular-based codebase. 2) We conduct comprehensive evaluations with 5 poisoning ratios, based on 4 models and 4 datasets, leading to 11,492 pairs of attack-against-defense evaluations in total. 3) Based on above evaluations, we present abundant analysis from 10 perspectives via 18 useful analysis tools, and provide several inspiring insights about backdoor learning. We hope that our efforts could build a solid foundation of backdoor learning to facilitate researchers to investigate existing algorithms, develop more innovative algorithms, and explore the intrinsic mechanism of backdoor learning. Finally, we have created a user-friendly website at http://backdoorbench.com, which collects all important information of BackdoorBench, including codebase, docs, leaderboard, and model Zoo. △ Less

Submitted 29 July, 2024; originally announced July 2024.

Comments: Substantial extensions based on our previous conference version "Backdoorbench: A comprehensive benchmark of backdoor learning" published at NeurIPS D&B Track 2022. 20 backdoor attack algorithms, 32 backdoor defense algorithms, 11000+ pairs of attack-against-defense evaluations, 10 analyses, 18 analysis tools

Thermodynamics and phase transition of Bardeen-AdS-class black holes

Authors: Shan-Ping Wu, Shao-Wen Wei

Abstract: In a generalized parameter space, regular black holes can be regarded as non-singular solutions under specific parameters in Einstein gravity theory coupled with non-linear electromagnetic fields. Following this concept, we investigate the thermodynamic states and phase transitions of Bardeen-AdS-class black holes, revealing that the system can be classified into two categories, Type I and Type II… ▽ More In a generalized parameter space, regular black holes can be regarded as non-singular solutions under specific parameters in Einstein gravity theory coupled with non-linear electromagnetic fields. Following this concept, we investigate the thermodynamic states and phase transitions of Bardeen-AdS-class black holes, revealing that the system can be classified into two categories, Type I and Type II, based on whether it adopts a pure Bardeen-AdS spacetime without event horizons or a Bardeen-AdS black hole as its phase state, each exhibiting distinct thermodynamic properties. If one includes the Bardeen-AdS black holes in the system (Type I), there will be three distinct black hole states and the phase transitions between them are analogous to the Reissner-Nordstrom-AdS black holes. On the other hand, if the pure Bardeen-AdS spacetime is included (Type II), an additional tiny black hole state emerges. A phase transition reminiscent of the Hawking-Page transition was found. Significantly, in this scenario, thermodynamical characteristic curves exhibit discontinuous behavior, which attributes to the multiple horizons. The presence of a Bardeen-AdS black hole within the phase structure of the Bardeen-AdS-class black hole profoundly modifies the thermodynamical properties of the system, highlighting the novel aspects of regular black holes. △ Less

Submitted 29 July, 2024; originally announced July 2024.

Comments: 18 pages, 11 figures

Incoherent diffractive production of jets in electron DIS off nuclei at high energy

Authors: Benjamin Rodriguez-Aguilar, D. N. Triantafyllopoulos, S. Y. Wei

Abstract: We study incoherent diffractive production of two and three jets in electron-nucleus deep inelastic scattering (DIS) at small $x_{\scriptscriptstyle \rm Bj}$ using the color dipole picture and the effective theory of the Color Glass Condensate (CGC). We consider color fluctuations in the CGC weight-function as the source of the nuclear break-up and the associated momentum transfer $\sqrt{|t|}$. We… ▽ More We study incoherent diffractive production of two and three jets in electron-nucleus deep inelastic scattering (DIS) at small $x_{\scriptscriptstyle \rm Bj}$ using the color dipole picture and the effective theory of the Color Glass Condensate (CGC). We consider color fluctuations in the CGC weight-function as the source of the nuclear break-up and the associated momentum transfer $\sqrt{|t|}$. We focus on the regime in which the two jets are almost back-to-back in transverse space and have transverse momenta $P_{\perp}$ much larger than both the momentum transfer and the saturation scale $Q_s$. The cross section for producing such a hard dijet is parametrically dominated by large size fluctuations in the projectile wave-function that scatter strongly and for which a third, semi-hard, jet appears in the final state. The 2 + 1 jets cross section can be written in a factorized form in terms of incoherent quark and gluon diffractive transverse momentum distributions (DTMDs) when the third jet is explicit, or incoherent diffractive parton distribution functions (DPDFs) when the third jet is integrated over. We find that the DPDFs and the corresponding cross section saturate logarithmically when $|t| \ll Q_s^2$, while they fall like $1/|t|^2$ in the regime $Q_s^2 \ll |t| \ll P_{\perp}^2$. We further show that there is no angular correlation between the hard jet momentum and the momentum transfer. For typical EIC kinematics the 2 jets and 2 + 1 jets cross sections are of the same order. △ Less

Submitted 24 July, 2024; originally announced July 2024.

Comments: 36 pages, 14 figures, 2 appendices

Theorizing neuro-induced relationships between cognitive diversity, motivation, grit and academic performance in multidisciplinary engineering education context

Authors: Duy Duong-Tran, Siqing Wei, Li Shen

Abstract: Nowadays, engineers need to tackle many unprecedented challenges that are often complex, and, most importantly, cannot be exhaustively compartmentalized into a single engineering discipline. In other words, most engineering problems need to be solved from a multidisciplinary approach. However, conventional engineering programs usually adopt pedagogical approaches specifically tailored to tradition… ▽ More Nowadays, engineers need to tackle many unprecedented challenges that are often complex, and, most importantly, cannot be exhaustively compartmentalized into a single engineering discipline. In other words, most engineering problems need to be solved from a multidisciplinary approach. However, conventional engineering programs usually adopt pedagogical approaches specifically tailored to traditional, niched engineering disciplines, which become increasingly deviated from the industry needs as those programs are typically designed and taught by instructors with highly specialized engineering training and credentials. To reduce the gap, more multidisciplinary engineering programs emerge by systematically stretching across all engineering fibers, and challenge the sub-optimal traditional pedagogy crowded in engineering classrooms. To further advance future-oriented pedagogy, in this work, we hypothesized neuro-induced linkages between how cognitively different learners are and how the linkages would affect learners in the knowledge acquisition process. We situate the neuro-induced linkages in the context of multidisciplinary engineering education and propose possible pedagogical approaches to actualize the implications of this conceptual framework. Our study, based on the innovative concept of brain fingerprint, would serve as a pioneer model to theorize key components of learner-centered multidisciplinary engineering pedagogy which centers on the key question: how do we motivate engineering students of different backgrounds from a neuro-inspired perspective? △ Less

Submitted 13 May, 2024; originally announced July 2024.

Comments: 10 pages, 2 figures

Harmonizing Visual Text Comprehension and Generation

Authors: Zhen Zhao, Jingqun Tang, Binghong Wu, Chunhui Lin, Shu Wei, Hao Liu, Xin Tan, Zhizhong Zhang, Can Huang, Yuan Xie

Abstract: In this work, we present TextHarmony, a unified and versatile multimodal generative model proficient in comprehending and generating visual text. Simultaneously generating images and texts typically results in performance degradation due to the inherent inconsistency between vision and language modalities. To overcome this challenge, existing approaches resort to modality-specific data for supervi… ▽ More In this work, we present TextHarmony, a unified and versatile multimodal generative model proficient in comprehending and generating visual text. Simultaneously generating images and texts typically results in performance degradation due to the inherent inconsistency between vision and language modalities. To overcome this challenge, existing approaches resort to modality-specific data for supervised fine-tuning, necessitating distinct model instances. We propose Slide-LoRA, which dynamically aggregates modality-specific and modality-agnostic LoRA experts, partially decoupling the multimodal generation space. Slide-LoRA harmonizes the generation of vision and language within a singular model instance, thereby facilitating a more unified generative process. Additionally, we develop a high-quality image caption dataset, DetailedTextCaps-100K, synthesized with a sophisticated closed-source MLLM to enhance visual text generation capabilities further. Comprehensive experiments across various benchmarks demonstrate the effectiveness of the proposed approach. Empowered by Slide-LoRA, TextHarmony achieves comparable performance to modality-specific fine-tuning results with only a 2% increase in parameters and shows an average improvement of 2.5% in visual text comprehension tasks and 4.0% in visual text generation tasks. Our work delineates the viability of an integrated approach to multimodal generation within the visual text domain, setting a foundation for subsequent inquiries. △ Less

Submitted 23 July, 2024; originally announced July 2024.

Large Language Model Enabled Semantic Communication Systems

Authors: Zhenyi Wang, Li Zou, Shengyun Wei, Feifan Liao, Jia Zhuo, Haibo Mi, Rongxuan Lai

Abstract: Large language models (LLMs) have recently demonstrated state-of-the-art performance across various natural language processing (NLP) tasks, achieving near-human levels in multiple language understanding challenges and aligning closely with the core principles of semantic communication. Inspired by LLMs' advancements in semantic processing, we propose an innovative LLM-enabled semantic communicati… ▽ More Large language models (LLMs) have recently demonstrated state-of-the-art performance across various natural language processing (NLP) tasks, achieving near-human levels in multiple language understanding challenges and aligning closely with the core principles of semantic communication. Inspired by LLMs' advancements in semantic processing, we propose an innovative LLM-enabled semantic communication system framework, named LLM-SC, that applies LLMs directly to the physical layer coding and decoding for the first time. By analyzing the relationship between the training process of LLMs and the optimization objectives of semantic communication, we propose training a semantic encoder through LLMs' tokenizer training and establishing a semantic knowledge base via the LLMs' unsupervised pre-training process. This knowledge base aids in constructing the optimal decoder by providing the prior probability of the transmitted language sequence. Based on this foundation, we derive the optimal decoding criterion for the receiver and introduce the beam search algorithm to further reduce the complexity. Furthermore, we assert that existing LLMs can be employed directly for LLM-SC without additional re-training or fine-tuning. Simulation results demonstrate that LLM-SC outperforms classical DeepSC at signal-to-noise ratios (SNR) exceeding 3 dB, enabling error-free transmission of semantic information under high SNR, which is unattainable by DeepSC. In addition to semantic-level performance, LLM-SC demonstrates compatibility with technical-level performance, achieving approximately 8 dB coding gain for a bit error ratio (BER) of $10^{-3}$ without any channel coding while maintaining the same joint source-channel coding rate as traditional communication systems. △ Less

Submitted 19 July, 2024; originally announced July 2024.

Comments: 13 pages, 5 figures

Universal Facial Encoding of Codec Avatars from VR Headsets

Authors: Shaojie Bai, Te-Li Wang, Chenghui Li, Akshay Venkatesh, Tomas Simon, Chen Cao, Gabriel Schwartz, Ryan Wrench, Jason Saragih, Yaser Sheikh, Shih-En Wei

Abstract: Faithful real-time facial animation is essential for avatar-mediated telepresence in Virtual Reality (VR). To emulate authentic communication, avatar animation needs to be efficient and accurate: able to capture both extreme and subtle expressions within a few milliseconds to sustain the rhythm of natural conversations. The oblique and incomplete views of the face, variability in the donning of he… ▽ More Faithful real-time facial animation is essential for avatar-mediated telepresence in Virtual Reality (VR). To emulate authentic communication, avatar animation needs to be efficient and accurate: able to capture both extreme and subtle expressions within a few milliseconds to sustain the rhythm of natural conversations. The oblique and incomplete views of the face, variability in the donning of headsets, and illumination variation due to the environment are some of the unique challenges in generalization to unseen faces. In this paper, we present a method that can animate a photorealistic avatar in realtime from head-mounted cameras (HMCs) on a consumer VR headset. We present a self-supervised learning approach, based on a cross-view reconstruction objective, that enables generalization to unseen users. We present a lightweight expression calibration mechanism that increases accuracy with minimal additional cost to run-time efficiency. We present an improved parameterization for precise ground-truth generation that provides robustness to environmental variation. The resulting system produces accurate facial animation for unseen users wearing VR headsets in realtime. We compare our approach to prior face-encoding methods demonstrating significant improvements in both quantitative metrics and qualitative results. △ Less

Submitted 17 July, 2024; originally announced July 2024.

Comments: SIGGRAPH 2024 (ACM Transactions on Graphics (TOG))

Journal ref: ACM Trans. Graph. 43, 4, Article 93 (July 2024), 22 pages.

Graph Dimension Attention Networks for Enterprise Credit Assessment

Authors: Shaopeng Wei, Beni Egressy, Xingyan Chen, Yu Zhao, Fuzhen Zhuang, Roger Wattenhofer, Gang Kou

Abstract: Enterprise credit assessment is critical for evaluating financial risk, and Graph Neural Networks (GNNs), with their advanced capability to model inter-entity relationships, are a natural tool to get a deeper understanding of these financial networks. However, existing GNN-based methodologies predominantly emphasize entity-level attention mechanisms for contagion risk aggregation, often overlookin… ▽ More Enterprise credit assessment is critical for evaluating financial risk, and Graph Neural Networks (GNNs), with their advanced capability to model inter-entity relationships, are a natural tool to get a deeper understanding of these financial networks. However, existing GNN-based methodologies predominantly emphasize entity-level attention mechanisms for contagion risk aggregation, often overlooking the heterogeneous importance of different feature dimensions, thus falling short in adequately modeling credit risk levels. To address this issue, we propose a novel architecture named Graph Dimension Attention Network (GDAN), which incorporates a dimension-level attention mechanism to capture fine-grained risk-related characteristics. Furthermore, we explore the interpretability of the GNN-based method in financial scenarios and propose a simple but effective data-centric explainer for GDAN, called GDAN-DistShift. DistShift provides edge-level interpretability by quantifying distribution shifts during the message-passing process. Moreover, we collected a real-world, multi-source Enterprise Credit Assessment Dataset (ECAD) and have made it accessible to the research community since high-quality datasets are lacking in this field. Extensive experiments conducted on ECAD demonstrate the effectiveness of our methods. In addition, we ran GDAN on the well-known datasets SMEsD and DBLP, also with excellent results. △ Less

Submitted 16 July, 2024; originally announced July 2024.

Charged spherically symmetric and slowly rotating charged black hole solutions in bumblebee gravity

Authors: Jia-Zhou Liu, Wen-Di Guo, Shao-Wen Wei, Yu-Xiao Liu

Abstract: In this paper, we present charged spherically symmetric black hole solutions and slowly rotating charged solutions in bumblebee gravity with and without a cosmological constant. The static spherically symmetric solutions describe the Reissner-Nordström-like black hole and ReissnerNordström-(anti) de Sitter-like black hole, while the stationary and axially symmetric soltuions describe Kerr-Newman-l… ▽ More In this paper, we present charged spherically symmetric black hole solutions and slowly rotating charged solutions in bumblebee gravity with and without a cosmological constant. The static spherically symmetric solutions describe the Reissner-Nordström-like black hole and ReissnerNordström-(anti) de Sitter-like black hole, while the stationary and axially symmetric soltuions describe Kerr-Newman-like black hole and Kerr-Newman-(anti) de Sitter-like black hole. We utilize the Hamilton-Jacobi formalism to study the shadows of the black holes. Additionally, we investigate the effect of the electric charge and Lorentz-violating parameters on the radius of the shadow reference circle and the distortion parameter. We find that the radius of the reference circle decreases with the Lorentz-violating parameter and charge parameter, while the distortion parameter increases with the Lorentz-violating parameter and the charge parameter. △ Less

Submitted 11 July, 2024; originally announced July 2024.

Comments: 25 pages, 6 figures

pythOS: A Python library for solving IVPs by operator splitting

Authors: Victoria Guenter, Siqi Wei, Raymond J. Spiteri

Abstract: Operator-splitting methods are widespread in the numerical solution of differential equations, especially the initial-value problems in ordinary differential equations that arise from a method-of-lines discretization of partial differential equations. Such problems can often be solved more effectively by treating the various terms individually with specialized methods rather than simultaneously in… ▽ More Operator-splitting methods are widespread in the numerical solution of differential equations, especially the initial-value problems in ordinary differential equations that arise from a method-of-lines discretization of partial differential equations. Such problems can often be solved more effectively by treating the various terms individually with specialized methods rather than simultaneously in a monolithic fashion. This paper describes \pythOS, a Python software library for the systematic solution of differential equations by operator-splitting methods. The functionality of \pythOS\ focuses on fractional-step methods, including those with real and complex coefficients, but it also implements additive Runge--Kutta methods, generalized additive Runge--Kutta methods, and multi-rate, and multi-rate infinitesimal methods. Experimentation with the solution of practical problems is facilitated through an interface to the \Firedrake\ library for the finite element spatial discretization of partial differential equations and further enhanced by the convenient implementation of exponential time-integration methods and fully implicit Runge--Kutta methods available from the \Irksome\ software library. The functionality of \pythOS\ as well as some less generally appreciated aspects of operator-splitting methods are demonstrated by means of examples. △ Less

Submitted 7 July, 2024; originally announced July 2024.

Robust Multimodal Learning via Representation Decoupling

Authors: Shicai Wei, Yang Luo, Yuji Wang, Chunbo Luo

Abstract: Multimodal learning robust to missing modality has attracted increasing attention due to its practicality. Existing methods tend to address it by learning a common subspace representation for different modality combinations. However, we reveal that they are sub-optimal due to their implicit constraint on intra-class representation. Specifically, the sample with different modalities within the same… ▽ More Multimodal learning robust to missing modality has attracted increasing attention due to its practicality. Existing methods tend to address it by learning a common subspace representation for different modality combinations. However, we reveal that they are sub-optimal due to their implicit constraint on intra-class representation. Specifically, the sample with different modalities within the same class will be forced to learn representations in the same direction. This hinders the model from capturing modality-specific information, resulting in insufficient learning. To this end, we propose a novel Decoupled Multimodal Representation Network (DMRNet) to assist robust multimodal learning. Specifically, DMRNet models the input from different modality combinations as a probabilistic distribution instead of a fixed point in the latent space, and samples embeddings from the distribution for the prediction module to calculate the task loss. As a result, the direction constraint from the loss minimization is blocked by the sampled representation. This relaxes the constraint on the inference representation and enables the model to capture the specific information for different modality combinations. Furthermore, we introduce a hard combination regularizer to prevent DMRNet from unbalanced training by guiding it to pay more attention to hard modality combinations. Finally, extensive experiments on multimodal classification and segmentation tasks demonstrate that the proposed DMRNet outperforms the state-of-the-art significantly. △ Less

Submitted 5 July, 2024; originally announced July 2024.

Comments: ECCV2024 17 pages

Segregation at prior austenite grain boundaries: the competition between boron and hydrogen

Authors: Guillaume Hachet, Ali Tehranchi, Hao Shi, Manoj Prabhakar, Shaolou Wei, Katja Angenendt, Stefan Zaefferer, Baptiste Gault, Binhan Sun, Dirk Ponge, Dierk Raabe

Abstract: The interaction between boron and hydrogen at grain boundaries has been investigated experimentally and numerically in boron-doped and boron-free martensitic steels using thermal desorption spectrometry (TDS) and ab initio calculations. The calculations show that boron, which mostly segregates into prior austenite grain boundaries (PAGBs), repels hydrogen. This behavior has also been observed usin… ▽ More The interaction between boron and hydrogen at grain boundaries has been investigated experimentally and numerically in boron-doped and boron-free martensitic steels using thermal desorption spectrometry (TDS) and ab initio calculations. The calculations show that boron, which mostly segregates into prior austenite grain boundaries (PAGBs), repels hydrogen. This behavior has also been observed using TDS measurements, with the disappearance of one peak when boron is incorporated into the microstructure. Additionally, the microstructure of both boron-doped and boron-free steels has been studied through electron backscattered diffraction, electron channeling contrast imaging, synchrotron X-ray measurements, and atom probe tomography. While both steels have a similar grain size, grain boundary distribution, and dislocation densities, pronounced boron segregation into PAGBs is observed for boron-doped steels. Therefore, the equilibrium hydrogen concentration in different trapping sites has been evaluated using the Langmuir-McLean approximation. This thermodynamic model shows that the distribution of hydrogen is identical for all traps when the total hydrogen concentration is low for boron-free steel. However, when it increases, traps of the lowest segregation energies (mostly PAGBs) are firstly saturated, which promotes failure initiation at this defect type. This finding partially explains why PAGBs are the weakest microstructure feature when martensitic steels are exposed to hydrogen-containing environments. △ Less

Submitted 4 July, 2024; originally announced July 2024.

Comments: Pre-printed version, also submitted to hal science: https://hal.science/hal-04631250

Adaptive sampling strategy for tolerance analysis of freeform optical surfaces based on critical ray aiming

Authors: Rundong Fan, Shili Wei, Zhuang Qian, Huiru Ji, Hao Tan, Yan Mo, Donglin Ma

Abstract: The tolerance analysis of freeform surfaces plays a crucial role in the development of advanced imaging systems. However, the intricate relationship between surface error and imaging quality poses significant challenges, necessitating dense sampling of featured rays during the computation process to ensure an accurate tolerance for different fields of view (FOVs). Here, we propose an adaptive samp… ▽ More The tolerance analysis of freeform surfaces plays a crucial role in the development of advanced imaging systems. However, the intricate relationship between surface error and imaging quality poses significant challenges, necessitating dense sampling of featured rays during the computation process to ensure an accurate tolerance for different fields of view (FOVs). Here, we propose an adaptive sampling strategy called "Critical Ray Aiming" for surface tolerance analysis. By identifying the most sensitive ray to wave aberration at each surface point, our methodology facilitates flexible sampling of the FOVs and entrance pupil (EP), achieving computational efficiency without compromising accuracy in determining tolerable surface error. We demonstrate the effectiveness of our method through tolerance analysis of two different freeform imaging systems. △ Less

Submitted 4 July, 2024; originally announced July 2024.

A pair of Second-order complex-valued, N-split operator-splitting methods

Authors: Raymond J. Spiteri, Siqi Wei

Abstract: The use of operator-splitting methods to solve differential equations is widespread, but the methods are generally only defined for a given number of operators, most commonly two. Most operator-splitting methods are not generalizable to problems with $N$ operators for arbitrary $N$. In fact, there are only two known methods that can be applied to general $N$-split problems: the first-order Lie--Tr… ▽ More The use of operator-splitting methods to solve differential equations is widespread, but the methods are generally only defined for a given number of operators, most commonly two. Most operator-splitting methods are not generalizable to problems with $N$ operators for arbitrary $N$. In fact, there are only two known methods that can be applied to general $N$-split problems: the first-order Lie--Trotter (or Godunov) method and the second-order Strang (or Strang--Marchuk) method. In this paper, we derive two second-order operator-splitting methods that also generalize to $N$-split problems. These methods are complex valued but have positive real parts, giving them favorable stability properties, and require few sub-integrations per stage, making them computationally inexpensive. They can also be used as base methods from which to construct higher-order $N$-split operator-splitting methods with positive real parts. We verify the orders of accuracy of these new $N$-split methods and demonstrate their favorable efficiency properties against well-known real-valued operator-splitting methods on both real-valued and complex-valued differential equations. △ Less

Submitted 2 July, 2024; originally announced July 2024.

Additively manufacturable high-strength aluminum alloys with thermally stable microstructures enabled by hybrid machine learning-based design

Authors: S. Mohadeseh Taheri-Mousavi, Michael Xu, Florian Hengsbach, Clay Houser, Zhaoxuan Ge, Benjamin Glaser, Shaolou Wei, Mikro Schaper, James M. LeBeau, Greg B. Olson, A. John Hart

Abstract: Additively manufactured (AM) structural components with complex geometries and tailored properties at voxel-size resolution will lead to significant leap in performance in various critical engineering applications. However, at each voxel, we first need to be able to design the alloy efficiently and reliably. We demonstrate a hybrid approach combining calculation of phase diagram (CALPHAD)-based in… ▽ More Additively manufactured (AM) structural components with complex geometries and tailored properties at voxel-size resolution will lead to significant leap in performance in various critical engineering applications. However, at each voxel, we first need to be able to design the alloy efficiently and reliably. We demonstrate a hybrid approach combining calculation of phase diagram (CALPHAD)-based integrated computational materials engineering (ICME) with machine learning and inverse design techniques and performed a full alloy design cycle of a novel Al alloy (Al-Er-Zr-Y-Yb-Ni) for AM from virtual predictions to experimental validation. We designed this alloy to exhibit high tensile strength at room temperature through nanoscale L1$_2$-phase precipitation which stabilizes the microstructure to maintain strength after high-temperature aging. We initially exploit a fine distribution of metastable eutectic ternary phases through rapid solidification, which serve as the source for the reactive elements enabling nanoscale precipitation of a high phase fraction of the thermally stable L1$_2$ strengthening phases. The strength of the 3D-printed samples manufactured via laser powder bed fusion (LPBF) from the designed composition is comparable to that of wrought Al 7075, and after high-temperature (400$^\circ$C) aging is 50% stronger than the best benchmark printable Al alloy1. The stable strengthening strategy is applicable to a wide range of alloys and rapid solidification processes, and our hybrid ML/CALPHAD numerical framework can be used for the efficient and robust design of alloy microstructures and properties, expanding the capabilities of additive as well as traditional manufacturing. △ Less

Submitted 25 June, 2024; originally announced June 2024.

MindSpore Quantum: A User-Friendly, High-Performance, and AI-Compatible Quantum Computing Framework

Authors: Xusheng Xu, Jiangyu Cui, Zidong Cui, Runhong He, Qingyu Li, Xiaowei Li, Yanling Lin, Jiale Liu, Wuxin Liu, Jiale Lu, Maolin Luo, Chufan Lyu, Shijie Pan, Mosharev Pavel, Runqiu Shu, Jialiang Tang, Ruoqian Xu, Shu Xu, Kang Yang, Fan Yu, Qingguo Zeng, Haiying Zhao, Qiang Zheng, Junyuan Zhou, Xu Zhou , et al. (14 additional authors not shown)

Abstract: We introduce MindSpore Quantum, a pioneering hybrid quantum-classical framework with a primary focus on the design and implementation of noisy intermediate-scale quantum (NISQ) algorithms. Leveraging the robust support of MindSpore, an advanced open-source deep learning training/inference framework, MindSpore Quantum exhibits exceptional efficiency in the design and training of variational quantum… ▽ More We introduce MindSpore Quantum, a pioneering hybrid quantum-classical framework with a primary focus on the design and implementation of noisy intermediate-scale quantum (NISQ) algorithms. Leveraging the robust support of MindSpore, an advanced open-source deep learning training/inference framework, MindSpore Quantum exhibits exceptional efficiency in the design and training of variational quantum algorithms on both CPU and GPU platforms, delivering remarkable performance. Furthermore, this framework places a strong emphasis on enhancing the operational efficiency of quantum algorithms when executed on real quantum hardware. This encompasses the development of algorithms for quantum circuit compilation and qubit mapping, crucial components for achieving optimal performance on quantum processors. In addition to the core framework, we introduce QuPack, a meticulously crafted quantum computing acceleration engine. QuPack significantly accelerates the simulation speed of MindSpore Quantum, particularly in variational quantum eigensolver (VQE), quantum approximate optimization algorithm (QAOA), and tensor network simulations, providing astonishing speed. This combination of cutting-edge technologies empowers researchers and practitioners to explore the frontiers of quantum computing with unprecedented efficiency and performance. △ Less

Submitted 10 July, 2024; v1 submitted 24 June, 2024; originally announced June 2024.

DN-CL: Deep Symbolic Regression against Noise via Contrastive Learning

Authors: Jingyi Liu, Yanjie Li, Lina Yu, Min Wu, Weijun Li, Wenqiang Li, Meilan Hao, Yusong Deng, Shu Wei

Abstract: Noise ubiquitously exists in signals due to numerous factors including physical, electronic, and environmental effects. Traditional methods of symbolic regression, such as genetic programming or deep learning models, aim to find the most fitting expressions for these signals. However, these methods often overlook the noise present in real-world data, leading to reduced fitting accuracy. To tackle… ▽ More Noise ubiquitously exists in signals due to numerous factors including physical, electronic, and environmental effects. Traditional methods of symbolic regression, such as genetic programming or deep learning models, aim to find the most fitting expressions for these signals. However, these methods often overlook the noise present in real-world data, leading to reduced fitting accuracy. To tackle this issue, we propose \textit{\textbf{D}eep Symbolic Regression against \textbf{N}oise via \textbf{C}ontrastive \textbf{L}earning (DN-CL)}. DN-CL employs two parameter-sharing encoders to embed data points from various data transformations into feature shields against noise. This model treats noisy data and clean data as different views of the ground-truth mathematical expressions. Distances between these features are minimized, utilizing contrastive learning to distinguish between 'positive' noise-corrected pairs and 'negative' contrasting pairs. Our experiments indicate that DN-CL demonstrates superior performance in handling both noisy and clean data, presenting a promising method of symbolic regression. △ Less

Submitted 20 June, 2024; originally announced June 2024.

Novel topological phenomena of timelike circular orbits for charged test particles

Authors: Xu Ye, Shao-Wen Wei

Abstract: The topological approach has recently been successfully employed to investigate timelike circular orbits for massive neutral test particles. The observed vanishing topological number implies that these timelike circular orbits occur in pairs. However, the behavior of charged test particles in this regard remains unexplored. To address this issue, our study focuses on examining the influence of par… ▽ More The topological approach has recently been successfully employed to investigate timelike circular orbits for massive neutral test particles. The observed vanishing topological number implies that these timelike circular orbits occur in pairs. However, the behavior of charged test particles in this regard remains unexplored. To address this issue, our study focuses on examining the influence of particle charge on the topology of timelike circular orbits within a spherically symmetrical black hole spacetime holding a nonvanishing radial electric field. We consider four distinct cases based on the charges of the particle and the black hole: unlike strong charge, unlike weak charge, like weak charge, and like strong charge. For each case, we calculate the corresponding topological number. Our results reveal that when the charge is large enough, the topological number takes a value of -1 instead of 0, which differs from the neutral particle scenario. Consequently, in cases of small charges, the timelike circular orbits appear in pairs, whereas in cases of larger charges, an additional unstable timelike circular orbit emerges. These findings shed light on the influence of the particle charge on the topological properties and number of timelike circular orbits. △ Less

Submitted 19 June, 2024; originally announced June 2024.

Comments: 30 pages, 29 figures

DiffPop: Plausibility-Guided Object Placement Diffusion for Image Composition

Authors: Jiacheng Liu, Hang Zhou, Shida Wei, Rui Ma

Abstract: In this paper, we address the problem of plausible object placement for the challenging task of realistic image composition. We propose DiffPop, the first framework that utilizes plausibility-guided denoising diffusion probabilistic model to learn the scale and spatial relations among multiple objects and the corresponding scene image. First, we train an unguided diffusion model to directly learn… ▽ More In this paper, we address the problem of plausible object placement for the challenging task of realistic image composition. We propose DiffPop, the first framework that utilizes plausibility-guided denoising diffusion probabilistic model to learn the scale and spatial relations among multiple objects and the corresponding scene image. First, we train an unguided diffusion model to directly learn the object placement parameters in a self-supervised manner. Then, we develop a human-in-the-loop pipeline which exploits human labeling on the diffusion-generated composite images to provide the weak supervision for training a structural plausibility classifier. The classifier is further used to guide the diffusion sampling process towards generating the plausible object placement. Experimental results verify the superiority of our method for producing plausible and diverse composite images on the new Cityscapes-OP dataset and the public OPA dataset, as well as demonstrate its potential in applications such as data augmentation and multi-object placement tasks. Our dataset and code will be released. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Lighting up the Photon Wigner Distribution via Dilepton Productions

Authors: Yu Shi, Lin Chen, Shu-Yi Wei, Bo-Wen Xiao

Abstract: We present a systematic investigation of lepton pair production through photon-photon fusion processes in heavy-ion collisions. It is demonstrated that the dilepton production at a given impact parameter ($b_\perp$) with a fixed transverse momentum imbalance ($q_\perp$) can be factorized into a unified formula in terms of the Wigner photon distribution of heavy nuclei. We show that this framework… ▽ More We present a systematic investigation of lepton pair production through photon-photon fusion processes in heavy-ion collisions. It is demonstrated that the dilepton production at a given impact parameter ($b_\perp$) with a fixed transverse momentum imbalance ($q_\perp$) can be factorized into a unified formula in terms of the Wigner photon distribution of heavy nuclei. We show that this framework provides a comprehensive description of all the relevant data from RHIC to the LHC, with a strong evidence that the quasi-real photon can be radiated not only from the nucleus as a whole, standing for the coherent contribution, but also from the sub-structures inside the nucleus, representing the incoherent contribution. Further predictions are made for the anisotropies in the correlations between $q_\perp$, $b_\perp$, and the dilepton transverse momentum ($P_\perp$). This will help us to constrain the photon Wigner distribution which plays a crucial role to study the gluonic matter of nucleus at small-$x$ through the diffractive photoproduction processes in heavy ion collision. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Comments: 7 pages (main text) plus 18 pages (supplemental material), 19 figures

Transformers Provably Learn Sparse Token Selection While Fully-Connected Nets Cannot

Authors: Zixuan Wang, Stanley Wei, Daniel Hsu, Jason D. Lee

Abstract: The transformer architecture has prevailed in various deep learning settings due to its exceptional capabilities to select and compose structural information. Motivated by these capabilities, Sanford et al. proposed the sparse token selection task, in which transformers excel while fully-connected networks (FCNs) fail in the worst case. Building upon that, we strengthen the FCN lower bound to an a… ▽ More The transformer architecture has prevailed in various deep learning settings due to its exceptional capabilities to select and compose structural information. Motivated by these capabilities, Sanford et al. proposed the sparse token selection task, in which transformers excel while fully-connected networks (FCNs) fail in the worst case. Building upon that, we strengthen the FCN lower bound to an average-case setting and establish an algorithmic separation of transformers over FCNs. Specifically, a one-layer transformer trained with gradient descent provably learns the sparse token selection task and, surprisingly, exhibits strong out-of-distribution length generalization. We provide empirical simulations to justify our theoretical findings. △ Less

Submitted 10 June, 2024; originally announced June 2024.

MOSA: Music Motion with Semantic Annotation Dataset for Cross-Modal Music Processing

Authors: Yu-Fen Huang, Nikki Moran, Simon Coleman, Jon Kelly, Shun-Hwa Wei, Po-Yin Chen, Yun-Hsin Huang, Tsung-Ping Chen, Yu-Chia Kuo, Yu-Chi Wei, Chih-Hsuan Li, Da-Yu Huang, Hsuan-Kai Kao, Ting-Wei Lin, Li Su

Abstract: In cross-modal music processing, translation between visual, auditory, and semantic content opens up new possibilities as well as challenges. The construction of such a transformative scheme depends upon a benchmark corpus with a comprehensive data infrastructure. In particular, the assembly of a large-scale cross-modal dataset presents major challenges. In this paper, we present the MOSA (Music m… ▽ More In cross-modal music processing, translation between visual, auditory, and semantic content opens up new possibilities as well as challenges. The construction of such a transformative scheme depends upon a benchmark corpus with a comprehensive data infrastructure. In particular, the assembly of a large-scale cross-modal dataset presents major challenges. In this paper, we present the MOSA (Music mOtion with Semantic Annotation) dataset, which contains high quality 3-D motion capture data, aligned audio recordings, and note-by-note semantic annotations of pitch, beat, phrase, dynamic, articulation, and harmony for 742 professional music performances by 23 professional musicians, comprising more than 30 hours and 570 K notes of data. To our knowledge, this is the largest cross-modal music dataset with note-level annotations to date. To demonstrate the usage of the MOSA dataset, we present several innovative cross-modal music information retrieval (MIR) and musical content generation tasks, including the detection of beats, downbeats, phrase, and expressive contents from audio, video and motion data, and the generation of musicians' body motion from given music audio. The dataset and codes are available alongside this publication (https://github.com/yufenhuang/MOSA-Music-mOtion-and-Semantic-Annotation-dataset). △ Less

Submitted 10 June, 2024; originally announced June 2024.

Comments: IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024. 14 pages, 7 figures. Dataset is available on: https://github.com/yufenhuang/MOSA-Music-mOtion-and-Semantic-Annotation-dataset/tree/main and https://zenodo.org/records/11393449

MLLM-SR: Conversational Symbolic Regression base Multi-Modal Large Language Models

Authors: Yanjie Li, Weijun Li, Lina Yu, Min Wu, Jingyi Liu, Wenqiang Li, Shu Wei, Yusong Deng

Abstract: Formulas are the language of communication between humans and nature. It is an important research topic of artificial intelligence to find expressions from observed data to reflect the relationship between each variable in the data, which is called a symbolic regression problem. The existing symbolic regression methods directly generate expressions according to the given observation data, and we c… ▽ More Formulas are the language of communication between humans and nature. It is an important research topic of artificial intelligence to find expressions from observed data to reflect the relationship between each variable in the data, which is called a symbolic regression problem. The existing symbolic regression methods directly generate expressions according to the given observation data, and we cannot require the algorithm to generate expressions that meet specific requirements according to the known prior knowledge. For example, the expression needs to contain $\sin$ or be symmetric, and so on. Even if it can, it often requires very complex operations, which is very inconvenient. In this paper, based on multi-modal large language models, we propose MLLM-SR, a conversational symbolic regression method that can generate expressions that meet the requirements simply by describing the requirements with natural language instructions. By experimenting on the Nguyen dataset, we can demonstrate that MLLM-SR leads the state-of-the-art baselines in fitting performance. More notably, we experimentally demonstrate that MLLM-SR can well understand the prior knowledge we add to the natural language instructions. Moreover, the addition of prior knowledge can effectively guide MLLM-SR to generate correct expressions. △ Less

Submitted 8 June, 2024; originally announced June 2024.

Comments: 13 pages,

Unveiling Selection Biases: Exploring Order and Token Sensitivity in Large Language Models

Authors: Sheng-Lun Wei, Cheng-Kuang Wu, Hen-Hsen Huang, Hsin-Hsi Chen

Abstract: In this paper, we investigate the phenomena of "selection biases" in Large Language Models (LLMs), focusing on problems where models are tasked with choosing the optimal option from an ordered sequence. We delve into biases related to option order and token usage, which significantly impact LLMs' decision-making processes. We also quantify the impact of these biases through an extensive empirical… ▽ More In this paper, we investigate the phenomena of "selection biases" in Large Language Models (LLMs), focusing on problems where models are tasked with choosing the optimal option from an ordered sequence. We delve into biases related to option order and token usage, which significantly impact LLMs' decision-making processes. We also quantify the impact of these biases through an extensive empirical analysis across multiple models and tasks. Furthermore, we propose mitigation strategies to enhance model performance. Our key contributions are threefold: 1) Precisely quantifying the influence of option order and token on LLMs, 2) Developing strategies to mitigate the impact of token and order sensitivity to enhance robustness, and 3) Offering a detailed analysis of sensitivity across models and tasks, which informs the creation of more stable and reliable LLM applications for selection problems. △ Less

Submitted 5 June, 2024; originally announced June 2024.

Comments: Accepted as a long findings paper at ACL 2024

P2PFormer: A Primitive-to-polygon Method for Regular Building Contour Extraction from Remote Sensing Images

Authors: Tao Zhang, Shiqing Wei, Yikang Zhou, Muying Luo, Wenling You, Shunping Ji

Abstract: Extracting building contours from remote sensing imagery is a significant challenge due to buildings' complex and diverse shapes, occlusions, and noise. Existing methods often struggle with irregular contours, rounded corners, and redundancy points, necessitating extensive post-processing to produce regular polygonal building contours. To address these challenges, we introduce a novel, streamlined… ▽ More Extracting building contours from remote sensing imagery is a significant challenge due to buildings' complex and diverse shapes, occlusions, and noise. Existing methods often struggle with irregular contours, rounded corners, and redundancy points, necessitating extensive post-processing to produce regular polygonal building contours. To address these challenges, we introduce a novel, streamlined pipeline that generates regular building contours without post-processing. Our approach begins with the segmentation of generic geometric primitives (which can include vertices, lines, and corners), followed by the prediction of their sequence. This allows for the direct construction of regular building contours by sequentially connecting the segmented primitives. Building on this pipeline, we developed P2PFormer, which utilizes a transformer-based architecture to segment geometric primitives and predict their order. To enhance the segmentation of primitives, we introduce a unique representation called group queries. This representation comprises a set of queries and a singular query position, which improve the focus on multiple midpoints of primitives and their efficient linkage. Furthermore, we propose an innovative implicit update strategy for the query position embedding aimed at sharpening the focus of queries on the correct positions and, consequently, enhancing the quality of primitive segmentation. Our experiments demonstrate that P2PFormer achieves new state-of-the-art performance on the WHU, CrowdAI, and WHU-Mix datasets, surpassing the previous SOTA PolyWorld by a margin of 2.7 AP and 6.5 AP75 on the largest CrowdAI dataset. We intend to make the code and trained weights publicly available to promote their use and facilitate further research. △ Less

Submitted 5 June, 2024; originally announced June 2024.

TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergy

Authors: Weichao Zhao, Hao Feng, Qi Liu, Jingqun Tang, Shu Wei, Binghong Wu, Lei Liao, Yongjie Ye, Hao Liu, Houqiang Li, Can Huang

Abstract: Tables contain factual and quantitative data accompanied by various structures and contents that pose challenges for machine comprehension. Previous methods generally design task-specific architectures and objectives for individual tasks, resulting in modal isolation and intricate workflows. In this paper, we present a novel large vision-language model, TabPedia, equipped with a concept synergy me… ▽ More Tables contain factual and quantitative data accompanied by various structures and contents that pose challenges for machine comprehension. Previous methods generally design task-specific architectures and objectives for individual tasks, resulting in modal isolation and intricate workflows. In this paper, we present a novel large vision-language model, TabPedia, equipped with a concept synergy mechanism. In this mechanism, all the involved diverse visual table understanding (VTU) tasks and multi-source visual embeddings are abstracted as concepts. This unified framework allows TabPedia to seamlessly integrate VTU tasks, such as table detection, table structure recognition, table querying, and table question answering, by leveraging the capabilities of large language models (LLMs). Moreover, the concept synergy mechanism enables table perception-related and comprehension-related tasks to work in harmony, as they can effectively leverage the needed clues from the corresponding source perception embeddings. Furthermore, to better evaluate the VTU task in real-world scenarios, we establish a new and comprehensive table VQA benchmark, ComTQA, featuring approximately 9,000 QA pairs. Extensive quantitative and qualitative experiments on both table perception and comprehension tasks, conducted across various public benchmarks, validate the effectiveness of our TabPedia. The superior performance further confirms the feasibility of using LLMs for understanding visual tables when all concepts work in synergy. The benchmark ComTQA has been open-sourced at https://huggingface.co/datasets/ByteDance/ComTQA. The source code and model will be released later. △ Less

Submitted 3 June, 2024; originally announced June 2024.

Comments: 20 pages, 8 figures

Bounds on f-Divergences between Distributions within Generalized Quasi-$\varepsilon$-Neighborhood

Authors: Xinchun Yu, Shuangqing Wei, Shao-Lun Huang, Xiao-Ping Zhang

Abstract: A general reverse Pinsker's inequality is derived to give an upper bound on f-divergences in terms of total variational distance when two distributions are close measured under our proposed generalized local information geometry framework. In addition, relationships between two f-divergences equipped with functions that are third order differentiable are established in terms of the lower and upper… ▽ More A general reverse Pinsker's inequality is derived to give an upper bound on f-divergences in terms of total variational distance when two distributions are close measured under our proposed generalized local information geometry framework. In addition, relationships between two f-divergences equipped with functions that are third order differentiable are established in terms of the lower and upper bounds of their ratio, when the underlying distributions are within a generalized quasi-$\varepsilon$-neighborhood. △ Less

Submitted 2 June, 2024; originally announced June 2024.

Unveiling and Mitigating Backdoor Vulnerabilities based on Unlearning Weight Changes and Backdoor Activeness

Authors: Weilin Lin, Li Liu, Shaokui Wei, Jianze Li, Hui Xiong

Abstract: The security threat of backdoor attacks is a central concern for deep neural networks (DNNs). Recently, without poisoned data, unlearning models with clean data and then learning a pruning mask have contributed to backdoor defense. Additionally, vanilla fine-tuning with those clean data can help recover the lost clean accuracy. However, the behavior of clean unlearning is still under-explored, and… ▽ More The security threat of backdoor attacks is a central concern for deep neural networks (DNNs). Recently, without poisoned data, unlearning models with clean data and then learning a pruning mask have contributed to backdoor defense. Additionally, vanilla fine-tuning with those clean data can help recover the lost clean accuracy. However, the behavior of clean unlearning is still under-explored, and vanilla fine-tuning unintentionally induces back the backdoor effect. In this work, we first investigate model unlearning from the perspective of weight changes and gradient norms, and find two interesting observations in the backdoored model: 1) the weight changes between poison and clean unlearning are positively correlated, making it possible for us to identify the backdoored-related neurons without using poisoned data; 2) the neurons of the backdoored model are more active (i.e., larger changes in gradient norm) than those in the clean model, suggesting the need to suppress the gradient norm during fine-tuning. Then, we propose an effective two-stage defense method. In the first stage, an efficient Neuron Weight Change (NWC)-based Backdoor Reinitialization is proposed based on observation 1). In the second stage, based on observation 2), we design an Activeness-Aware Fine-Tuning to replace the vanilla fine-tuning. Extensive experiments, involving eight backdoor attacks on three benchmark datasets, demonstrate the superior performance of our proposed method compared to recent state-of-the-art backdoor defense approaches. △ Less

Submitted 30 May, 2024; originally announced May 2024.

Efficient Orchestrated AI Workflows Execution on Scale-out Spatial Architecture

Authors: Jinyi Deng, Xinru Tang, Zhiheng Yue, Guangyang Lu, Qize Yang, Jiahao Zhang, Jinxi Li, Chao Li, Shaojun Wei, Yang Hu, Shouyi Yin

Abstract: Given the increasing complexity of AI applications, traditional spatial architectures frequently fall short. Our analysis identifies a pattern of interconnected, multi-faceted tasks encompassing both AI and general computational processes. In response, we have conceptualized "Orchestrated AI Workflows," an approach that integrates various tasks with logic-driven decisions into dynamic, sophisticat… ▽ More Given the increasing complexity of AI applications, traditional spatial architectures frequently fall short. Our analysis identifies a pattern of interconnected, multi-faceted tasks encompassing both AI and general computational processes. In response, we have conceptualized "Orchestrated AI Workflows," an approach that integrates various tasks with logic-driven decisions into dynamic, sophisticated workflows. Specifically, we find that the intrinsic Dual Dynamicity of Orchestrated AI Workflows, namely dynamic execution times and frequencies of Task Blocks, can be effectively represented using the Orchestrated Workflow Graph. Furthermore, the intrinsic Dual Dynamicity poses challenges to existing spatial architecture, namely Indiscriminate Resource Allocation, Reactive Load Rebalancing, and Contagious PEA Idleness. To overcome these challenges, we present Octopus, a scale-out spatial architecture and a suite of advanced scheduling strategies optimized for executing Orchestrated AI Workflows, such as the Discriminate Dual-Scheduling Mechanism, Adaptive TBU Scheduling Strategy, and Proactive Cluster Scheduling Strategy. Our evaluations demonstrate that Octopus significantly outperforms traditional architectures in handling the dynamic demands of Orchestrated AI Workflows, and possesses robust scalability in large scale hardware such as wafer-scale chip. △ Less

Submitted 21 May, 2024; originally announced May 2024.

Mitigating Backdoor Attack by Injecting Proactive Defensive Backdoor

Authors: Shaokui Wei, Hongyuan Zha, Baoyuan Wu

Abstract: Data-poisoning backdoor attacks are serious security threats to machine learning models, where an adversary can manipulate the training dataset to inject backdoors into models. In this paper, we focus on in-training backdoor defense, aiming to train a clean model even when the dataset may be potentially poisoned. Unlike most existing methods that primarily detect and remove/unlearn suspicious samp… ▽ More Data-poisoning backdoor attacks are serious security threats to machine learning models, where an adversary can manipulate the training dataset to inject backdoors into models. In this paper, we focus on in-training backdoor defense, aiming to train a clean model even when the dataset may be potentially poisoned. Unlike most existing methods that primarily detect and remove/unlearn suspicious samples to mitigate malicious backdoor attacks, we propose a novel defense approach called PDB (Proactive Defensive Backdoor). Specifically, PDB leverages the "home field" advantage of defenders by proactively injecting a defensive backdoor into the model during training. Taking advantage of controlling the training process, the defensive backdoor is designed to suppress the malicious backdoor effectively while remaining secret to attackers. In addition, we introduce a reversible mapping to determine the defensive target label. During inference, PDB embeds a defensive trigger in the inputs and reverses the model's prediction, suppressing malicious backdoor and ensuring the model's utility on the original task. Experimental results across various datasets and models demonstrate that our approach achieves state-of-the-art defense performance against a wide range of backdoor attacks. △ Less

Submitted 25 May, 2024; originally announced May 2024.

Comments: 13 pages, 5 figures and 5 tables

Closed-form Symbolic Solutions: A New Perspective on Solving Partial Differential Equations

Authors: Shu Wei, Yanjie Li, Lina Yu, Min Wu, Weijun Li, Meilan Hao, Wenqiang Li, Jingyi Liu, Yusong Deng

Abstract: Solving partial differential equations (PDEs) in Euclidean space with closed-form symbolic solutions has long been a dream for mathematicians. Inspired by deep learning, Physics-Informed Neural Networks (PINNs) have shown great promise in numerically solving PDEs. However, since PINNs essentially approximate solutions within the continuous function space, their numerical solutions fall short in bo… ▽ More Solving partial differential equations (PDEs) in Euclidean space with closed-form symbolic solutions has long been a dream for mathematicians. Inspired by deep learning, Physics-Informed Neural Networks (PINNs) have shown great promise in numerically solving PDEs. However, since PINNs essentially approximate solutions within the continuous function space, their numerical solutions fall short in both precision and interpretability compared to symbolic solutions. This paper proposes a novel framework: a closed-form \textbf{Sym}bolic framework for \textbf{PDE}s (SymPDE), exploring the use of deep reinforcement learning to directly obtain symbolic solutions for PDEs. SymPDE alleviates the challenges PINNs face in fitting high-frequency and steeply changing functions. To our knowledge, no prior work has implemented this approach. Experiments on solving the Poisson's equation and heat equation in time-independent and spatiotemporal dynamical systems respectively demonstrate that SymPDE can provide accurate closed-form symbolic solutions for various types of PDEs. △ Less

Submitted 23 May, 2024; originally announced May 2024.

Quantum Resonant Dimensionality Reduction and Its Application in Quantum Machine Learning

Authors: Fan Yang, Furong Wang, Xusheng Xu, Pao Gao, Tao Xin, ShiJie Wei, Guilu Long

Abstract: Quantum computing is a promising candidate for accelerating machine learning tasks. Limited by the control accuracy of current quantum hardware, reducing the consumption of quantum resources is the key to achieving quantum advantage. Here, we propose a quantum resonant dimension reduction (QRDR) algorithm based on the quantum resonant transition to reduce the dimension of input data and accelerate… ▽ More Quantum computing is a promising candidate for accelerating machine learning tasks. Limited by the control accuracy of current quantum hardware, reducing the consumption of quantum resources is the key to achieving quantum advantage. Here, we propose a quantum resonant dimension reduction (QRDR) algorithm based on the quantum resonant transition to reduce the dimension of input data and accelerate the quantum machine learning algorithms. After QRDR, the dimension of input data $N$ can be reduced into desired scale $R$, and the effective information of the original data will be preserved correspondingly, which will reduce the computational complexity of subsequent quantum machine learning algorithms or quantum storage. QRDR operates with polylogarithmic time complexity and reduces the error dependency from the order of $1/ε^3$ to the order of $1/ε$, compared to existing algorithms. We demonstrate the performance of our algorithm combining with two types of quantum classifiers, quantum support vector machines and quantum convolutional neural networks, for classifying underwater detection targets and quantum many-body phase respectively. The simulation results indicate that reduced data improved the processing efficiency and accuracy following the application of QRDR. As quantum machine learning continues to advance, our algorithm has the potential to be utilized in a variety of computing fields. △ Less

Submitted 21 May, 2024; originally announced May 2024.

MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answering

Authors: Jingqun Tang, Qi Liu, Yongjie Ye, Jinghui Lu, Shu Wei, Chunhui Lin, Wanqing Li, Mohamad Fitri Faiz Bin Mahmood, Hao Feng, Zhen Zhao, Yanjie Wang, Yuliang Liu, Hao Liu, Xiang Bai, Can Huang

Abstract: Text-Centric Visual Question Answering (TEC-VQA) in its proper format not only facilitates human-machine interaction in text-centric visual environments but also serves as a de facto gold proxy to evaluate AI models in the domain of text-centric scene understanding. Nonetheless, most existing TEC-VQA benchmarks have focused on high-resource languages like English and Chinese. Despite pioneering wo… ▽ More Text-Centric Visual Question Answering (TEC-VQA) in its proper format not only facilitates human-machine interaction in text-centric visual environments but also serves as a de facto gold proxy to evaluate AI models in the domain of text-centric scene understanding. Nonetheless, most existing TEC-VQA benchmarks have focused on high-resource languages like English and Chinese. Despite pioneering works to expand multilingual QA pairs in non-text-centric VQA datasets through translation engines, the translation-based protocol encounters a substantial "visual-textual misalignment" problem when applied to TEC-VQA. Specifically, it prioritizes the text in question-answer pairs while disregarding the visual text present in images. Moreover, it fails to address complexities related to nuanced meaning, contextual distortion, language bias, and question-type diversity. In this work, we tackle multilingual TEC-VQA by introducing MTVQA, the first benchmark featuring high-quality human expert annotations across 9 diverse languages, consisting of 6,778 question-answer pairs across 2,116 images. Further, by comprehensively evaluating numerous state-of-the-art Multimodal Large Language Models (MLLMs), including GPT-4o, GPT-4V, Claude3, and Gemini, on the MTVQA dataset, it is evident that there is still a large room for performance improvement, underscoring the value of MTVQA. Additionally, we supply multilingual training data within the MTVQA dataset, demonstrating that straightforward fine-tuning with this data can substantially enhance multilingual TEC-VQA performance. We aspire that MTVQA will offer the research community fresh insights and stimulate further exploration in multilingual visual text comprehension. The project homepage is available at https://bytedance.github.io/MTVQA/. △ Less

Submitted 11 June, 2024; v1 submitted 20 May, 2024; originally announced May 2024.

A Versatile Framework for Analyzing Galaxy Image Data by Implanting Human-in-the-loop on a Large Vision Model

Authors: Mingxiang Fu, Yu Song, Jiameng Lv, Liang Cao, Peng Jia, Nan Li, Xiangru Li, Jifeng Liu, A-Li Luo, Bo Qiu, Shiyin Shen, Liangping Tu, Lili Wang, Shoulin Wei, Haifeng Yang, Zhenping Yi, Zhiqiang Zou

Abstract: The exponential growth of astronomical datasets provides an unprecedented opportunity for humans to gain insight into the Universe. However, effectively analyzing this vast amount of data poses a significant challenge. Astronomers are turning to deep learning techniques to address this, but the methods are limited by their specific training sets, leading to considerable duplicate workloads too. He… ▽ More The exponential growth of astronomical datasets provides an unprecedented opportunity for humans to gain insight into the Universe. However, effectively analyzing this vast amount of data poses a significant challenge. Astronomers are turning to deep learning techniques to address this, but the methods are limited by their specific training sets, leading to considerable duplicate workloads too. Hence, as an example to present how to overcome the issue, we built a framework for general analysis of galaxy images, based on a large vision model (LVM) plus downstream tasks (DST), including galaxy morphological classification, image restoration, object detection, parameter extraction, and more. Considering the low signal-to-noise ratio of galaxy images and the imbalanced distribution of galaxy categories, we have incorporated a Human-in-the-loop (HITL) module into our large vision model, which leverages human knowledge to enhance the reliability and interpretability of processing galaxy images interactively. The proposed framework exhibits notable few-shot learning capabilities and versatile adaptability to all the abovementioned tasks on galaxy images in the DESI legacy imaging surveys. Expressly, for object detection, trained by 1000 data points, our DST upon the LVM achieves an accuracy of 96.7%, while ResNet50 plus Mask R-CNN gives an accuracy of 93.1%; for morphology classification, to obtain AUC ~0.9, LVM plus DST and HITL only requests 1/50 training sets compared to ResNet18. Expectedly, multimodal data can be integrated similarly, which opens up possibilities for conducting joint analyses with datasets spanning diverse domains in the era of multi-message astronomy. △ Less

Submitted 17 May, 2024; originally announced May 2024.

Comments: 26 pages, 10 figures, to be published on Chinese Physics C

Thermodynamical topology with multiple defect curves for dyonic AdS black holes

Authors: Zi-Qing Chen, Shao-Wen Wei

Abstract: Dyonic black holes with quasitopological electromagnetism exhibit an intriguing phase diagram with two separated first-order coexistence curves. In this paper, we aim to uncover its influence on the black hole thermodynamical topology. At first, we investigate the phase transition and phase diagram of the dyonic black holes. Comparing with previous study that there is no black hole phase transitio… ▽ More Dyonic black holes with quasitopological electromagnetism exhibit an intriguing phase diagram with two separated first-order coexistence curves. In this paper, we aim to uncover its influence on the black hole thermodynamical topology. At first, we investigate the phase transition and phase diagram of the dyonic black holes. Comparing with previous study that there is no black hole phase transition region for a middle pressure, we find this region can narrow or disappear by fine tuning the coupling parameter. Instead, two first-order phase transitions can be observed. Importantly, we uncover that such novel phase diagram shall lead to a multiple defect curve phenomenon in black hole topology where each dyonic black hole is treated as one defect in the thermodynamical parameter space. By examining the topology, it is shown that there could be one, three, or five black hole states for given pressure and temperature. For each case, the topological number is calculated. Our results show that the topological number always takes value of +1, keeping unchanged even when the multiple defect curves appear. Therefore, our study provides an important ingredient on understanding the black hole thermodynamical topology. △ Less

Submitted 13 May, 2024; originally announced May 2024.

Comments: 15 pages and 12 figures

Array SAR 3D Sparse Imaging Based on Regularization by Denoising Under Few Observed Data

Authors: Yangyang Wang, Xu Zhan, Jing Gao, Jinjie Yao, Shunjun Wei, JianSheng Bai

Abstract: Array synthetic aperture radar (SAR) three-dimensional (3D) imaging can obtain 3D information of the target region, which is widely used in environmental monitoring and scattering information measurement. In recent years, with the development of compressed sensing (CS) theory, sparse signal processing is used in array SAR 3D imaging. Compared with matched filter (MF), sparse SAR imaging can effect… ▽ More Array synthetic aperture radar (SAR) three-dimensional (3D) imaging can obtain 3D information of the target region, which is widely used in environmental monitoring and scattering information measurement. In recent years, with the development of compressed sensing (CS) theory, sparse signal processing is used in array SAR 3D imaging. Compared with matched filter (MF), sparse SAR imaging can effectively improve image quality. However, sparse imaging based on handcrafted regularization functions suffers from target information loss in few observed SAR data. Therefore, in this article, a general 3D sparse imaging framework based on Regulation by Denoising (RED) and proximal gradient descent type method for array SAR is presented. Firstly, we construct explicit prior terms via state-of-the-art denoising operators instead of regularization functions, which can improve the accuracy of sparse reconstruction and preserve the structure information of the target. Then, different proximal gradient descent type methods are presented, including a generalized alternating projection (GAP) and an alternating direction method of multiplier (ADMM), which is suitable for high-dimensional data processing. Additionally, the proposed method has robust convergence, which can achieve sparse reconstruction of 3D SAR in few observed SAR data. Extensive simulations and real data experiments are conducted to analyze the performance of the proposed method. The experimental results show that the proposed method has superior sparse reconstruction performance. △ Less

Submitted 26 May, 2024; v1 submitted 9 May, 2024; originally announced May 2024.

FOBNN: Fast Oblivious Binarized Neural Network Inference

Authors: Xin Chen, Zhili Chen, Benchang Dong, Shiwen Wei, Lin Chen, Daojing He

Abstract: The superior performance of deep learning has propelled the rise of Deep Learning as a Service, enabling users to transmit their private data to service providers for model execution and inference retrieval. Nevertheless, the primary concern remains safeguarding the confidentiality of sensitive user data while optimizing the efficiency of secure protocols. To address this, we develop a fast oblivi… ▽ More The superior performance of deep learning has propelled the rise of Deep Learning as a Service, enabling users to transmit their private data to service providers for model execution and inference retrieval. Nevertheless, the primary concern remains safeguarding the confidentiality of sensitive user data while optimizing the efficiency of secure protocols. To address this, we develop a fast oblivious binarized neural network inference framework, FOBNN. Specifically, we customize binarized convolutional neural networks to enhance oblivious inference, design two fast algorithms for binarized convolutions, and optimize network structures experimentally under constrained costs. Initially, we meticulously analyze the range of intermediate values in binarized convolutions to minimize bit representation, resulting in the Bit Length Bounding (BLB) algorithm. Subsequently, leveraging the efficiency of bitwise operations in BLB, we further enhance performance by employing pure bitwise operations for each binary digit position, yielding the Layer-wise Bit Accumulation (LBA) algorithm. Theoretical analysis validates FOBNN's security and indicates up to $2 \times$ improvement in computational and communication costs compared to the state-of-the-art method. We demonstrates our framework's effectiveness in RNA function prediction within bioinformatics. Rigorous experimental assessments confirm that our oblivious inference solutions not only maintain but often exceed the original accuracy, surpassing prior efforts. △ Less

Submitted 5 May, 2024; originally announced May 2024.

Dihadron helicity correlation in photon-nucleus collisions

Authors: Zhao-Xuan Chen, Hui Dong, Shu-Yi Wei

Abstract: The helicity correlation of two back-to-back hadrons is a powerful tool that makes it possible to probe the longitudinal spin transfer, $G_{1L}$, in unpolarized hadronic collisions. In this work, we investigate the helicity correlation of back-to-back dihadrons produced in photon-nucleus collisions with both space-like and quasireal photons and explore its potential in understanding the flavor dep… ▽ More The helicity correlation of two back-to-back hadrons is a powerful tool that makes it possible to probe the longitudinal spin transfer, $G_{1L}$, in unpolarized hadronic collisions. In this work, we investigate the helicity correlation of back-to-back dihadrons produced in photon-nucleus collisions with both space-like and quasireal photons and explore its potential in understanding the flavor dependence of spin-dependent fragmentation functions. We present helicity amplitudes of partonic scatterings with both virtual and real photons and make numerical predictions for the dihadron helicity correlations at the future Electron Ion Collider experiment and the current RHIC/LHC ultra-peripheral collision experiment. Future experimental measurements can also illuminate the fragmentation function of circularly polarized gluons. △ Less

Submitted 23 August, 2024; v1 submitted 29 April, 2024; originally announced April 2024.

Comments: 12 pages

Unveiling the Boson Peaks in Amorphous Phase-Change Materials

Authors: Jens Moesgaard, Tomoki Fujita, Shuai Wei

Abstract: The Boson peak is a universal phenomenon in amorphous solids. It can be observed as an anomalous contribution to the low-temperature heat capacity over the Debye model. Amorphous phase-change materials (PCMs) such as Ge-Sb-Te are a family of poor glass formers with fast crystallization kinetics, being of interest for phase-change memory applications. So far, whether Boson peaks exist in PCMs is un… ▽ More The Boson peak is a universal phenomenon in amorphous solids. It can be observed as an anomalous contribution to the low-temperature heat capacity over the Debye model. Amorphous phase-change materials (PCMs) such as Ge-Sb-Te are a family of poor glass formers with fast crystallization kinetics, being of interest for phase-change memory applications. So far, whether Boson peaks exist in PCMs is unknown and, if they do, their relevance to PCM properties is unclear. Here, we investigate the thermodynamic properties of the pseudo-binary compositions on the tie-line between Ge15Te85 and Ge15Sb85 from a few Kelvins to the liquidus temperatures. Our results demonstrate the evidence of the pronounced Boson peaks in heat capacity below 10 K in the amorphous phase of all compositions. By fitting the data using the Debye model combined with the Einstein model, we can extract the characteristic parameters of the Boson peaks and attribute their origin to the excess vibrational modes of dynamic defects in the amorphous solids. We find that these parameters correlate almost linearly with the Sb-content of the alloys, despite the nonmonotonic behaviors in glass forming abilities. A larger contribution of excess vibrational modes correlates with a larger width of enthalpy relaxation below the glass transition temperature Tg. In a broader context, we show that the correlations of the characteristic parameters of the Boson peaks with Tg and kinetic fragility, vary according to the type of bonding. Specifically, metallic glasses and conventional covalent glasses exhibit distinct patterns of dependence, whereas PCMs manifest characteristics that lie in between. A deeper understanding of the Boson peaks in PCMs holds the promise to enable predictions of material properties at higher temperatures based on features observed in low-temperature heat capacity. △ Less

Submitted 26 April, 2024; originally announced April 2024.

3D deep learning for enhanced atom probe tomography analysis of nanoscale microstructures

Authors: Jiwei Yu, Zhangwei Wang, Aparna Saksena, Shaolou Wei, Ye Wei, Timoteo Colnaghi, Andreas Marek, Markus Rampp, Min Song, Baptiste Gault, Yue Li

Abstract: Quantitative analysis of microstructural features on the nanoscale, including precipitates, local chemical orderings (LCOs) or structural defects (e.g. stacking faults) plays a pivotal role in understanding the mechanical and physical responses of engineering materials. Atom probe tomography (APT), known for its exceptional combination of chemical sensitivity and sub-nanometer resolution, primaril… ▽ More Quantitative analysis of microstructural features on the nanoscale, including precipitates, local chemical orderings (LCOs) or structural defects (e.g. stacking faults) plays a pivotal role in understanding the mechanical and physical responses of engineering materials. Atom probe tomography (APT), known for its exceptional combination of chemical sensitivity and sub-nanometer resolution, primarily identifies microstructures through compositional segregations. However, this fails when there is no significant segregation, as can be the case for LCOs and stacking faults. Here, we introduce a 3D deep learning approach, AtomNet, designed to process APT point cloud data at the single-atom level for nanoscale microstructure extraction, simultaneously considering compositional and structural information. AtomNet is showcased in segmenting L12-type nanoprecipitates from the matrix in an AlLiMg alloy, irrespective of crystallographic orientations, which outperforms previous methods. AtomNet also allows for 3D imaging of L10-type LCOs in an AuCu alloy, a challenging task for conventional analysis due to their small size and subtle compositional differences. Finally, we demonstrate the use of AtomNet for revealing 2D stacking faults in a Co-based superalloy, without any defected training data, expanding the capabilities of APT for automated exploration of hidden microstructures. AtomNet pushes the boundaries of APT analysis, and holds promise in establishing precise quantitative microstructure-property relationships across a diverse range of metallic materials. △ Less

Submitted 25 April, 2024; originally announced April 2024.

Research on OPF control of three-phase four-wire low-voltage distribution network considering uncertainty

Authors: Rui Wang, Xiaoqing Bai, Shengquan Huang, Shoupu Wei

Abstract: As power systems become more complex and uncertain, low-voltage distribution networks face numerous challenges, including three-phase imbalances caused by asymmetrical loads and distributed energy resources. We propose a robust stochastic optimization (RSO) based optimal power flow (OPF) control method for three-phase, four-wire low-voltage distribution networks that consider uncertainty to addres… ▽ More As power systems become more complex and uncertain, low-voltage distribution networks face numerous challenges, including three-phase imbalances caused by asymmetrical loads and distributed energy resources. We propose a robust stochastic optimization (RSO) based optimal power flow (OPF) control method for three-phase, four-wire low-voltage distribution networks that consider uncertainty to address these issues. Using historical data and deep learning classification methods, the proposed method simulates optimal system behaviour without requiring communication infrastructure. The simulation results verify that the proposed method effectively controls the voltage and current amplitude while minimizing the operational cost and three-phase imbalance within acceptable limits. The proposed method shows promise for managing uncertainties and optimizing performance in low-voltage distribution networks. △ Less

Submitted 23 April, 2024; originally announced April 2024.

Comments: systems optimization, robust optimization, local control

Transformation operators and the Kastler-Kalau-Walze type theorems on 4-dimensional manifolds

Authors: Sining Wei, Yong Wang

Abstract: In this paper, we compute the lower-dimensional volume Vol(1,1) about transformation operators for 4- dimensional spin manifolds with boundary and we also get the Kastler-Kalau-Walze type theorem about transformation operators on 4-dimensional compact manifolds with boundary. In this paper, we compute the lower-dimensional volume Vol(1,1) about transformation operators for 4- dimensional spin manifolds with boundary and we also get the Kastler-Kalau-Walze type theorem about transformation operators on 4-dimensional compact manifolds with boundary. △ Less

Submitted 12 July, 2024; v1 submitted 22 April, 2024; originally announced April 2024.

Comments: arXiv admin note: text overlap with arXiv:2211.06125. arXiv admin note: text overlap with arXiv:2211.06125

TextSquare: Scaling up Text-Centric Visual Instruction Tuning

Authors: Jingqun Tang, Chunhui Lin, Zhen Zhao, Shu Wei, Binghong Wu, Qi Liu, Hao Feng, Yang Li, Siqi Wang, Lei Liao, Wei Shi, Yuliang Liu, Hao Liu, Yuan Xie, Xiang Bai, Can Huang

Abstract: Text-centric visual question answering (VQA) has made great strides with the development of Multimodal Large Language Models (MLLMs), yet open-source models still fall short of leading models like GPT4V and Gemini, partly due to a lack of extensive, high-quality instruction tuning data. To this end, we introduce a new approach for creating a massive, high-quality instruction-tuning dataset, Square… ▽ More Text-centric visual question answering (VQA) has made great strides with the development of Multimodal Large Language Models (MLLMs), yet open-source models still fall short of leading models like GPT4V and Gemini, partly due to a lack of extensive, high-quality instruction tuning data. To this end, we introduce a new approach for creating a massive, high-quality instruction-tuning dataset, Square-10M, which is generated using closed-source MLLMs. The data construction process, termed Square, consists of four steps: Self-Questioning, Answering, Reasoning, and Evaluation. Our experiments with Square-10M led to three key findings: 1) Our model, TextSquare, considerably surpasses open-source previous state-of-the-art Text-centric MLLMs and sets a new standard on OCRBench(62.2%). It even outperforms top-tier models like GPT4V and Gemini in 6 of 10 text-centric benchmarks. 2) Additionally, we demonstrate the critical role of VQA reasoning data in offering comprehensive contextual insights for specific questions. This not only improves accuracy but also significantly mitigates hallucinations. Specifically, TextSquare scores an average of 75.1% across four general VQA and hallucination evaluation datasets, outperforming previous state-of-the-art models. 3) Notably, the phenomenon observed in scaling text-centric VQA datasets reveals a vivid pattern: the exponential increase of instruction tuning data volume is directly proportional to the improvement in model performance, thereby validating the necessity of the dataset scale and the high quality of Square-10M. △ Less

Submitted 19 April, 2024; originally announced April 2024.