Showing 1–50 of 784 results for author: Gao, W

Spherical quasi-interpolation using scaled zonal kernels

Authors: Zhengjie Sun, Wenwu Gao, Xingping Sun

Abstract: We propose and study a new quasi-interpolation method on spheres featuring the following two-phase construction and analysis. In Phase I, we analyze and characterize a large family of zonal kernels (e.g., the spherical version of Poisson kernel, Gaussian, compactly-supported radial kernels), so that the underlying spherical convolution operators (upon the introduction of a scaling parameter) attai… ▽ More We propose and study a new quasi-interpolation method on spheres featuring the following two-phase construction and analysis. In Phase I, we analyze and characterize a large family of zonal kernels (e.g., the spherical version of Poisson kernel, Gaussian, compactly-supported radial kernels), so that the underlying spherical convolution operators (upon the introduction of a scaling parameter) attains a high-order of approximation to target functions. In Phase II, we discretize the spherical integrals utilizing quadrature rules of optimal order to produce the final quasi-interpolants. Numerical experiments demonstrate that the new quasi-interpolation algorithm is robust and amenable to integrated as well as distributed ways of implementation. Moreover, the underlying error-analysis shows that by fine-tuning the scaling parameter in the radial kernels employed, the resulting quasi-interpolants achieve a well-balanced trade-off between approximation and sampling errors. △ Less

Submitted 27 August, 2024; originally announced August 2024.

MSC Class: 43A90; 41A25; 41A55; 65D12; 65D32

Fire-Flyer AI-HPC: A Cost-Effective Software-Hardware Co-Design for Deep Learning

Authors: Wei An, Xiao Bi, Guanting Chen, Shanhuang Chen, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Wenjun Gao, Kang Guan, Jianzhong Guo, Yongqiang Guo, Zhe Fu, Ying He, Panpan Huang, Jiashi Li, Wenfeng Liang, Xiaodong Liu, Xin Liu, Yiyuan Liu, Yuxuan Liu, Shanghao Lu, Xuan Lu, Xiaotao Nie, Tian Pei , et al. (27 additional authors not shown)

Abstract: The rapid progress in Deep Learning (DL) and Large Language Models (LLMs) has exponentially increased demands of computational power and bandwidth. This, combined with the high costs of faster computing chips and interconnects, has significantly inflated High Performance Computing (HPC) construction costs. To address these challenges, we introduce the Fire-Flyer AI-HPC architecture, a synergistic… ▽ More The rapid progress in Deep Learning (DL) and Large Language Models (LLMs) has exponentially increased demands of computational power and bandwidth. This, combined with the high costs of faster computing chips and interconnects, has significantly inflated High Performance Computing (HPC) construction costs. To address these challenges, we introduce the Fire-Flyer AI-HPC architecture, a synergistic hardware-software co-design framework and its best practices. For DL training, we deployed the Fire-Flyer 2 with 10,000 PCIe A100 GPUs, achieved performance approximating the DGX-A100 while reducing costs by half and energy consumption by 40%. We specifically engineered HFReduce to accelerate allreduce communication and implemented numerous measures to keep our Computation-Storage Integrated Network congestion-free. Through our software stack, including HaiScale, 3FS, and HAI-Platform, we achieved substantial scalability by overlapping computation and communication. Our system-oriented experience from DL training provides valuable insights to drive future advancements in AI-HPC. △ Less

Submitted 26 August, 2024; originally announced August 2024.

Comments: This is the preprint version of the paper accepted for presentation at the 2024 International Conference for High Performance Computing, Networking, Storage, and Analysis (SC'24). \c{opyright} 2024 IEEE. Personal use of this material is permitted. For other uses, permission from IEEE must be obtained. Please refer to IEEE Xplore for the final published version

Room-temperature Optically Detected Magnetic Resonance of Telecom Single Photon Emitters in GaN

Authors: John J. H. Eng, Zhengzhi Jiang, Max Meunier, Abdullah Rasmita, Haoran Zhang, Yuzhe Yang, Feifei Zhou, Hongbing Cai, Zhaogang Dong, Jesús Zúñiga Pérez, Weibo Gao

Abstract: Solid-state defects susceptible of spin manipulation hold great promise for scalable quantum technology. To broaden their utility, operating at room temperature and emitting in the telecom wavelength range are desired, eliminating cryogenic requirements and leveraging existing optical fiber infrastructure for transmitting the quantum information. To that end, we report that telecom single photon e… ▽ More Solid-state defects susceptible of spin manipulation hold great promise for scalable quantum technology. To broaden their utility, operating at room temperature and emitting in the telecom wavelength range are desired, eliminating cryogenic requirements and leveraging existing optical fiber infrastructure for transmitting the quantum information. To that end, we report that telecom single photon emitters (SPEs) in gallium nitride (GaN) exhibit optically detected magnetic resonance (ODMR) at room temperature. The analysis of ODMR as a function of magnetic field orientation enables the determination of the orientation of the spin quantization axis with respect to the GaN crystalline lattice. The optical transitions dynamics are analyzed to gain further insight into the transition rates dominating ODMR. Our findings, coupled with GaN's mature fabrication technology, could facilitate the realization of scalable quantum technology. △ Less

Submitted 26 August, 2024; originally announced August 2024.

Could Bibliometrics Reveal Top Science and Technology Achievements and Researchers? The Case for Evaluatology-based Science and Technology Evaluation

Authors: Guoxin Kang, Wanling Gao, Lei Wang, Chunjie Luo, Hainan Ye, Qian He, Shaopeng Dai, Jianfeng Zhan

Abstract: By utilizing statistical methods to analyze bibliographic data, bibliometrics faces inherent limitations in identifying the most significant science and technology achievements and researchers. To overcome this challenge, we present an evaluatology-based science and technology evaluation methodology. At the heart of this approach lies the concept of an extended evaluation condition, encompassing e… ▽ More By utilizing statistical methods to analyze bibliographic data, bibliometrics faces inherent limitations in identifying the most significant science and technology achievements and researchers. To overcome this challenge, we present an evaluatology-based science and technology evaluation methodology. At the heart of this approach lies the concept of an extended evaluation condition, encompassing eight crucial components derived from a field. We define four relationships that illustrate the connections among various achievements based on their mapped extended EC components, as well as their temporal and citation links. Within a relationship under an extended evaluation condition, evaluators can effectively compare these achievements by carefully addressing the influence of confounding variables. We establish a real-world evaluation system encompassing an entire collection of achievements, each of which is mapped to several components of an extended EC. Within a specific field like chip technology or open source, we construct a perfect evaluation model that can accurately trace the evolution and development of all achievements in terms of four relationships based on the real-world evaluation system. Building upon the foundation of the perfect evaluation model, we put forth four-round rules to eliminate non-significant achievements by utilizing four relationships. This process allows us to establish a pragmatic evaluation model that effectively captures the essential achievements, serving as a curated collection of the top N achievements within a specific field during a specific timeframe. We present a case study on the top 100 Chip achievements which highlights its practical application and efficacy in identifying significant achievements and researchers that otherwise can not be identified by using bibliometrics. △ Less

Submitted 22 August, 2024; originally announced August 2024.

Comments: 18 pages, 8 figures, and 2 tables

Through-the-Wall Radar Human Activity Micro-Doppler Signature Representation Method Based on Joint Boulic-Sinusoidal Pendulum Model

Authors: Xiaopeng Yang, Weicheng Gao, Xiaodong Qu, Zeyu Ma, Hao Zhang

Abstract: With the help of micro-Doppler signature, ultra-wideband (UWB) through-the-wall radar (TWR) enables the reconstruction of range and velocity information of limb nodes to accurately identify indoor human activities. However, existing methods are usually trained and validated directly using range-time maps (RTM) and Doppler-time maps (DTM), which have high feature redundancy and poor generalization… ▽ More With the help of micro-Doppler signature, ultra-wideband (UWB) through-the-wall radar (TWR) enables the reconstruction of range and velocity information of limb nodes to accurately identify indoor human activities. However, existing methods are usually trained and validated directly using range-time maps (RTM) and Doppler-time maps (DTM), which have high feature redundancy and poor generalization ability. In order to solve this problem, this paper proposes a human activity micro-Doppler signature representation method based on joint Boulic-sinusoidal pendulum motion model. In detail, this paper presents a simplified joint Boulic-sinusoidal pendulum human motion model by taking head, torso, both hands and feet into consideration improved from Boulic-Thalmann kinematic model. The paper also calculates the minimum number of key points needed to describe the Doppler and micro-Doppler information sufficiently. Both numerical simulations and experiments are conducted to verify the effectiveness. The results demonstrate that the proposed number of key points of micro-Doppler signature can precisely represent the indoor human limb node motion characteristics, and substantially improve the generalization capability of the existing methods for different testers. △ Less

Submitted 21 August, 2024; originally announced August 2024.

Comments: 17 pages, 14 figures, 7 tables, in IEEE Transactions on Microwave Theory and Techniques, 2024

MSC Class: 94 ACM Class: I.5.1

A Deconfounding Approach to Climate Model Bias Correction

Authors: Wentao Gao, Jiuyong Li, Debo Cheng, Lin Liu, Jixue Liu, Thuc Duy Le, Xiaojing Du, Xiongren Chen, Yanchang Zhao, Yun Chen

Abstract: Global Climate Models (GCMs) are crucial for predicting future climate changes by simulating the Earth systems. However, GCM outputs exhibit systematic biases due to model uncertainties, parameterization simplifications, and inadequate representation of complex climate phenomena. Traditional bias correction methods, which rely on historical observation data and statistical techniques, often neglec… ▽ More Global Climate Models (GCMs) are crucial for predicting future climate changes by simulating the Earth systems. However, GCM outputs exhibit systematic biases due to model uncertainties, parameterization simplifications, and inadequate representation of complex climate phenomena. Traditional bias correction methods, which rely on historical observation data and statistical techniques, often neglect unobserved confounders, leading to biased results. This paper proposes a novel bias correction approach to utilize both GCM and observational data to learn a factor model that captures multi-cause latent confounders. Inspired by recent advances in causality based time series deconfounding, our method first constructs a factor model to learn latent confounders from historical data and then applies them to enhance the bias correction process using advanced time series forecasting models. The experimental results demonstrate significant improvements in the accuracy of precipitation outputs. By addressing unobserved confounders, our approach offers a robust and theoretically grounded solution for climate model bias correction. △ Less

Submitted 21 August, 2024; originally announced August 2024.

Estimating Peer Direct and Indirect Effects in Observational Network Data

Authors: Xiaojing Du, Jiuyong Li, Debo Cheng, Lin Liu, Wentao Gao, Xiongren Chen

Abstract: Estimating causal effects is crucial for decision-makers in many applications, but it is particularly challenging with observational network data due to peer interactions. Many algorithms have been proposed to estimate causal effects involving network data, particularly peer effects, but they often overlook the variety of peer effects. To address this issue, we propose a general setting which cons… ▽ More Estimating causal effects is crucial for decision-makers in many applications, but it is particularly challenging with observational network data due to peer interactions. Many algorithms have been proposed to estimate causal effects involving network data, particularly peer effects, but they often overlook the variety of peer effects. To address this issue, we propose a general setting which considers both peer direct effects and peer indirect effects, and the effect of an individual's own treatment, and provide identification conditions of these causal effects and proofs. To estimate these causal effects, we utilize attention mechanisms to distinguish the influences of different neighbors and explore high-order neighbor effects through multi-layer graph neural networks (GNNs). Additionally, to control the dependency between node features and representations, we incorporate the Hilbert-Schmidt Independence Criterion (HSIC) into the GNN, fully utilizing the structural information of the graph, to enhance the robustness and accuracy of the model. Extensive experiments on two semi-synthetic datasets confirm the effectiveness of our approach. Our theoretical findings have the potential to improve intervention strategies in networked systems, with applications in areas such as social networks and epidemiology. △ Less

Submitted 21 August, 2024; originally announced August 2024.

Comments: AAAI

E-Bench: Subjective-Aligned Benchmark Suite for Text-Driven Video Editing Quality Assessment

Authors: Shangkun Sun, Xiaoyu Liang, Songlin Fan, Wenxu Gao, Wei Gao

Abstract: Text-driven video editing has recently experienced rapid development. Despite this, evaluating edited videos remains a considerable challenge. Current metrics tend to fail to align with human perceptions, and effective quantitative metrics for video editing are still notably absent. To address this, we introduce E-Bench, a benchmark suite tailored to the assessment of text-driven video editing. Th… ▽ More Text-driven video editing has recently experienced rapid development. Despite this, evaluating edited videos remains a considerable challenge. Current metrics tend to fail to align with human perceptions, and effective quantitative metrics for video editing are still notably absent. To address this, we introduce E-Bench, a benchmark suite tailored to the assessment of text-driven video editing. This suite includes E-Bench DB, a video quality assessment (VQA) database for video editing. E-Bench DB encompasses a diverse set of source videos featuring various motions and subjects, along with multiple distinct editing prompts, editing results from 8 different models, and the corresponding Mean Opinion Scores (MOS) from 24 human annotators. Based on E-Bench DB, we further propose E-Bench QA, a quantitative human-aligned measurement for the text-driven video editing task. In addition to the aesthetic, distortion, and other visual quality indicators that traditional VQA methods emphasize, E-Bench QA focuses on the text-video alignment and the relevance modeling between source and edited videos. It proposes a new assessment network for video editing that attains superior performance in alignment with human preferences. To the best of our knowledge, E-Bench introduces the first quality assessment dataset for video editing and an effective subjective-aligned quantitative metric for this domain. All data and code will be publicly available at https://github.com/littlespray/E-Bench. △ Less

Submitted 21 August, 2024; originally announced August 2024.

Inverse model for network construction: (δ(G), I' (G)) -> G

Authors: Wei Gao, Yaojun Chen, Hainan Zhang

Abstract: The isolated toughness variant is a salient parameter for measuring the vulnerability of networks, which is inherently related to fractional factors (used to characterize the feasibility of data transmission). The combination of minimum degree and the corresponding tight bound of isolated toughness variant for fractional factors provide reference standards for network construction. However, previo… ▽ More The isolated toughness variant is a salient parameter for measuring the vulnerability of networks, which is inherently related to fractional factors (used to characterize the feasibility of data transmission). The combination of minimum degree and the corresponding tight bound of isolated toughness variant for fractional factors provide reference standards for network construction. However, previous advances only focused on how to select the optimal parameter criteria from Pareto front, without any suggestion for the construction of specific networks. To overcome this deficiency, this paper proposes an inverse model from $(δ(G),I'(G))$ to $G$, by means of evolutionary computing approach, we propose a novel inverse model to obtain the optimal solutions for candidate graphs. The main procedure is composed of pseudo-greedy acceleration, cross-mutation and diversity enhancement modules. The practicality of the algorithm is verified by means of pilot experiments. The code in this paper is made public on https://github.com/AizhEngHN/Inverse-model-for-network-construction-G-I-G-G. △ Less

Submitted 20 August, 2024; originally announced August 2024.

Comments: 22 pages, 7 figures

Image-based Freeform Handwriting Authentication with Energy-oriented Self-Supervised Learning

Authors: Jingyao Wang, Luntian Mou, Changwen Zheng, Wen Gao

Abstract: Freeform handwriting authentication verifies a person's identity from their writing style and habits in messy handwriting data. This technique has gained widespread attention in recent years as a valuable tool for various fields, e.g., fraud prevention and cultural heritage protection. However, it still remains a challenging task in reality due to three reasons: (i) severe damage, (ii) complex hig… ▽ More Freeform handwriting authentication verifies a person's identity from their writing style and habits in messy handwriting data. This technique has gained widespread attention in recent years as a valuable tool for various fields, e.g., fraud prevention and cultural heritage protection. However, it still remains a challenging task in reality due to three reasons: (i) severe damage, (ii) complex high-dimensional features, and (iii) lack of supervision. To address these issues, we propose SherlockNet, an energy-oriented two-branch contrastive self-supervised learning framework for robust and fast freeform handwriting authentication. It consists of four stages: (i) pre-processing: converting manuscripts into energy distributions using a novel plug-and-play energy-oriented operator to eliminate the influence of noise; (ii) generalized pre-training: learning general representation through two-branch momentum-based adaptive contrastive learning with the energy distributions, which handles the high-dimensional features and spatial dependencies of handwriting; (iii) personalized fine-tuning: calibrating the learned knowledge using a small amount of labeled data from downstream tasks; and (iv) practical application: identifying individual handwriting from scrambled, missing, or forged data efficiently and conveniently. Considering the practicality, we construct EN-HA, a novel dataset that simulates data forgery and severe damage in real applications. Finally, we conduct extensive experiments on six benchmark datasets including our EN-HA, and the results prove the robustness and efficiency of SherlockNet. △ Less

Submitted 18 August, 2024; originally announced August 2024.

Comments: Accepted by TMM

LLM-PCGC: Large Language Model-based Point Cloud Geometry Compression

Authors: Yuqi Ye, Wei Gao

Abstract: The key to effective point cloud compression is to obtain a robust context model consistent with complex 3D data structures. Recently, the advancement of large language models (LLMs) has highlighted their capabilities not only as powerful generators for in-context learning and generation but also as effective compressors. These dual attributes of LLMs make them particularly well-suited to meet the… ▽ More The key to effective point cloud compression is to obtain a robust context model consistent with complex 3D data structures. Recently, the advancement of large language models (LLMs) has highlighted their capabilities not only as powerful generators for in-context learning and generation but also as effective compressors. These dual attributes of LLMs make them particularly well-suited to meet the demands of data compression. Therefore, this paper explores the potential of using LLM for compression tasks, focusing on lossless point cloud geometry compression (PCGC) experiments. However, applying LLM directly to PCGC tasks presents some significant challenges, i.e., LLM does not understand the structure of the point cloud well, and it is a difficult task to fill the gap between text and point cloud through text description, especially for large complicated and small shapeless point clouds. To address these problems, we introduce a novel architecture, namely the Large Language Model-based Point Cloud Geometry Compression (LLM-PCGC) method, using LLM to compress point cloud geometry information without any text description or aligning operation. By utilizing different adaptation techniques for cross-modality representation alignment and semantic consistency, including clustering, K-tree, token mapping invariance, and Low Rank Adaptation (LoRA), the proposed method can translate LLM to a compressor/generator for point cloud. To the best of our knowledge, this is the first structure to employ LLM as a compressor for point cloud data. Experiments demonstrate that the LLM-PCGC outperforms the other existing methods significantly, by achieving -40.213% bit rate reduction compared to the reference software of MPEG Geometry-based Point Cloud Compression (G-PCC) standard, and by achieving -2.267% bit rate reduction compared to the state-of-the-art learning-based method. △ Less

Submitted 16 August, 2024; originally announced August 2024.

DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Search

Authors: Huajian Xin, Z. Z. Ren, Junxiao Song, Zhihong Shao, Wanjia Zhao, Haocheng Wang, Bo Liu, Liyue Zhang, Xuan Lu, Qiushi Du, Wenjun Gao, Qihao Zhu, Dejian Yang, Zhibin Gou, Z. F. Wu, Fuli Luo, Chong Ruan

Abstract: We introduce DeepSeek-Prover-V1.5, an open-source language model designed for theorem proving in Lean 4, which enhances DeepSeek-Prover-V1 by optimizing both training and inference processes. Pre-trained on DeepSeekMath-Base with specialization in formal mathematical languages, the model undergoes supervised fine-tuning using an enhanced formal theorem proving dataset derived from DeepSeek-Prover-… ▽ More We introduce DeepSeek-Prover-V1.5, an open-source language model designed for theorem proving in Lean 4, which enhances DeepSeek-Prover-V1 by optimizing both training and inference processes. Pre-trained on DeepSeekMath-Base with specialization in formal mathematical languages, the model undergoes supervised fine-tuning using an enhanced formal theorem proving dataset derived from DeepSeek-Prover-V1. Further refinement is achieved through reinforcement learning from proof assistant feedback (RLPAF). Beyond the single-pass whole-proof generation approach of DeepSeek-Prover-V1, we propose RMaxTS, a variant of Monte-Carlo tree search that employs an intrinsic-reward-driven exploration strategy to generate diverse proof paths. DeepSeek-Prover-V1.5 demonstrates significant improvements over DeepSeek-Prover-V1, achieving new state-of-the-art results on the test set of the high school level miniF2F benchmark ($63.5\%$) and the undergraduate level ProofNet benchmark ($25.3\%$). △ Less

Submitted 15 August, 2024; originally announced August 2024.

Optimal overlapping tomography

Authors: Kiara Hansenne, Rui Qu, Lisa T. Weinbrenner, Carlos de Gois, Haifei Wang, Yang Ming, Zhengning Yang, Paweł Horodecki, Weibo Gao, Otfried Gühne

Abstract: Characterising large scale quantum systems is central for fundamental physics as well as for applications of quantum technologies. While a full characterisation requires exponentially increasing effort, focusing on application-relevant information can often lead to significantly simplified analysis. Overlapping tomography is such a scheme, which allows to obtain all the information contained in sp… ▽ More Characterising large scale quantum systems is central for fundamental physics as well as for applications of quantum technologies. While a full characterisation requires exponentially increasing effort, focusing on application-relevant information can often lead to significantly simplified analysis. Overlapping tomography is such a scheme, which allows to obtain all the information contained in specific subsystems of multi-particle quantum systems in an efficient manner, but the ultimate limits of this approach remained elusive. We present protocols for optimal overlapping tomography with respect to different figures of merit. First, by providing algorithmic approaches based on graph theory we find the optimal scheme for Pauli measurements on qubits, relating it to the problem of covering arrays in combinatorics. This significantly reduces the measurement effort, showing for instance that two-body overlapping tomography of nearest neighbours in multiqubit quantum systems can always be performed with nine Pauli settings. Second, we prove that the optimal scheme using general projective measurements requires only $3^k$ settings to reconstruct all $k$-body marginals, independently of the system size. Finally, we demonstrate the practical applicability of our methods in a six-photon experiment. Our results will find applications in learning noise and interaction patterns in quantum computers as well as characterising fermionic systems in quantum chemistry. △ Less

Submitted 11 August, 2024; originally announced August 2024.

Digitized Phase Change Material Heterostack for Diffractive Optical Neural Network

Authors: Ruiyang Chen, Cunxi Yu, Weilu Gao

Abstract: All-optical and fully reconfigurable diffractive optical neural network (DONN) architectures are promising for high-throughput and energy-efficient machine learning (ML) hardware accelerators for broad applications. However, current device and system implementations have limited performance. This work demonstrates a novel diffractive device architecture, which is named digitized heterostack and co… ▽ More All-optical and fully reconfigurable diffractive optical neural network (DONN) architectures are promising for high-throughput and energy-efficient machine learning (ML) hardware accelerators for broad applications. However, current device and system implementations have limited performance. This work demonstrates a novel diffractive device architecture, which is named digitized heterostack and consists of multiple layers of nonvolatile phase change materials (PCMs) with different thicknesses. This architecture can both leverage the advantages of PCM optical properties and mitigate challenges associated with implementing multilevel operations in a single PCM layer. Proof-of-concept experiments demonstrate the electrical tuning of one PCM layer in a spatial light modulation device, and thermal analysis guides the design of DONN devices and systems to avoid thermal crosstalk if individual heterostacks are assembled into an array. Further, heterostacks containing three PCM layers are designed to have a large phase modulation range and uniform coverage and the ML performance of DONN systems with designed heterostacks is evaluated. The developed device architecture provides new opportunities for desirable energy-efficient, fast-reconfigured, and compact DONN systems in the future. △ Less

Submitted 2 August, 2024; originally announced August 2024.

A Reinforcement Learning Based Motion Planner for Quadrotor Autonomous Flight in Dense Environment

Authors: Zhaohong Liu, Wenxuan Gao, Yinshuai Sun, Peng Dong

Abstract: Quadrotor motion planning is critical for autonomous flight in complex environments, such as rescue operations. Traditional methods often employ trajectory generation optimization and passive time allocation strategies, which can limit the exploitation of the quadrotor's dynamic capabilities and introduce delays and inaccuracies. To address these challenges, we propose a novel motion planning fram… ▽ More Quadrotor motion planning is critical for autonomous flight in complex environments, such as rescue operations. Traditional methods often employ trajectory generation optimization and passive time allocation strategies, which can limit the exploitation of the quadrotor's dynamic capabilities and introduce delays and inaccuracies. To address these challenges, we propose a novel motion planning framework that integrates visibility path searching and reinforcement learning (RL) motion generation. Our method constructs collision-free paths using heuristic search and visibility graphs, which are then refined by an RL policy to generate low-level motion commands. We validate our approach in simulated indoor environments, demonstrating better performance than traditional methods in terms of time span. △ Less

Submitted 5 August, 2024; v1 submitted 1 August, 2024; originally announced August 2024.

High-order quasi-interpolation with generalized Gaussian kernels restricted over tori

Authors: Wenwu Gao, Zhengjie Sun, Changwei Wang

Abstract: The paper proposes a novel and efficient quasi-interpolation scheme with high approximation order for periodic function approximation over tori. The resulting quasi-interpolation takes the form of Schoenberg's tensor-product generalized Gaussian kernels restricted over circles. Notably, theoretical analysis shows that it achieves the highest approximation order equal to the order of the generalize… ▽ More The paper proposes a novel and efficient quasi-interpolation scheme with high approximation order for periodic function approximation over tori. The resulting quasi-interpolation takes the form of Schoenberg's tensor-product generalized Gaussian kernels restricted over circles. Notably, theoretical analysis shows that it achieves the highest approximation order equal to the order of the generalized Strang-Fix condition satisfied by the generalized Gaussian kernels. This is in sharp contrast to classical quasi-interpolation counterparts, which often provide much lower approximation orders than those dictated by the generalized Strang-Fix conditions satisfied by the kernels. Furthermore, we construct a sparse grid counterpart for high-dimensional periodic function approximation to alleviate the curse of dimensionality. Numerical simulations provided at the end of the paper demonstrate that our quasi-interpolation scheme is simple and computationally efficient. △ Less

Submitted 30 July, 2024; originally announced July 2024.

MSC Class: 41A30; 41A25; 42B05; 65D15

A Local Modal Outer-Product-Gradient Estimator for Dimension Reduction

Authors: Zheng Li, Chong Ding, Wei Gao

Abstract: Sufficient dimension reduction (SDR) is a valuable approach for handling high-dimensional data. Outer Product Gradient (OPG) is an popular approach. However, because of focusing the mean regression function, OPG may ignore some directions of central subspace (CS) when the distribution of errors is symmetric about zero. The mode of a distribution can provide an important summary of data. A Local Mo… ▽ More Sufficient dimension reduction (SDR) is a valuable approach for handling high-dimensional data. Outer Product Gradient (OPG) is an popular approach. However, because of focusing the mean regression function, OPG may ignore some directions of central subspace (CS) when the distribution of errors is symmetric about zero. The mode of a distribution can provide an important summary of data. A Local Modal OPG (LMOPG) and its algorithm through mode regression are proposed to estimate the basis of CS with skew errors distribution. The estimator shows the consistent and asymptotic normal distribution under some mild conditions. Monte Carlo simulation is used to evaluate the performance and demonstrate the efficiency and robustness of the proposed method. △ Less

Submitted 30 July, 2024; originally announced July 2024.

Federated Learning as a Service for Hierarchical Edge Networks with Heterogeneous Models

Authors: Wentao Gao, Omid Tavallaie, Shuaijun Chen, Albert Zomaya

Abstract: Federated learning (FL) is a distributed Machine Learning (ML) framework that is capable of training a new global model by aggregating clients' locally trained models without sharing users' original data. Federated learning as a service (FLaaS) offers a privacy-preserving approach for training machine learning models on devices with various computational resources. Most proposed FL-based methods t… ▽ More Federated learning (FL) is a distributed Machine Learning (ML) framework that is capable of training a new global model by aggregating clients' locally trained models without sharing users' original data. Federated learning as a service (FLaaS) offers a privacy-preserving approach for training machine learning models on devices with various computational resources. Most proposed FL-based methods train the same model in all client devices regardless of their computational resources. However, in practical Internet of Things (IoT) scenarios, IoT devices with limited computational resources may not be capable of training models that client devices with greater hardware performance hosted. Most of the existing FL frameworks that aim to solve the problem of aggregating heterogeneous models are designed for Independent and Identical Distributed (IID) data, which may make it hard to reach the target algorithm performance when encountering non-IID scenarios. To address these problems in hierarchical networks, in this paper, we propose a heterogeneous aggregation framework for hierarchical edge systems called HAF-Edge. In our proposed framework, we introduce a communication-efficient model aggregation method designed for FL systems with two-level model aggregations running at the edge and cloud levels. This approach enhances the convergence rate of the global model by leveraging selective knowledge transfer during the aggregation of heterogeneous models. To the best of our knowledge, this work is pioneering in addressing the problem of aggregating heterogeneous models within hierarchical FL systems spanning IoT, edge, and cloud environments. We conducted extensive experiments to validate the performance of our proposed method. The evaluation results demonstrate that HAF-Edge significantly outperforms state-of-the-art methods. △ Less

Submitted 30 July, 2024; originally announced July 2024.

OptiMUS-0.3: Using Large Language Models to Model and Solve Optimization Problems at Scale

Authors: Ali AhmadiTeshnizi, Wenzhi Gao, Herman Brunborg, Shayan Talaei, Madeleine Udell

Abstract: Optimization problems are pervasive in sectors from manufacturing and distribution to healthcare. However, most such problems are still solved heuristically by hand rather than optimally by state-of-the art solvers because the expertise required to formulate and solve these problems limits the widespread adoption of optimization tools and techniques. We introduce a Large Language Model (LLM)-based… ▽ More Optimization problems are pervasive in sectors from manufacturing and distribution to healthcare. However, most such problems are still solved heuristically by hand rather than optimally by state-of-the art solvers because the expertise required to formulate and solve these problems limits the widespread adoption of optimization tools and techniques. We introduce a Large Language Model (LLM)-based system designed to formulate and solve (mixed integer) linear programming problems from their natural language descriptions. Our system is capable of developing mathematical models, writing and debugging solver code, evaluating the generated solutions, and improving efficiency and correctness of its model and code based on these evaluations. OptiMUS-0.3 utilizes a modular structure to process problems, allowing it to handle problems with long descriptions and complex data without long prompts. Experiments demonstrate that OptiMUS-0.3 outperforms existing state-of-the-art methods on easy datasets by more than 12% and on hard datasets (including a new dataset, NLP4LP, released with this paper that features long and complex problems) by more than 8%. △ Less

Submitted 28 July, 2024; originally announced July 2024.

Comments: This paper documents OptiMUS-0.3, improving on OptiMUS-0.1 (arXiv:2310.06116) and OptiMUS-0.2 (arXiv:2402.10172). arXiv admin note: text overlap with arXiv:2402.10172

Intrinsic Nonlinear Spin Hall Effect and Manipulation of Perpendicular Magnetization

Authors: Hui Wang, Huiying Liu, Xukun Feng, Jin Cao, Weikang Wu, Shen Lai, Weibo Gao, Cong Xiao, Shengyuan A. Yang

Abstract: We propose an intrinsic nonlinear spin Hall effect, which enables the generation of collinearly-polarized spin current in a large class of nonmagnetic materials with the corresponding linear response being symmetry-forbidden. This opens a new avenue for field-free switching of perpendicular magnetization, which is required for the next-generation information storage technology. We develop the micr… ▽ More We propose an intrinsic nonlinear spin Hall effect, which enables the generation of collinearly-polarized spin current in a large class of nonmagnetic materials with the corresponding linear response being symmetry-forbidden. This opens a new avenue for field-free switching of perpendicular magnetization, which is required for the next-generation information storage technology. We develop the microscopic theory of this effect, and clarify its quantum origin in band geometric quantities which can be enhanced by topological nodal features. Combined with first-principles calculations, we predict pronounced effects at room temperature in topological metals $\mathrm{PbTaSe_{2}}$ and PdGa. Our work establishes a fundamental nonlinear response in spin transport, and opens the door to exploring spintronic applications based on nonlinear spin Hall effect. △ Less

Submitted 25 July, 2024; originally announced July 2024.

Active Loop Closure for OSM-guided Robotic Mapping in Large-Scale Urban Environments

Authors: Wei Gao, Zezhou Sun, Mingle Zhao, Cheng-Zhong Xu, Hui Kong

Abstract: The autonomous mapping of large-scale urban scenes presents significant challenges for autonomous robots. To mitigate the challenges, global planning, such as utilizing prior GPS trajectories from OpenStreetMap (OSM), is often used to guide the autonomous navigation of robots for mapping. However, due to factors like complex terrain, unexpected body movement, and sensor noise, the uncertainty of t… ▽ More The autonomous mapping of large-scale urban scenes presents significant challenges for autonomous robots. To mitigate the challenges, global planning, such as utilizing prior GPS trajectories from OpenStreetMap (OSM), is often used to guide the autonomous navigation of robots for mapping. However, due to factors like complex terrain, unexpected body movement, and sensor noise, the uncertainty of the robot's pose estimates inevitably increases over time, ultimately leading to the failure of robotic mapping. To address this issue, we propose a novel active loop closure procedure, enabling the robot to actively re-plan the previously planned GPS trajectory. The method can guide the robot to re-visit the previous places where the loop-closure detection can be performed to trigger the back-end optimization, effectively reducing errors and uncertainties in pose estimation. The proposed active loop closure mechanism is implemented and embedded into a real-time OSM-guided robot mapping framework. Empirical results on several large-scale outdoor scenarios demonstrate its effectiveness and promising performance. △ Less

Submitted 24 July, 2024; originally announced July 2024.

Crystals with Transformers on Graphs, for Prediction of Unconventional Crystal Material Properties and the Benchmark

Authors: Hongyi Wang, Ji Sun, Jinzhe Liang, Li Zhai, Zitian Tang, Zijian Li, Wei Zhai, Xusheng Wang, Weihao Gao, Sheng Gong, Bolong Huang, Hua Zhang

Abstract: The ionic bonding across the lattice and ordered microscopic structures endow crystals with unique symmetry and determine their macroscopic properties. Unconventional crystals, in particular, exhibit non-traditional lattice structures or possess exotic physical properties, making them intriguing subjects for investigation. Therefore, to accurately predict the physical and chemical properties of cr… ▽ More The ionic bonding across the lattice and ordered microscopic structures endow crystals with unique symmetry and determine their macroscopic properties. Unconventional crystals, in particular, exhibit non-traditional lattice structures or possess exotic physical properties, making them intriguing subjects for investigation. Therefore, to accurately predict the physical and chemical properties of crystals, it is crucial to consider long-range orders. While GNN excels at capturing the local environment of atoms in crystals, they often face challenges in effectively capturing longer-ranged interactions due to their limited depth. In this paper, we propose CrysToGraph ($\textbf{Crys}$tals with $\textbf{T}$ransformers $\textbf{o}$n $\textbf{Graph}$s), a novel transformer-based geometric graph network designed specifically for unconventional crystalline systems, and UnconvBench, a comprehensive benchmark to evaluate models' predictive performance on unconventional crystal materials such as defected crystals, low-dimension crystals and MOF. CrysToGraph effectively captures short-range interactions with transformer-based graph convolution blocks as well as long-range interactions with graph-wise transformer blocks. CrysToGraph proofs its effectiveness in modelling unconventional crystal materials in multiple tasks, and moreover, it outperforms most existing methods, achieving new state-of-the-art results on the benchmarks of both unconventional crystals and traditional crystals. △ Less

Submitted 22 July, 2024; originally announced July 2024.

D$^4$M: Dataset Distillation via Disentangled Diffusion Model

Authors: Duo Su, Junjie Hou, Weizhi Gao, Yingjie Tian, Bowen Tang

Abstract: Dataset distillation offers a lightweight synthetic dataset for fast network training with promising test accuracy. To imitate the performance of the original dataset, most approaches employ bi-level optimization and the distillation space relies on the matching architecture. Nevertheless, these approaches either suffer significant computational costs on large-scale datasets or experience performa… ▽ More Dataset distillation offers a lightweight synthetic dataset for fast network training with promising test accuracy. To imitate the performance of the original dataset, most approaches employ bi-level optimization and the distillation space relies on the matching architecture. Nevertheless, these approaches either suffer significant computational costs on large-scale datasets or experience performance decline on cross-architectures. We advocate for designing an economical dataset distillation framework that is independent of the matching architectures. With empirical observations, we argue that constraining the consistency of the real and synthetic image spaces will enhance the cross-architecture generalization. Motivated by this, we introduce Dataset Distillation via Disentangled Diffusion Model (D$^4$M), an efficient framework for dataset distillation. Compared to architecture-dependent methods, D$^4$M employs latent diffusion model to guarantee consistency and incorporates label information into category prototypes. The distilled datasets are versatile, eliminating the need for repeated generation of distinct datasets for various architectures. Through comprehensive experiments, D$^4$M demonstrates superior performance and robust generalization, surpassing the SOTA methods across most aspects. △ Less

Submitted 21 July, 2024; originally announced July 2024.

Comments: Accepted to CVPR 2024

Intelligent Artistic Typography: A Comprehensive Review of Artistic Text Design and Generation

Authors: Yuhang Bai, Zichuan Huang, Wenshuo Gao, Shuai Yang, Jiaying Liu

Abstract: Artistic text generation aims to amplify the aesthetic qualities of text while maintaining readability. It can make the text more attractive and better convey its expression, thus enjoying a wide range of application scenarios such as social media display, consumer electronics, fashion, and graphic design. Artistic text generation includes artistic text stylization and semantic typography. Artisti… ▽ More Artistic text generation aims to amplify the aesthetic qualities of text while maintaining readability. It can make the text more attractive and better convey its expression, thus enjoying a wide range of application scenarios such as social media display, consumer electronics, fashion, and graphic design. Artistic text generation includes artistic text stylization and semantic typography. Artistic text stylization concentrates on the text effect overlaid upon the text, such as shadows, outlines, colors, glows, and textures. By comparison, semantic typography focuses on the deformation of the characters to strengthen their visual representation by mimicking the semantic understanding within the text. This overview paper provides an introduction to both artistic text stylization and semantic typography, including the taxonomy, the key ideas of representative methods, and the applications in static and dynamic artistic text generation. Furthermore, the dataset and evaluation metrics are introduced, and the future directions of artistic text generation are discussed. A comprehensive list of artistic text generation models studied in this review is available at https://github.com/williamyang1991/Awesome-Artistic-Typography/. △ Less

Submitted 20 July, 2024; originally announced July 2024.

Comments: GitHub Page: https://github.com/williamyang1991/Awesome-Artistic-Typography/

Cluster Sliding Ferroelectricity in Trilayer Quasi-Hexagonal C60

Authors: Xuefei Wang, Yanhan Ren, Shi Qiu, Fan Zhang, Xueao Li, Junfeng Gao, Weiwei Gao, Jijun Zhao

Abstract: Electric polarization typically originates from non-centrosymmetric charge distributions. Since chemical bonds between atoms of the same elements favor centrosymmetric crystal structures and symmetrically distributed electron charges, elemental ferroelectrics are extremely rare. In comparison to atoms, elemental clusters are less symmetric and typically have various preferred orientations in cryst… ▽ More Electric polarization typically originates from non-centrosymmetric charge distributions. Since chemical bonds between atoms of the same elements favor centrosymmetric crystal structures and symmetrically distributed electron charges, elemental ferroelectrics are extremely rare. In comparison to atoms, elemental clusters are less symmetric and typically have various preferred orientations in crystals. Consequently, the assembly of clusters with different orientations tends to break the inversion symmetry. Based on this concept, we show that sliding ferroelectricity naturally emerges in trilayer quasi-hexagonal phase (qHP) C60, a cluster-assembled carbon allotrope recently synthesized. Trilayer qHP C60's have several stable polar structures, which are distinguishable in second-harmonic generation (SHG) responses. Compared to previously found elemental ferroelectrics, trilayer qHP C60's have sizable band gaps and some of them have both switchable out-of-plane and in-plane polarizations. Remarkably, the out-of-plane and in-plane polarizations are decoupled, enabling an easy-to-implement construction of Van der Waals homostructures with ferroelectrically switchable chirality. △ Less

Submitted 18 July, 2024; originally announced July 2024.

Comments: 5 figures

Observation of Ferromagnetic Phase in the Second Moiré Band of Twisted MoTe2

Authors: Liheng An, Haiyang Pan, Wen-Xuan Qiu, Naizhou Wang, Shihao Ru, Qinghai Tan, Xuran Dai, Xiangbin Cai, Qiuyu Shang, Xiufang Lu, Hao Jiang, Xiaodan Lyu, Kenji Watanabe, Takashi Taniguchi, Fengcheng Wu, Wei-bo Gao

Abstract: Flat bands and electron correlation in moiré lattices give rise to many exotic phases, including Mott insulators, superconductivity, and topological states. Within the first moiré band, integer and fractional quantum anomalous Hall effects have been observed in twisted bilayer MoTe2 (tMoTe2) at one hole doping and fractional doping per moiré unit cell, respectively. When the second moiré band is f… ▽ More Flat bands and electron correlation in moiré lattices give rise to many exotic phases, including Mott insulators, superconductivity, and topological states. Within the first moiré band, integer and fractional quantum anomalous Hall effects have been observed in twisted bilayer MoTe2 (tMoTe2) at one hole doping and fractional doping per moiré unit cell, respectively. When the second moiré band is fully hole doped, quantum spin Hall insulator has also been reported in tMoTe2 at a certain twist angle. Exotic topological states together with ferromagnetic (FM) states in the high moiré band can potentially exist as well. In this study, we report the observation of a FM phase in the second moiré band in tMoTe2. The FM phase can be tuned by both the doping level and displacement field. At filling around 2.58 holes per moiré unit cell, the FM phase reaches a Curie temperature of 3.5 K. A large displacement field can suppress the FM phase, like the FM phase at the filling of -1. Our results demonstrate the realization of time-reversal symmetry-breaking states in the higher moiré bands in tMoTe2. △ Less

Submitted 18 July, 2024; originally announced July 2024.

Comments: Main text: 13 pages, 5 figures. Supplementary: 11 pages, 15 figures

Stream State-tying for Sign Language Recognition

Authors: Jiyong Ma, Wen Gao, Chunli Wang

Abstract: In this paper, a novel approach to sign language recognition based on state tying in each of data streams is presented. In this framework, it is assumed that hand gesture signal is represented in terms of six synchronous data streams, i.e., the left/right hand position, left/right hand orientation and left/right handshape. This approach offers a very accurate representation of the sign space and k… ▽ More In this paper, a novel approach to sign language recognition based on state tying in each of data streams is presented. In this framework, it is assumed that hand gesture signal is represented in terms of six synchronous data streams, i.e., the left/right hand position, left/right hand orientation and left/right handshape. This approach offers a very accurate representation of the sign space and keeps the number of parameters reasonably small in favor of a fast decoding. Experiments were carried out for 5177 Chinese signs. The real time isolated recognition rate is 94.8%. For continuous sign recognition, the word correct rate is 91.4%. Keywords: Sign language recognition; Automatic sign language translation; Hand gesture recognition; Hidden Markov models; State-tying; Multimodal user interface; Virtual reality; Man-machine systems. △ Less

Submitted 21 April, 2024; originally announced July 2024.

SACNet: A Spatially Adaptive Convolution Network for 2D Multi-organ Medical Segmentation

Authors: Lin Zhang, Wenbo Gao, Jie Yi, Yunyun Yang

Abstract: Multi-organ segmentation in medical image analysis is crucial for diagnosis and treatment planning. However, many factors complicate the task, including variability in different target categories and interference from complex backgrounds. In this paper, we utilize the knowledge of Deformable Convolution V3 (DCNv3) and multi-object segmentation to optimize our Spatially Adaptive Convolution Network… ▽ More Multi-organ segmentation in medical image analysis is crucial for diagnosis and treatment planning. However, many factors complicate the task, including variability in different target categories and interference from complex backgrounds. In this paper, we utilize the knowledge of Deformable Convolution V3 (DCNv3) and multi-object segmentation to optimize our Spatially Adaptive Convolution Network (SACNet) in three aspects: feature extraction, model architecture, and loss constraint, simultaneously enhancing the perception of different segmentation targets. Firstly, we propose the Adaptive Receptive Field Module (ARFM), which combines DCNv3 with a series of customized block-level and architecture-level designs similar to transformers. This module can capture the unique features of different organs by adaptively adjusting the receptive field according to various targets. Secondly, we utilize ARFM as building blocks to construct the encoder-decoder of SACNet and partially share parameters between the encoder and decoder, making the network wider rather than deeper. This design achieves a shared lightweight decoder and a more parameter-efficient and effective framework. Lastly, we propose a novel continuity dynamic adjustment loss function, based on t-vMF dice loss and cross-entropy loss, to better balance easy and complex classes in segmentation. Experiments on 3D slice datasets from ACDC and Synapse demonstrate that SACNet delivers superior segmentation performance in multi-organ segmentation tasks compared to several existing methods. △ Less

Submitted 14 July, 2024; originally announced July 2024.

RBMD: A molecular dynamics package enabling to simulate 10 million all-atom particles in a single graphics processing unit

Authors: Weihang Gao, Teng Zhao, Yongfa Guo, Jiuyang Liang, Huan Liu, Maoying Luo, Zedong Luo, Wei Qin, Yichao Wang, Qi Zhou, Shi Jin, Zhenli Xu

Abstract: This paper introduces a random-batch molecular dynamics (RBMD) package for fast simulations of particle systems at the nano/micro scale. Different from existing packages, the RBMD uses random batch methods for nonbonded interactions of particle systems. The long-range part of Coulomb interactions is calculated in Fourier space by the random batch Ewald algorithm, which achieves linear complexity a… ▽ More This paper introduces a random-batch molecular dynamics (RBMD) package for fast simulations of particle systems at the nano/micro scale. Different from existing packages, the RBMD uses random batch methods for nonbonded interactions of particle systems. The long-range part of Coulomb interactions is calculated in Fourier space by the random batch Ewald algorithm, which achieves linear complexity and superscalability, surpassing classical lattice-based Ewald methods. For the short-range part, the random batch list algorithm is used to construct neighbor lists, significantly reducing both computational and memory costs. The RBMD is implemented on GPU-CPU heterogeneous architectures, with classical force fields for all-atom systems. Benchmark systems are used to validate accuracy and performance of the package. Comparison with the particle-particle particle-mesh method and the Verlet list method in the LAMMPS package is performed on three different NVIDIA GPUs, demonstrating high efficiency of the RBMD on heterogeneous architectures. Our results also show that the RBMD enables simulations on a single GPU with a CPU core up to 10 million particles. Typically, for systems of one million particles, the RBMD allows simulating all-atom systems with a high efficiency of 8.20 ms per step, demonstrating the attractive feature for running large-scale simulations of practical applications on a desktop machine. △ Less

Submitted 22 August, 2024; v1 submitted 12 July, 2024; originally announced July 2024.

Comments: 26 pages, 8 figures

Power Optimization and Deep Learning for Channel Estimation of Active IRS-Aided IoT

Authors: Yan Wang, Feng Shu, Rongen Dong, Wei Gao, Qi Zhang, Jiajia Liu

Abstract: In this paper, channel estimation of an active intelligent reflecting surface (IRS) aided uplink Internet of Things (IoT) network is investigated. Firstly, the least square (LS) estimators for the direct channel and the cascaded channel are presented, respectively. The corresponding mean square errors (MSE) of channel estimators are derived. Subsequently, in order to evaluate the influence of adju… ▽ More In this paper, channel estimation of an active intelligent reflecting surface (IRS) aided uplink Internet of Things (IoT) network is investigated. Firstly, the least square (LS) estimators for the direct channel and the cascaded channel are presented, respectively. The corresponding mean square errors (MSE) of channel estimators are derived. Subsequently, in order to evaluate the influence of adjusting the transmit power at the IoT devices or the reflected power at the active IRS on Sum-MSE performance, two situations are considered. In the first case, under the total power sum constraint of the IoT devices and active IRS, the closed-form expression of the optimal power allocation factor is derived. In the second case, when the transmit power at the IoT devices is fixed, there exists an optimal reflective power at active IRS. To further improve the estimation performance, the convolutional neural network (CNN)-based direct channel estimation (CDCE) algorithm and the CNN-based cascaded channel estimation (CCCE) algorithm are designed. Finally, simulation results demonstrate the existence of an optimal power allocation strategy that minimizes the Sum-MSE, and further validate the superiority of the proposed CDCE / CCCE algorithms over their respective traditional LS and minimum mean square error (MMSE) baselines. △ Less

Submitted 15 July, 2024; v1 submitted 12 July, 2024; originally announced July 2024.

Toward Efficient Deep Spiking Neuron Networks:A Survey On Compression

Authors: Hui Xie, Ge Yang, Wenjuan Gao

Abstract: With the rapid development of deep learning, Deep Spiking Neural Networks (DSNNs) have emerged as promising due to their unique spike event processing and asynchronous computation. When deployed on neuromorphic chips, DSNNs offer significant power advantages over Deep Artificial Neural Networks (DANNs) and eliminate time and energy consuming multiplications due to the binary nature of spikes (0 or… ▽ More With the rapid development of deep learning, Deep Spiking Neural Networks (DSNNs) have emerged as promising due to their unique spike event processing and asynchronous computation. When deployed on neuromorphic chips, DSNNs offer significant power advantages over Deep Artificial Neural Networks (DANNs) and eliminate time and energy consuming multiplications due to the binary nature of spikes (0 or 1). Additionally, DSNNs excel in processing temporal information, making them potentially superior for handling temporal data compared to DANNs. However, their deep network structure and numerous parameters result in high computational costs and energy consumption, limiting real-life deployment. To enhance DSNNs efficiency, researchers have adapted methods from DANNs, such as pruning, quantization, and knowledge distillation, and developed specific techniques like reducing spike firing and pruning time steps. While previous surveys have covered DSNNs algorithms, hardware deployment, and general overviews, focused research on DSNNs compression and efficiency has been lacking. This survey addresses this gap by concentrating on efficient DSNNs and their compression methods. It begins with an exploration of DSNNs' biological background and computational units, highlighting differences from DANNs. It then delves into various compression methods, including pruning, quantization, knowledge distillation, and reducing spike firing, and concludes with suggestions for future research directions. △ Less

Submitted 3 June, 2024; originally announced July 2024.

Establishing Rigorous and Cost-effective Clinical Trials for Artificial Intelligence Models

Authors: Wanling Gao, Yunyou Huang, Dandan Cui, Zhuoming Yu, Wenjing Liu, Xiaoshuang Liang, Jiahui Zhao, Jiyue Xie, Hao Li, Li Ma, Ning Ye, Yumiao Kang, Dingfeng Luo, Peng Pan, Wei Huang, Zhongmou Liu, Jizhong Hu, Gangyuan Zhao, Chongrong Jiang, Fan Huang, Tianyi Wei, Suqin Tang, Bingjie Xia, Zhifei Zhang, Jianfeng Zhan

Abstract: A profound gap persists between artificial intelligence (AI) and clinical practice in medicine, primarily due to the lack of rigorous and cost-effective evaluation methodologies. State-of-the-art and state-of-the-practice AI model evaluations are limited to laboratory studies on medical datasets or direct clinical trials with no or solely patient-centered controls. Moreover, the crucial role of cl… ▽ More A profound gap persists between artificial intelligence (AI) and clinical practice in medicine, primarily due to the lack of rigorous and cost-effective evaluation methodologies. State-of-the-art and state-of-the-practice AI model evaluations are limited to laboratory studies on medical datasets or direct clinical trials with no or solely patient-centered controls. Moreover, the crucial role of clinicians in collaborating with AI, pivotal for determining its impact on clinical practice, is often overlooked. For the first time, we emphasize the critical necessity for rigorous and cost-effective evaluation methodologies for AI models in clinical practice, featuring patient/clinician-centered (dual-centered) AI randomized controlled trials (DC-AI RCTs) and virtual clinician-based in-silico trials (VC-MedAI) as an effective proxy for DC-AI RCTs. Leveraging 7500 diagnosis records from two-step inaugural DC-AI RCTs across 14 medical centers with 125 clinicians, our results demonstrate the necessity of DC-AI RCTs and the effectiveness of VC-MedAI. Notably, VC-MedAI performs comparably to human clinicians, replicating insights and conclusions from prospective DC-AI RCTs. We envision DC-AI RCTs and VC-MedAI as pivotal advancements, presenting innovative and transformative evaluation methodologies for AI models in clinical practice, offering a preclinical-like setting mirroring conventional medicine, and reshaping development paradigms in a cost-effective and fast-iterative manner. Chinese Clinical Trial Registration: ChiCTR2400086816. △ Less

Submitted 28 July, 2024; v1 submitted 11 July, 2024; originally announced July 2024.

Comments: 24 pages

Understanding is Compression

Authors: Ziguang Li, Chao Huang, Xuliang Wang, Haibo Hu, Cole Wyeth, Dongbo Bu, Quan Yu, Wen Gao, Xingwu Liu, Ming Li

Abstract: Modern data compression methods are slowly reaching their limits after 80 years of research, millions of papers, and wide range of applications. Yet, the extravagant 6G communication speed requirement raises a major open question for revolutionary new ideas of data compression. We have previously shown all understanding or learning are compression, under reasonable assumptions. Large language mo… ▽ More Modern data compression methods are slowly reaching their limits after 80 years of research, millions of papers, and wide range of applications. Yet, the extravagant 6G communication speed requirement raises a major open question for revolutionary new ideas of data compression. We have previously shown all understanding or learning are compression, under reasonable assumptions. Large language models (LLMs) understand data better than ever before. Can they help us to compress data? The LLMs may be seen to approximate the uncomputable Solomonoff induction. Therefore, under this new uncomputable paradigm, we present LMCompress. LMCompress shatters all previous lossless compression algorithms, doubling the lossless compression ratios of JPEG-XL for images, FLAC for audios, and H.264 for videos, and quadrupling the compression ratio of bz2 for texts. The better a large model understands the data, the better LMCompress compresses. △ Less

Submitted 20 August, 2024; v1 submitted 23 June, 2024; originally announced July 2024.

Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI

Authors: Yang Liu, Weixing Chen, Yongjie Bai, Xiaodan Liang, Guanbin Li, Wen Gao, Liang Lin

Abstract: Embodied Artificial Intelligence (Embodied AI) is crucial for achieving Artificial General Intelligence (AGI) and serves as a foundation for various applications that bridge cyberspace and the physical world. Recently, the emergence of Multi-modal Large Models (MLMs) and World Models (WMs) have attracted significant attention due to their remarkable perception, interaction, and reasoning capabilit… ▽ More Embodied Artificial Intelligence (Embodied AI) is crucial for achieving Artificial General Intelligence (AGI) and serves as a foundation for various applications that bridge cyberspace and the physical world. Recently, the emergence of Multi-modal Large Models (MLMs) and World Models (WMs) have attracted significant attention due to their remarkable perception, interaction, and reasoning capabilities, making them a promising architecture for the brain of embodied agents. However, there is no comprehensive survey for Embodied AI in the era of MLMs. In this survey, we give a comprehensive exploration of the latest advancements in Embodied AI. Our analysis firstly navigates through the forefront of representative works of embodied robots and simulators, to fully understand the research focuses and their limitations. Then, we analyze four main research targets: 1) embodied perception, 2) embodied interaction, 3) embodied agent, and 4) sim-to-real adaptation, covering the state-of-the-art methods, essential paradigms, and comprehensive datasets. Additionally, we explore the complexities of MLMs in virtual and real embodied agents, highlighting their significance in facilitating interactions in dynamic digital and physical environments. Finally, we summarize the challenges and limitations of embodied AI and discuss their potential future directions. We hope this survey will serve as a foundational reference for the research community and inspire continued innovation. The associated project can be found at https://github.com/HCPLab-SYSU/Embodied_AI_Paper_List. △ Less

Submitted 25 August, 2024; v1 submitted 9 July, 2024; originally announced July 2024.

Comments: The first comprehensive review of Embodied AI in the era of MLMs, 39 pages. We also provide the paper list for Embodied AI: https://github.com/HCPLab-SYSU/Embodied_AI_Paper_List

Double-Ended Synthesis Planning with Goal-Constrained Bidirectional Search

Authors: Kevin Yu, Jihye Roh, Ziang Li, Wenhao Gao, Runzhong Wang, Connor W. Coley

Abstract: Computer-aided synthesis planning (CASP) algorithms have demonstrated expert-level abilities in planning retrosynthetic routes to molecules of low to moderate complexity. However, current search methods assume the sufficiency of reaching arbitrary building blocks, failing to address the common real-world constraint where using specific molecules is desired. To this end, we present a formulation of… ▽ More Computer-aided synthesis planning (CASP) algorithms have demonstrated expert-level abilities in planning retrosynthetic routes to molecules of low to moderate complexity. However, current search methods assume the sufficiency of reaching arbitrary building blocks, failing to address the common real-world constraint where using specific molecules is desired. To this end, we present a formulation of synthesis planning with starting material constraints. Under this formulation, we propose Double-Ended Synthesis Planning (DESP), a novel CASP algorithm under a bidirectional graph search scheme that interleaves expansions from the target and from the goal starting materials to ensure constraint satisfiability. The search algorithm is guided by a goal-conditioned cost network learned offline from a partially observed hypergraph of valid chemical reactions. We demonstrate the utility of DESP in improving solve rates and reducing the number of search expansions by biasing synthesis planning towards expert goals on multiple new benchmarks. DESP can make use of existing one-step retrosynthesis models, and we anticipate its performance to scale as these one-step model capabilities improve. △ Less

Submitted 8 July, 2024; originally announced July 2024.

Comments: 10 pages main, 4 figures

PCAC-GAN: A Sparse-Tensor-Based Generative Adversarial Network for 3D Point Cloud Attribute Compression

Authors: Xiaolong Mao, Hui Yuan, Xin Lu, Raouf Hamzaoui, Wei Gao

Abstract: Learning-based methods have proven successful in compressing geometric information for point clouds. For attribute compression, however, they still lag behind non-learning-based methods such as the MPEG G-PCC standard. To bridge this gap, we propose a novel deep learning-based point cloud attribute compression method that uses a generative adversarial network (GAN) with sparse convolution layers.… ▽ More Learning-based methods have proven successful in compressing geometric information for point clouds. For attribute compression, however, they still lag behind non-learning-based methods such as the MPEG G-PCC standard. To bridge this gap, we propose a novel deep learning-based point cloud attribute compression method that uses a generative adversarial network (GAN) with sparse convolution layers. Our method also includes a module that adaptively selects the resolution of the voxels used to voxelize the input point cloud. Sparse vectors are used to represent the voxelized point cloud, and sparse convolutions process the sparse tensors, ensuring computational efficiency. To the best of our knowledge, this is the first application of GANs to compress point cloud attributes. Our experimental results show that our method outperforms existing learning-based techniques and rivals the latest G-PCC test model (TMC13v23) in terms of visual quality. △ Less

Submitted 19 July, 2024; v1 submitted 8 July, 2024; originally announced July 2024.

Comments: 14 pages, 5 figures, Accepted by Computational Visual Media

MSC Class: 94J20 ACM Class: I.4.2

Journal ref: Computational Visual Media, 2024

A Survey of Models for Cognitive Diagnosis: New Developments and Future Directions

Authors: Fei Wang, Weibo Gao, Qi Liu, Jiatong Li, Guanhao Zhao, Zheng Zhang, Zhenya Huang, Mengxiao Zhu, Shijin Wang, Wei Tong, Enhong Chen

Abstract: Cognitive diagnosis has been developed for decades as an effective measurement tool to evaluate human cognitive status such as ability level and knowledge mastery. It has been applied to a wide range of fields including education, sport, psychological diagnosis, etc. By providing better awareness of cognitive status, it can serve as the basis for personalized services such as well-designed medical… ▽ More Cognitive diagnosis has been developed for decades as an effective measurement tool to evaluate human cognitive status such as ability level and knowledge mastery. It has been applied to a wide range of fields including education, sport, psychological diagnosis, etc. By providing better awareness of cognitive status, it can serve as the basis for personalized services such as well-designed medical treatment, teaching strategy and vocational training. This paper aims to provide a survey of current models for cognitive diagnosis, with more attention on new developments using machine learning-based methods. By comparing the model structures, parameter estimation algorithms, model evaluation methods and applications, we provide a relatively comprehensive review of the recent trends in cognitive diagnosis models. Further, we discuss future directions that are worthy of exploration. In addition, we release two Python libraries: EduData for easy access to some relevant public datasets we have collected, and EduCDM that implements popular CDMs to facilitate both applications and research purposes. △ Less

Submitted 7 July, 2024; originally announced July 2024.

Giant Second Harmonic Generation from Wafer-Scale Aligned Chiral Carbon Nanotubes

Authors: Rui Xu, Jacques Doumani, Viktor Labuntsov, Nina Hong, Anna-Christina Samaha, Weiran Tu, Fuyang Tay, Elizabeth Blackert, Jiaming Luo, Mario El Tahchi, Weilu Gao, Jun Lou, Yohei Yomogida, Kazuhiro Yanagi, Riichiro Saito, Vasili Perebeinos, Andrey Baydin, Junichiro Kono, Hanyu Zhu

Abstract: Chiral carbon nanotubes (CNTs) are direct-gap semiconductors with optical properties governed by one-dimensional excitons with enormous oscillator strengths. Each species of chiral CNTs has an enantiomeric pair of left- and right-handed CNTs with nearly identical properties, but enantiomer-dependent phenomena can emerge, especially in nonlinear optical processes. Theoretical studies have predicted… ▽ More Chiral carbon nanotubes (CNTs) are direct-gap semiconductors with optical properties governed by one-dimensional excitons with enormous oscillator strengths. Each species of chiral CNTs has an enantiomeric pair of left- and right-handed CNTs with nearly identical properties, but enantiomer-dependent phenomena can emerge, especially in nonlinear optical processes. Theoretical studies have predicted strong second-order nonlinearities for chiral CNTs, but there has been no experimental verification due to the lack of macroscopically ordered assemblies of single-enantiomer chiral CNTs. Here for the first time, we report the synthesis of centimeter-scale films of densely packed and aligned single-enantiomer chiral CNTs that exhibit micro-fabrication compatibility. We observe giant second harmonic generation (SHG) emission from the chiral CNT film, which originates from the intrinsic chirality and inversion symmetry breaking of the atomic structure of chiral CNTs. The observed value of the dominant element of the second-order nonlinear optical susceptibility tensor reaches $1.5\times 10^{3}$ pm/V at a pump wavelength of 1030 nm, corresponding to the lowest-energy excitonic resonance. Our calculations based on many-body theory correctly estimate the spectrum and magnitude of such excitonically enhanced optical nonlinearity. These results are promising for developing scalable chiral-CNT electronics, nonlinear photonics and photonic quantum computing. △ Less

Submitted 5 July, 2024; originally announced July 2024.

Benchmarking Complex Instruction-Following with Multiple Constraints Composition

Authors: Bosi Wen, Pei Ke, Xiaotao Gu, Lindong Wu, Hao Huang, Jinfeng Zhou, Wenchuang Li, Binxin Hu, Wendy Gao, Jiaxin Xu, Yiming Liu, Jie Tang, Hongning Wang, Minlie Huang

Abstract: Instruction following is one of the fundamental capabilities of large language models (LLMs). As the ability of LLMs is constantly improving, they have been increasingly applied to deal with complex human instructions in real-world scenarios. Therefore, how to evaluate the ability of complex instruction-following of LLMs has become a critical research problem. Existing benchmarks mainly focus on m… ▽ More Instruction following is one of the fundamental capabilities of large language models (LLMs). As the ability of LLMs is constantly improving, they have been increasingly applied to deal with complex human instructions in real-world scenarios. Therefore, how to evaluate the ability of complex instruction-following of LLMs has become a critical research problem. Existing benchmarks mainly focus on modeling different types of constraints in human instructions while neglecting the composition of different constraints, which is an indispensable constituent in complex instructions. To this end, we propose ComplexBench, a benchmark for comprehensively evaluating the ability of LLMs to follow complex instructions composed of multiple constraints. We propose a hierarchical taxonomy for complex instructions, including 4 constraint types, 19 constraint dimensions, and 4 composition types, and manually collect a high-quality dataset accordingly. To make the evaluation reliable, we augment LLM-based evaluators with rules to effectively verify whether generated texts can satisfy each constraint and composition. Furthermore, we obtain the final evaluation score based on the dependency structure determined by different composition types. ComplexBench identifies significant deficiencies in existing LLMs when dealing with complex instructions with multiple constraints composition. △ Less

Submitted 11 July, 2024; v1 submitted 4 July, 2024; originally announced July 2024.

Comments: 20 pages, 7 figures

Prediction-Free Coordinated Dispatch of Microgrid: A Data-Driven Online Optimization Approach

Authors: Kaidi Huang, Lin Cheng, Ning Qi, David Wenzhong Gao

Abstract: The integration of renewable energy sources (RES) into microgrids poses challenges to reliable and economic operation due to the inherent uncertainty and volatility of RES. Contrary to the previous dispatch methods that require precise predictions of RES, this paper proposes a novel prediction-free and data-driven coordinated dispatch framework for reliable microgrid operations. In the offline sta… ▽ More The integration of renewable energy sources (RES) into microgrids poses challenges to reliable and economic operation due to the inherent uncertainty and volatility of RES. Contrary to the previous dispatch methods that require precise predictions of RES, this paper proposes a novel prediction-free and data-driven coordinated dispatch framework for reliable microgrid operations. In the offline stage, ex-post optimal dispatch sequences are generated based on historical dispatch from "God' s-eye view". The sequences offer a global reference and are sequentially updated based on the newly observed data. Subsequently, we propose an adaptive virtual-queue-based online convex optimization (OCO) method to generate the real-time control policy of microgrid, which aim to minimize the instant operation cost while tracking the offline reference. We provide theoretical proof that the proposed method outperforms the existing OCO methods and admits sublinear dynamic regret bound and sublinear hard cumulative constraint violation bound for OCO with time-varying constraints. Case study illustrates that the proposed method outperforms state-of-the-art methods in terms of economic optimality, computational efficiency, and security. △ Less

Submitted 4 July, 2024; originally announced July 2024.

IntentionNet: Map-Lite Visual Navigation at the Kilometre Scale

Authors: Wei Gao, Bo Ai, Joel Loo, Vinay, David Hsu

Abstract: This work explores the challenges of creating a scalable and robust robot navigation system that can traverse both indoor and outdoor environments to reach distant goals. We propose a navigation system architecture called IntentionNet that employs a monolithic neural network as the low-level planner/controller, and uses a general interface that we call intentions to steer the controller. The paper… ▽ More This work explores the challenges of creating a scalable and robust robot navigation system that can traverse both indoor and outdoor environments to reach distant goals. We propose a navigation system architecture called IntentionNet that employs a monolithic neural network as the low-level planner/controller, and uses a general interface that we call intentions to steer the controller. The paper proposes two types of intentions, Local Path and Environment (LPE) and Discretised Local Move (DLM), and shows that DLM is robust to significant metric positioning and mapping errors. The paper also presents Kilo-IntentionNet, an instance of the IntentionNet system using the DLM intention that is deployed on a Boston Dynamics Spot robot, and which successfully navigates through complex indoor and outdoor environments over distances of up to a kilometre with only noisy odometry. △ Less

Submitted 3 July, 2024; originally announced July 2024.

Dielectric Fano Nanoantennas for Enabling Sub-Nanosecond Lifetimes in NV-based Single Photon Emitters

Authors: Shu An, Dmitry Kalashnikov, Wenqiao Shi, Zackaria Mahfoud, Ah Bian Chew, Yan Liu, Jing Wu, Di Zhu, Weibo Gao, Cheng-Wei Qiu, Victor Leong, Zhaogang Dong

Abstract: Solid-state quantum emitters are essential sources of single photons, and enhancing their emission rates is of paramount importance for applications in quantum communications, computing, and metrology. One approach is to couple quantum emitters with resonant photonic nanostructures, where the emission rate is enhanced due to the Purcell effect. Dielectric nanoantennas are promising as they provide… ▽ More Solid-state quantum emitters are essential sources of single photons, and enhancing their emission rates is of paramount importance for applications in quantum communications, computing, and metrology. One approach is to couple quantum emitters with resonant photonic nanostructures, where the emission rate is enhanced due to the Purcell effect. Dielectric nanoantennas are promising as they provide strong emission enhancement compared to plasmonic ones, which suffer from high Ohmic loss. Here, we designed and fabricated a dielectric Fano resonator based on a pair of silicon (Si) ellipses and a disk, which supports the mode hybridization between quasi-bound-states-in-the-continuum (quasi-BIC) and Mie resonance. We demonstrated the performance of the developed resonant system by interfacing it with single photon emitters (SPEs) based on nitrogen-vacancy (NV-) centers in nanodiamonds (NDs). We observed that the interfaced emitters have a Purcell enhancement factor of ~10, with sub-ns emission lifetime and a polarization contrast of 9. Our results indicate a promising method for developing efficient and compact single-photon sources for integrated quantum photonics applications. △ Less

Submitted 3 July, 2024; originally announced July 2024.

Comments: 20 pages, 4 figures

Learning Robust 3D Representation from CLIP via Dual Denoising

Authors: Shuqing Luo, Bowen Qu, Wei Gao

Abstract: In this paper, we explore a critical yet under-investigated issue: how to learn robust and well-generalized 3D representation from pre-trained vision language models such as CLIP. Previous works have demonstrated that cross-modal distillation can provide rich and useful knowledge for 3D data. However, like most deep learning models, the resultant 3D learning network is still vulnerable to adversar… ▽ More In this paper, we explore a critical yet under-investigated issue: how to learn robust and well-generalized 3D representation from pre-trained vision language models such as CLIP. Previous works have demonstrated that cross-modal distillation can provide rich and useful knowledge for 3D data. However, like most deep learning models, the resultant 3D learning network is still vulnerable to adversarial attacks especially the iterative attack. In this work, we propose Dual Denoising, a novel framework for learning robust and well-generalized 3D representations from CLIP. It combines a denoising-based proxy task with a novel feature denoising network for 3D pre-training. Additionally, we propose utilizing parallel noise inference to enhance the generalization of point cloud features under cross domain settings. Experiments show that our model can effectively improve the representation learning performance and adversarial robustness of the 3D learning network under zero-shot settings without adversarial training. Our code is available at https://github.com/luoshuqing2001/Dual_Denoising. △ Less

Submitted 30 June, 2024; originally announced July 2024.

Efficient Evolutionary Search Over Chemical Space with Large Language Models

Authors: Haorui Wang, Marta Skreta, Cher-Tian Ser, Wenhao Gao, Lingkai Kong, Felix Strieth-Kalthoff, Chenru Duan, Yuchen Zhuang, Yue Yu, Yanqiao Zhu, Yuanqi Du, Alán Aspuru-Guzik, Kirill Neklyudov, Chao Zhang

Abstract: Molecular discovery, when formulated as an optimization problem, presents significant computational challenges because optimization objectives can be non-differentiable. Evolutionary Algorithms (EAs), often used to optimize black-box objectives in molecular discovery, traverse chemical space by performing random mutations and crossovers, leading to a large number of expensive objective evaluations… ▽ More Molecular discovery, when formulated as an optimization problem, presents significant computational challenges because optimization objectives can be non-differentiable. Evolutionary Algorithms (EAs), often used to optimize black-box objectives in molecular discovery, traverse chemical space by performing random mutations and crossovers, leading to a large number of expensive objective evaluations. In this work, we ameliorate this shortcoming by incorporating chemistry-aware Large Language Models (LLMs) into EAs. Namely, we redesign crossover and mutation operations in EAs using LLMs trained on large corpora of chemical information. We perform extensive empirical studies on both commercial and open-source models on multiple tasks involving property optimization, molecular rediscovery, and structure-based drug design, demonstrating that the joint usage of LLMs with EAs yields superior performance over all baseline models across single- and multi-objective settings. We demonstrate that our algorithm improves both the quality of the final solution and convergence speed, thereby reducing the number of required objective evaluations. Our code is available at http://github.com/zoom-wang112358/MOLLEO △ Less

Submitted 2 July, 2024; v1 submitted 23 June, 2024; originally announced June 2024.

Younger: The First Dataset for Artificial Intelligence-Generated Neural Network Architecture

Authors: Zhengxin Yang, Wanling Gao, Luzhou Peng, Yunyou Huang, Fei Tang, Jianfeng Zhan

Abstract: Designing and optimizing neural network architectures typically requires extensive expertise, starting with handcrafted designs and then manual or automated refinement. This dependency presents a significant barrier to rapid innovation. Recognizing the complexity of automatically generating neural network architecture from scratch, we introduce Younger, a pioneering dataset to advance this ambitio… ▽ More Designing and optimizing neural network architectures typically requires extensive expertise, starting with handcrafted designs and then manual or automated refinement. This dependency presents a significant barrier to rapid innovation. Recognizing the complexity of automatically generating neural network architecture from scratch, we introduce Younger, a pioneering dataset to advance this ambitious goal. Derived from over 174K real-world models across more than 30 tasks from various public model hubs, Younger includes 7,629 unique architectures, and each is represented as a directed acyclic graph with detailed operator-level information. The dataset facilitates two primary design paradigms: global, for creating complete architectures from scratch, and local, for detailed architecture component refinement. By establishing these capabilities, Younger contributes to a new frontier, Artificial Intelligence-Generated Neural Network Architecture (AIGNNA). Our experiments explore the potential and effectiveness of Younger for automated architecture generation and, as a secondary benefit, demonstrate that Younger can serve as a benchmark dataset, advancing the development of graph neural networks. We release the dataset and code publicly to lower the entry barriers and encourage further research in this challenging area. △ Less

Submitted 19 June, 2024; originally announced June 2024.

Comments: 31 pages, 29 figures, 11 tables

VLBiasBench: A Comprehensive Benchmark for Evaluating Bias in Large Vision-Language Model

Authors: Jie Zhang, Sibo Wang, Xiangkui Cao, Zheng Yuan, Shiguang Shan, Xilin Chen, Wen Gao

Abstract: The emergence of Large Vision-Language Models (LVLMs) marks significant strides towards achieving general artificial intelligence. However, these advancements are tempered by the outputs that often reflect biases, a concern not yet extensively investigated. Existing benchmarks are not sufficiently comprehensive in evaluating biases due to their limited data scale, single questioning format and nar… ▽ More The emergence of Large Vision-Language Models (LVLMs) marks significant strides towards achieving general artificial intelligence. However, these advancements are tempered by the outputs that often reflect biases, a concern not yet extensively investigated. Existing benchmarks are not sufficiently comprehensive in evaluating biases due to their limited data scale, single questioning format and narrow sources of bias. To address this problem, we introduce VLBiasBench, a benchmark aimed at evaluating biases in LVLMs comprehensively. In VLBiasBench, we construct a dataset encompassing nine distinct categories of social biases, including age, disability status, gender, nationality, physical appearance, race, religion, profession, social economic status and two intersectional bias categories (race x gender, and race x social economic status). To create a large-scale dataset, we use Stable Diffusion XL model to generate 46,848 high-quality images, which are combined with different questions to form 128,342 samples. These questions are categorized into open and close ended types, fully considering the sources of bias and comprehensively evaluating the biases of LVLM from multiple perspectives. We subsequently conduct extensive evaluations on 15 open-source models as well as one advanced closed-source model, providing some new insights into the biases revealing from these models. Our benchmark is available at https://github.com/Xiangkui-Cao/VLBiasBench. △ Less

Submitted 20 June, 2024; originally announced June 2024.

A programmable wafer-scale chiroptical heterostructure of twisted aligned carbon nanotubes and phase change materials

Authors: Jichao Fan, Ruiyang Chen, Minhan Lou, Haoyu Xie, Nina Hong, Yingheng Tang, Weilu Gao

Abstract: The ability to design and dynamically control chiroptical responses in solid-state matter at wafer scale enables new opportunities in various areas. Here we present a full stack of computer-aided designs and experimental implementations of a dynamically programmable, unified, scalable chiroptical heterostructure containing twisted aligned one-dimensional (1D) carbon nanotubes (CNTs) and non-volati… ▽ More The ability to design and dynamically control chiroptical responses in solid-state matter at wafer scale enables new opportunities in various areas. Here we present a full stack of computer-aided designs and experimental implementations of a dynamically programmable, unified, scalable chiroptical heterostructure containing twisted aligned one-dimensional (1D) carbon nanotubes (CNTs) and non-volatile phase change materials (PCMs). We develop a software infrastructure based on high-performance machine learning frameworks, including differentiable programming and derivative-free optimization, to efficiently optimize the tunability of both excitonic reciprocal and linear-anisotropy-induced nonreciprocal circular dichroism (CD) responses. We experimentally implement designed heterostructures with wafer-scale self-assembled aligned CNTs and deposited PCMs. We dynamically program reciprocal and nonreciprocal CD responses by inducing phase transitions of PCMs, and nonreciprocal responses display polarity reversal of CD upon sample flipping in broadband spectral ranges. All experimental results agree with simulations. Further, we demonstrate that the vertical dimension of heterostructure is scalable with the number of stacking layers and aligned CNTs play dual roles - the layer to produce CD responses and the Joule heating electrode to electrically program PCMs. This heterostructure platform is versatile and expandable to a library of 1D nanomaterials and electro-optic materials for exploring novel chiral phenomena and photonic and optoelectronic devices. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Understanding the intrinsic framework of the Hall-Petch relationship of metals from the view of the electronic-structure level

Authors: Xin Li, Wang Gao, Qing Jiang

Abstract: The relationship between grain size and yield strength of metals follows the Hall-Petch relationship σ = σ0 + kd^-0.5; however, the specific physical factors that affect the coefficients σ0 and k of this relationship remain unclear. Here we propose the intrinsic descriptors to determine the Hall-Petch relation across different metals and alloys. Inspired by the tight-binding theory, we find that σ… ▽ More The relationship between grain size and yield strength of metals follows the Hall-Petch relationship σ = σ0 + kd^-0.5; however, the specific physical factors that affect the coefficients σ0 and k of this relationship remain unclear. Here we propose the intrinsic descriptors to determine the Hall-Petch relation across different metals and alloys. Inspired by the tight-binding theory, we find that σ0 strongly depends on the group and period number, the valence-electron number and electronegativity, while k is determined by the cohesive energy. Our framework establishes a predictive structure-property relationship for the size-dependent yield strength of various metals, and unravels that both the coefficients of the Hall-Petch relationship physically originate from the d-band properties. This novel correlation provides a new perspective for understanding the mechanical strength of metals, which is useful for the design of high-performance materials. △ Less

Submitted 18 June, 2024; originally announced June 2024.

DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence

Authors: DeepSeek-AI, Qihao Zhu, Daya Guo, Zhihong Shao, Dejian Yang, Peiyi Wang, Runxin Xu, Y. Wu, Yukun Li, Huazuo Gao, Shirong Ma, Wangding Zeng, Xiao Bi, Zihui Gu, Hanwei Xu, Damai Dai, Kai Dong, Liyue Zhang, Yishi Piao, Zhibin Gou, Zhenda Xie, Zhewen Hao, Bingxuan Wang, Junxiao Song, Deli Chen , et al. (15 additional authors not shown)

Abstract: We present DeepSeek-Coder-V2, an open-source Mixture-of-Experts (MoE) code language model that achieves performance comparable to GPT4-Turbo in code-specific tasks. Specifically, DeepSeek-Coder-V2 is further pre-trained from an intermediate checkpoint of DeepSeek-V2 with additional 6 trillion tokens. Through this continued pre-training, DeepSeek-Coder-V2 substantially enhances the coding and mathe… ▽ More We present DeepSeek-Coder-V2, an open-source Mixture-of-Experts (MoE) code language model that achieves performance comparable to GPT4-Turbo in code-specific tasks. Specifically, DeepSeek-Coder-V2 is further pre-trained from an intermediate checkpoint of DeepSeek-V2 with additional 6 trillion tokens. Through this continued pre-training, DeepSeek-Coder-V2 substantially enhances the coding and mathematical reasoning capabilities of DeepSeek-V2, while maintaining comparable performance in general language tasks. Compared to DeepSeek-Coder-33B, DeepSeek-Coder-V2 demonstrates significant advancements in various aspects of code-related tasks, as well as reasoning and general capabilities. Additionally, DeepSeek-Coder-V2 expands its support for programming languages from 86 to 338, while extending the context length from 16K to 128K. In standard benchmark evaluations, DeepSeek-Coder-V2 achieves superior performance compared to closed-source models such as GPT4-Turbo, Claude 3 Opus, and Gemini 1.5 Pro in coding and math benchmarks. △ Less

Submitted 17 June, 2024; originally announced June 2024.

Joint Power Allocation and Beamforming Design for Active IRS-Aided Directional Modulation Secure Systems

Authors: Yifan Zhao, Xiaoyu Wang, Kaibo Zhou, Xuehui Wang, Yan Wang, Wei Gao, Ruiqi Liu, Feng Shu

Abstract: Since the secrecy rate (SR) performance improvement obtained by secure directional modulation (DM) network is limited, an active intelligent reflective surface (IRS)-assisted DM network is considered to attain a high SR. To address the SR maximization problem, a novel method based on Lagrangian dual transform and closed-form fractional programming algorithm (LDT-CFFP) is proposed, where the soluti… ▽ More Since the secrecy rate (SR) performance improvement obtained by secure directional modulation (DM) network is limited, an active intelligent reflective surface (IRS)-assisted DM network is considered to attain a high SR. To address the SR maximization problem, a novel method based on Lagrangian dual transform and closed-form fractional programming algorithm (LDT-CFFP) is proposed, where the solutions to base station (BS) beamforming vectors and IRS reflection coefficient matrix are achieved. However, the computational complexity of LDT-CFFP method is high . To reduce its complexity, a blocked IRS-assisted DM network is designed. To meet the requirements of the network performance, a power allocation (PA) strategy is proposed and adopted in the system. Specifically, the system power between BS and IRS, as well as the transmission power for confidential messages (CM) and artificial noise (AN) from the BS, are allocated separately. Then we put forward null-space projection (NSP) method, maximum-ratio-reflecting (MRR) algorithm and PA strategy (NSP-MRR-PA) to solve the SR maximization problem. The CF solutions to BS beamforming vectors and IRS reflection coefficient matrix are respectively attained via NSP and MRR algorithms. For the PA factors, we take advantage of exhaustive search (ES) algorithm, particle swarm optimization (PSO) and simulated annealing (SA) algorithm to search for the solutions. From simulation results, it is verified that the LDT-CFFP method derives a higher SR gain over NSP-MRR-PA method. For NSP-MRR-PA method, the number of IRS units in each block possesses a significant SR performance. In addition, the application PA strategies, namely ES, PSO, SA methods outperforms the other PA strategies with fixed PA factors. △ Less

Submitted 25 June, 2024; v1 submitted 13 June, 2024; originally announced June 2024.

Comments: Directional modulation, active intelligent reflective surface, Lagrangian dual transformation, fractional programming, power allocation