Showing 201–250 of 784 results for author: Gao, W

Physics-aware Roughness Optimization for Diffractive Optical Neural Networks

Authors: Shanglin Zhou, Yingjie Li, Minhan Lou, Weilu Gao, Zhijie Shi, Cunxi Yu, Caiwen Ding

Abstract: As a representative next-generation device/circuit technology beyond CMOS, diffractive optical neural networks (DONNs) have shown promising advantages over conventional deep neural networks due to extreme fast computation speed (light speed) and low energy consumption. However, there is a mismatch, i.e., significant prediction accuracy loss, between the DONN numerical modelling and physical optica… ▽ More As a representative next-generation device/circuit technology beyond CMOS, diffractive optical neural networks (DONNs) have shown promising advantages over conventional deep neural networks due to extreme fast computation speed (light speed) and low energy consumption. However, there is a mismatch, i.e., significant prediction accuracy loss, between the DONN numerical modelling and physical optical device deployment, because of the interpixel interaction within the diffractive layers. In this work, we propose a physics-aware diffractive optical neural network training framework to reduce the performance difference between numerical modeling and practical deployment. Specifically, we propose the roughness modeling regularization in the training process and integrate the physics-aware sparsification method to introduce sparsity to the phase masks to reduce sharp phase changes between adjacent pixels in diffractive layers. We further develop $2π$ periodic optimization to reduce the roughness of the phase masks to preserve the performance of DONN. Experiment results demonstrate that, compared to state-of-the-arts, our physics-aware optimization can provide $35.7\%$, $34.2\%$, $28.1\%$, and $27.3\%$ reduction in roughness with only accuracy loss on MNIST, FMNIST, KMNIST, and EMNIST, respectively. △ Less

Submitted 3 April, 2023; originally announced April 2023.

Comments: This paper is accepted by the Design Automation Conference (DAC), 2023

ACM Class: I.2

IV Regressions without Exclusion Restrictions

Authors: Wayne Yuan Gao, Rui Wang

Abstract: We study identification and estimation of endogenous linear and nonlinear regression models without excluded instrumental variables, based on the standard mean independence condition and a nonlinear relevance condition. Based on the identification results, we propose two semiparametric estimators as well as a discretization-based estimator that does not require any nonparametric regressions. We es… ▽ More We study identification and estimation of endogenous linear and nonlinear regression models without excluded instrumental variables, based on the standard mean independence condition and a nonlinear relevance condition. Based on the identification results, we propose two semiparametric estimators as well as a discretization-based estimator that does not require any nonparametric regressions. We establish their asymptotic normality and demonstrate via simulations their robust finite-sample performances with respect to exclusion restrictions violations and endogeneity. Our approach is applied to study the returns to education, and to test the direct effects of college proximity indicators as well as family background variables on the outcome. △ Less

Submitted 30 July, 2023; v1 submitted 2 April, 2023; originally announced April 2023.

A Local Concentration-based Descriptor Predicting the Stacking Fault Energy of Refractory High Entropy Alloys

Authors: Cong Ma, Wang Gao, Qing Jiang

Abstract: Stacking fault energy (SFE) is an essential parameter for characterizing mechanical properties. However, in high entropy alloys (HEAs), the local chemical environment varies significantly across different stacking fault planes, resulting in a substantial fluctuation of SFE values rather than a unique value, which prohibits the prediction of the local SFE. Herein, we proposed an effective descripto… ▽ More Stacking fault energy (SFE) is an essential parameter for characterizing mechanical properties. However, in high entropy alloys (HEAs), the local chemical environment varies significantly across different stacking fault planes, resulting in a substantial fluctuation of SFE values rather than a unique value, which prohibits the prediction of the local SFE. Herein, we proposed an effective descriptor based on the local concentration ratio near stacking fault to quantitatively predict the local SFE of refractory HEAs. We find that the role of a given element in determining SFE strongly depends on its valence-electron number relative to other components and the contribution of its s- and d-electrons to its cohesive properties, which can be understood in the framework of the tight-binding model. Notably, the descriptor not only unifies the local nature of SFE from simple alloys to HEAs but also helps to quickly design HEAs as the involved parameters are easily accessible. △ Less

Submitted 26 March, 2023; originally announced March 2023.

Comments: 20 pages,10 figures

Topological edge and corner states in coupled wave lattices in nonlinear polariton condensates

Authors: Tobias Schneider, Wenlong Gao, Thomas Zentgraf, Stefan Schumacher, Xuekai Ma

Abstract: Topological states have been widely investigated in different types of systems and lattices. In the present work, we report on topological edge states in double-wave (DW) chains, which can be described by a generalized Aubry-André-Harper (AAH) model. For the specific system of a driven-dissipative exciton polariton system we show that in such potential chains, different types of edge states can fo… ▽ More Topological states have been widely investigated in different types of systems and lattices. In the present work, we report on topological edge states in double-wave (DW) chains, which can be described by a generalized Aubry-André-Harper (AAH) model. For the specific system of a driven-dissipative exciton polariton system we show that in such potential chains, different types of edge states can form. For resonant optical excitation, we further find that the optical nonlinearity leads to a multistability of different edge states. This includes topologically protected edge states evolved directly from individual linear eigenstates as well as additional edge states that originate from nonlinearity-induced localization of bulk states. Extending the system into two dimensions (2D) by stacking horizontal DW chains in the vertical direction, we also create 2D multi-wave lattices. In such 2D lattices multiple Su-Schrieffer-Heeger (SSH) chains appear along the vertical direction. The combination of DW chains in the horizontal and SSH chains in the vertical direction then results in the formation of higher-order topological insulator corner states. Multistable corner states emerge in the nonlinear regime. △ Less

Submitted 16 January, 2024; v1 submitted 22 March, 2023; originally announced March 2023.

Wearing Masks Implies Refuting Trump?: Towards Target-specific User Stance Prediction across Events in COVID-19 and US Election 2020

Authors: Hong Zhang, Haewoon Kwak, Wei Gao, Jisun An

Abstract: People who share similar opinions towards controversial topics could form an echo chamber and may share similar political views toward other topics as well. The existence of such connections, which we call connected behavior, gives researchers a unique opportunity to predict how one would behave for a future event given their past behaviors. In this work, we propose a framework to conduct connecte… ▽ More People who share similar opinions towards controversial topics could form an echo chamber and may share similar political views toward other topics as well. The existence of such connections, which we call connected behavior, gives researchers a unique opportunity to predict how one would behave for a future event given their past behaviors. In this work, we propose a framework to conduct connected behavior analysis. Neural stance detection models are trained on Twitter data collected on three seemingly independent topics, i.e., wearing a mask, racial equality, and Trump, to detect people's stance, which we consider as their online behavior in each topic-related event. Our results reveal a strong connection between the stances toward the three topical events and demonstrate the power of past behaviors in predicting one's future behavior. △ Less

Submitted 21 March, 2023; originally announced March 2023.

Comments: 10 pages, 2 pages, WebSci 2023, April 30-May 1, 2023, Evanston, TX, USA

Diffusion-Based 3D Human Pose Estimation with Multi-Hypothesis Aggregation

Authors: Wenkang Shan, Zhenhua Liu, Xinfeng Zhang, Zhao Wang, Kai Han, Shanshe Wang, Siwei Ma, Wen Gao

Abstract: In this paper, a novel Diffusion-based 3D Pose estimation (D3DP) method with Joint-wise reProjection-based Multi-hypothesis Aggregation (JPMA) is proposed for probabilistic 3D human pose estimation. On the one hand, D3DP generates multiple possible 3D pose hypotheses for a single 2D observation. It gradually diffuses the ground truth 3D poses to a random distribution, and learns a denoiser conditi… ▽ More In this paper, a novel Diffusion-based 3D Pose estimation (D3DP) method with Joint-wise reProjection-based Multi-hypothesis Aggregation (JPMA) is proposed for probabilistic 3D human pose estimation. On the one hand, D3DP generates multiple possible 3D pose hypotheses for a single 2D observation. It gradually diffuses the ground truth 3D poses to a random distribution, and learns a denoiser conditioned on 2D keypoints to recover the uncontaminated 3D poses. The proposed D3DP is compatible with existing 3D pose estimators and supports users to balance efficiency and accuracy during inference through two customizable parameters. On the other hand, JPMA is proposed to assemble multiple hypotheses generated by D3DP into a single 3D pose for practical use. It reprojects 3D pose hypotheses to the 2D camera plane, selects the best hypothesis joint-by-joint based on the reprojection errors, and combines the selected joints into the final pose. The proposed JPMA conducts aggregation at the joint level and makes use of the 2D prior information, both of which have been overlooked by previous approaches. Extensive experiments on Human3.6M and MPI-INF-3DHP datasets show that our method outperforms the state-of-the-art deterministic and probabilistic approaches by 1.5% and 8.9%, respectively. Code is available at https://github.com/paTRICK-swk/D3DP. △ Less

Submitted 22 August, 2023; v1 submitted 21 March, 2023; originally announced March 2023.

Comments: ICCV 2023

Boundary Unlearning

Authors: Min Chen, Weizhuo Gao, Gaoyang Liu, Kai Peng, Chen Wang

Abstract: The practical needs of the ``right to be forgotten'' and poisoned data removal call for efficient \textit{machine unlearning} techniques, which enable machine learning models to unlearn, or to forget a fraction of training data and its lineage. Recent studies on machine unlearning for deep neural networks (DNNs) attempt to destroy the influence of the forgetting data by scrubbing the model paramet… ▽ More The practical needs of the ``right to be forgotten'' and poisoned data removal call for efficient \textit{machine unlearning} techniques, which enable machine learning models to unlearn, or to forget a fraction of training data and its lineage. Recent studies on machine unlearning for deep neural networks (DNNs) attempt to destroy the influence of the forgetting data by scrubbing the model parameters. However, it is prohibitively expensive due to the large dimension of the parameter space. In this paper, we refocus our attention from the parameter space to the decision space of the DNN model, and propose Boundary Unlearning, a rapid yet effective way to unlearn an entire class from a trained DNN model. The key idea is to shift the decision boundary of the original DNN model to imitate the decision behavior of the model retrained from scratch. We develop two novel boundary shift methods, namely Boundary Shrink and Boundary Expanding, both of which can rapidly achieve the utility and privacy guarantees. We extensively evaluate Boundary Unlearning on CIFAR-10 and Vggface2 datasets, and the results show that Boundary Unlearning can effectively forget the forgetting class on image classification and face recognition tasks, with an expected speed-up of $17\times$ and $19\times$, respectively, compared with retraining from the scratch. △ Less

Submitted 20 March, 2023; originally announced March 2023.

Comments: To be published in Proc.of 34th IEEE/CVF CVPR, 2023

Estimation of the Directions for Unknown Parameters in Semiparametric Models

Authors: Jinyue Han, Jun Wang, Wei Gao, Man-Lai Tang

Abstract: Semiparametric models are useful in econometrics, social sciences and medicine application. In this paper, a new estimator based on least square methods is proposed to estimate the direction of unknown parameters in semi-parametric models. The proposed estimator is consistent and has asymptotic distribution under mild conditions without the knowledge of the form of link function. Simulations show… ▽ More Semiparametric models are useful in econometrics, social sciences and medicine application. In this paper, a new estimator based on least square methods is proposed to estimate the direction of unknown parameters in semi-parametric models. The proposed estimator is consistent and has asymptotic distribution under mild conditions without the knowledge of the form of link function. Simulations show that the proposed estimator is significantly superior to maximum score estimator given by Manski (1975) for binary response variables. When the error term is long-tailed distributions or distribution with infinity moments, the proposed estimator perform well. Its application is illustrated with data of exporting participation of manufactures in Guangdong. △ Less

Submitted 9 March, 2023; originally announced March 2023.

Comments: 19 pages

MSC Class: 62G05 ACM Class: G.3

Transition from Diffusive to Superdiffusive Transport in Carbon Nanotube Networks via Nematic Order Control

Authors: Michael Wais, Filchito Renee G. Bagsican, Natsumi Komatsu, Weilu Gao, Kazunori Serita, Hironaru Murakami, Karsten Held, Iwao Kawayama, Junichiro Kono, Marco Battiato, Masayoshi Tonouchi

Abstract: The one-dimensional confinement of quasiparticles in individual carbon nanotubes (CNTs) leads to extremely anisotropic electronic and optical properties. In a macroscopic ensemble of randomly oriented CNTs, this anisotropy disappears together with other properties that make them attractive for certain device applications. The question however remains if not only anisotropy, but other types of beha… ▽ More The one-dimensional confinement of quasiparticles in individual carbon nanotubes (CNTs) leads to extremely anisotropic electronic and optical properties. In a macroscopic ensemble of randomly oriented CNTs, this anisotropy disappears together with other properties that make them attractive for certain device applications. The question however remains if not only anisotropy, but other types of behaviours are suppressed by disorder. Here, we compare the dynamics of quasiparticles under strong electric fields in aligned and random CNT networks using a combination of terahertz emission and photocurrent experiments and out-of-equilibrium numerical simulations. We find that the degree of alignment strongly influences the excited quasiparticles' dynamics, rerouting the thermalisation pathways. This is, in particular, evidenced in the high-energy, high-momentum electronic population (probed through the formation of low energy excitons via exciton impact ionization) and the transport regime evolving from diffusive to superdiffusive. △ Less

Submitted 11 May, 2023; v1 submitted 3 March, 2023; originally announced March 2023.

Comments: 17 pages, 5 figures

Moiré Synergy: An Emerging Game Changer by Moiré of Moiré

Authors: Xiangbin Cai, Weibo Gao

Abstract: Moiré superlattices of tunable wavelengths and the further developed moiré of moiré systems, by artificially assembling two-dimensional (2D) van der Waals (vdW) materials as designed, have brought up a versatile toolbox to explore fascinating condensed mater physics and their stimulating physicochemical functionalities. In this Perspective, we briefly review the recent progress in the emerging fie… ▽ More Moiré superlattices of tunable wavelengths and the further developed moiré of moiré systems, by artificially assembling two-dimensional (2D) van der Waals (vdW) materials as designed, have brought up a versatile toolbox to explore fascinating condensed mater physics and their stimulating physicochemical functionalities. In this Perspective, we briefly review the recent progress in the emerging field of moiré synergy, highlighting the synergetic effects arising in distinct dual moiré heterostructures of graphene and transition metal dichalcogenides (TMDCs). A spectrum of moiré of moiré configurations, the advanced characterization and the exploitation efforts on the moiré-moiré interactions will be discussed. Finally, we look out for urgent challenges to be conquered in the community and some potential research directions in the near future. △ Less

Submitted 3 March, 2023; originally announced March 2023.

Comments: 14 pages, 4 figures

Interlayer donor-acceptor pair excitons in MoSe2/WSe2 moiré heterobilayer

Authors: Hongbing Cai, Abdullah Rasmita, Qinghai Tan, Jia-Min Lai, Ruihua He, Disheng Chen, Naizhou Wang, Zhao Mu, Zumeng Huang, Zhaowei Zhang, John J. H. Eng, Yuanda Liu, Yongzhi She, Nan Pan, Xiaoping Wang, Xiaogang Liu, Jun Zhang, Weibo Gao

Abstract: Localized interlayer excitons (LIXs) in two-dimensional moiré superlattices exhibit sharp and dense emission peaks, making them promising as highly tunable single-photon sources. However, the fundamental nature of these LIXs is still elusive. Here, we show the donor-acceptor pair (DAP) mechanism as one of the origins of these excitonic peaks. Numerical simulation results of the DAP model agree wit… ▽ More Localized interlayer excitons (LIXs) in two-dimensional moiré superlattices exhibit sharp and dense emission peaks, making them promising as highly tunable single-photon sources. However, the fundamental nature of these LIXs is still elusive. Here, we show the donor-acceptor pair (DAP) mechanism as one of the origins of these excitonic peaks. Numerical simulation results of the DAP model agree with the experimental photoluminescence spectra of LIX in the moiré MoSe2/WSe2 heterobilayer. In particular, we find that the emission energy-lifetime correlation and the nonmonotonic power dependence of the lifetime agree well with the DAP IX model. Our results provide insight into the physical mechanism of LIX formation in moiré heterostructures and pave new directions for engineering interlayer exciton properties in moiré superlattices. △ Less

Submitted 25 February, 2023; originally announced February 2023.

WPC: Whole-picture Workload Characterization

Authors: Lei Wang, Kaiyong Yang, Chenxi Wang, Wanling Gao, Chunjie Luo, Fan Zhang, Zhongxin Ge, Li Zhang, Guoxin Kang, Jianfeng Zhan

Abstract: This article raises an important and challenging workload characterization issue: can we uncover each critical component across the stacks contributing what percentages to any specific bottleneck? The typical critical components include languages, programming frameworks, runtime environments, instruction set architectures (ISA), operating systems (OS), and microarchitecture. Tackling this issue co… ▽ More This article raises an important and challenging workload characterization issue: can we uncover each critical component across the stacks contributing what percentages to any specific bottleneck? The typical critical components include languages, programming frameworks, runtime environments, instruction set architectures (ISA), operating systems (OS), and microarchitecture. Tackling this issue could help propose a systematic methodology to guide the software and hardware co-design and critical component optimizations. We propose a whole-picture workload characterization (WPC) methodology to answer the above issue. In essence, WPC is an iterative ORFE loop consisting of four steps: Observation, Reference, Fusion, and Exploration. WPC observes different level data (observation), fuses and normalizes the performance data (fusion) with respect to the well-designed standard reference workloads suite (reference), and explores the software and hardware co-design space (exploration) to investigate the impacts of critical components across the stacks. We build and open-source the WPC tool. Our evaluations confirm WPC can quantitatively reveal the contributions of the language, framework, runtime environment, ISA, OS, and microarchitecture to the primary pipeline efficiency. △ Less

Submitted 24 February, 2023; originally announced February 2023.

DCNetBench: Scaleable Data Center Network Benchmarking

Authors: Ke Liu, Wanling Gao, Chunjie Luo, Cheng Huang, Chunxin Lan, Zhenxing Zhang, Lei Wang, Xiwen He, Nan Li, Jianfeng Zhan

Abstract: Data center networking is the central infrastructure of the modern information society. However, benchmarking them is very challenging as the real-world network traffic is difficult to model, and Internet service giants treat the network traffic as confidential. Several industries have published a few publicly available network traces. However, these traces are collected from specific data center… ▽ More Data center networking is the central infrastructure of the modern information society. However, benchmarking them is very challenging as the real-world network traffic is difficult to model, and Internet service giants treat the network traffic as confidential. Several industries have published a few publicly available network traces. However, these traces are collected from specific data center environments, e.g., applications, network topology, protocols, and hardware devices, and thus cannot be scaled to different users, underlying technologies, and varying benchmarking requirements. This article argues we should scale different data center applications and environments in designing, implementing, and evaluating data center networking benchmarking. We build DCNetBench, the first application-driven data center network benchmarking that can scale to different users, underlying technologies, and varying benchmarking requirements. The methodology is as follows. We built an emulated system that can simulate networking with different configurations. Then we run applications on the emulated systems to capture the realistic network traffic patterns; we analyze and classify these patterns to model and replay those traces. Finally, we provide an automatic benchmarking framework to support this pipeline. The evaluations on DCNetBench show its scaleability, effectiveness, and diversity for data center network benchmarking. △ Less

Submitted 23 February, 2023; originally announced February 2023.

Comments: 19 pages, 15 figures

Scientific Computing with Diffractive Optical Neural Networks

Authors: Ruiyang Chen, Yingheng Tang, Jianzhu Ma, Weilu Gao

Abstract: Diffractive optical neural networks (DONNs) have been emerging as a high-throughput and energy-efficient hardware platform to perform all-optical machine learning (ML) in machine vision systems. However, the current demonstrated applications of DONNs are largely straightforward image classification tasks, which undermines the prospect of developing and utilizing such hardware for other ML applicat… ▽ More Diffractive optical neural networks (DONNs) have been emerging as a high-throughput and energy-efficient hardware platform to perform all-optical machine learning (ML) in machine vision systems. However, the current demonstrated applications of DONNs are largely straightforward image classification tasks, which undermines the prospect of developing and utilizing such hardware for other ML applications. Here, we numerically and experimentally demonstrate the deployment of an all-optical reconfigurable DONNs system for scientific computing, including guiding two-dimensional quantum material synthesis, predicting the properties of nanomaterials and small molecular cancer drugs, predicting the device response of nanopatterned integrated photonic power splitters, and the dynamic stabilization of an inverted pendulum with reinforcement learning. Despite a large variety of input data structures, we develop a universal feature engineering approach to convert categorical input features to the images that can be processed in the DONNs system. Our results open up new opportunities of employing DONNs systems for a broad range of ML applications. △ Less

Submitted 12 February, 2023; originally announced February 2023.

Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey

Authors: Xiao Wang, Guangyao Chen, Guangwu Qian, Pengcheng Gao, Xiao-Yong Wei, Yaowei Wang, Yonghong Tian, Wen Gao

Abstract: With the urgent demand for generalized deep models, many pre-trained big models are proposed, such as BERT, ViT, GPT, etc. Inspired by the success of these models in single domains (like computer vision and natural language processing), the multi-modal pre-trained big models have also drawn more and more attention in recent years. In this work, we give a comprehensive survey of these models and ho… ▽ More With the urgent demand for generalized deep models, many pre-trained big models are proposed, such as BERT, ViT, GPT, etc. Inspired by the success of these models in single domains (like computer vision and natural language processing), the multi-modal pre-trained big models have also drawn more and more attention in recent years. In this work, we give a comprehensive survey of these models and hope this paper could provide new insights and helps fresh researchers to track the most cutting-edge works. Specifically, we firstly introduce the background of multi-modal pre-training by reviewing the conventional deep learning, pre-training works in natural language process, computer vision, and speech. Then, we introduce the task definition, key challenges, and advantages of multi-modal pre-training models (MM-PTMs), and discuss the MM-PTMs with a focus on data, objectives, network architectures, and knowledge enhanced pre-training. After that, we introduce the downstream tasks used for the validation of large-scale MM-PTMs, including generative, classification, and regression tasks. We also give visualization and analysis of the model parameters and results on representative downstream tasks. Finally, we point out possible research directions for this topic that may benefit future works. In addition, we maintain a continuously updated paper list for large-scale pre-trained multi-modal big models: https://github.com/wangxiao5791509/MultiModal_BigModels_Survey. This paper has been published by the journal Machine Intelligence Research (MIR), https://link.springer.com/article/10.1007/s11633-022-1410-8, DOI: 10.1007/s11633-022-1410-8, vol. 20, no. 4, pp. 447-482, 2023. △ Less

Submitted 10 April, 2024; v1 submitted 20 February, 2023; originally announced February 2023.

Comments: Accepted by Machine Intelligence Research (MIR)

Stationary Point Losses for Robust Model

Authors: Weiwei Gao, Dazhi Zhang, Yao Li, Zhichang Guo, Ovanes Petrosian

Abstract: The inability to guarantee robustness is one of the major obstacles to the application of deep learning models in security-demanding domains. We identify that the most commonly used cross-entropy (CE) loss does not guarantee robust boundary for neural networks. CE loss sharpens the neural network at the decision boundary to achieve a lower loss, rather than pushing the boundary to a more robust po… ▽ More The inability to guarantee robustness is one of the major obstacles to the application of deep learning models in security-demanding domains. We identify that the most commonly used cross-entropy (CE) loss does not guarantee robust boundary for neural networks. CE loss sharpens the neural network at the decision boundary to achieve a lower loss, rather than pushing the boundary to a more robust position. A robust boundary should be kept in the middle of samples from different classes, thus maximizing the margins from the boundary to the samples. We think this is due to the fact that CE loss has no stationary point. In this paper, we propose a family of new losses, called stationary point (SP) loss, which has at least one stationary point on the correct classification side. We proved that robust boundary can be guaranteed by SP loss without losing much accuracy. With SP loss, larger perturbations are required to generate adversarial examples. We demonstrate that robustness is improved under a variety of adversarial attacks by applying SP loss. Moreover, robust boundary learned by SP loss also performs well on imbalanced datasets. △ Less

Submitted 19 February, 2023; originally announced February 2023.

Comments: 13 pages, 12 figures

Microscopic Energy Storage Mechanism of Dielectric Polymer-Coated Supercapacitors

Authors: Weihang Gao, Teng Zhao, Shian Dong, Xingyi Huang, Zhenli Xu

Abstract: Supercapacitors have been attracting significant attention as promising energy storage devices. However, the voltage window limitation associated with electrolyte solutions has hindered the improvement of their capacitance. To address this issue and enhance the energy storage capabilities of general traditional supercapacitors, we put forward the dipole induced effects observed in the theoretical… ▽ More Supercapacitors have been attracting significant attention as promising energy storage devices. However, the voltage window limitation associated with electrolyte solutions has hindered the improvement of their capacitance. To address this issue and enhance the energy storage capabilities of general traditional supercapacitors, we put forward the dipole induced effects observed in the theoretical framework of the electric double-layer structure. The molecular dynamics results demonstrate that, compared to traditional systems, an improvement of over 50% in integral capacitance at low voltages is achieved. Moreover, a new material-based experimental results obtained from a dielectric supercapacitor employing a hydrated electrolyte solution corroborated the effectiveness of our proposed model, yielding consistent outcomes. We attribute the large capacitance variation to the reorientation of the dipoles, which induces the neutral-to-bilayer transition and the overscreening-to-steric transition, consistent with the polarization process of the polymer in the experiment. We further investigate the capacitance variations under different dipole parameters, such as varying the number of layers, different number densities and different spacings, thereby enriching the experimental results with additional conclusions not previously obtained. This work presents a novel approach that exploits dipole-induced capacitance effects, paving the way for further advances in the field of energy storage technology. △ Less

Submitted 25 June, 2023; v1 submitted 19 February, 2023; originally announced February 2023.

CMLCompiler: A Unified Compiler for Classical Machine Learning

Authors: Xu Wen, Wanling Gao, Anzheng Li, Lei Wang, Zihan Jiang, Jianfeng Zhan

Abstract: Classical machine learning (CML) occupies nearly half of machine learning pipelines in production applications. Unfortunately, it fails to utilize the state-of-the-practice devices fully and performs poorly. Without a unified framework, the hybrid deployments of deep learning (DL) and CML also suffer from severe performance and portability issues. This paper presents the design of a unified compil… ▽ More Classical machine learning (CML) occupies nearly half of machine learning pipelines in production applications. Unfortunately, it fails to utilize the state-of-the-practice devices fully and performs poorly. Without a unified framework, the hybrid deployments of deep learning (DL) and CML also suffer from severe performance and portability issues. This paper presents the design of a unified compiler, called CMLCompiler, for CML inference. We propose two unified abstractions: operator representations and extended computational graphs. The CMLCompiler framework performs the conversion and graph optimization based on two unified abstractions, then outputs an optimized computational graph to DL compilers or frameworks. We implement CMLCompiler on TVM. The evaluation shows CMLCompiler's portability and superior performance. It achieves up to 4.38$\times$ speedup on CPU, 3.31$\times$ speedup on GPU, and 5.09$\times$ speedup on IoT devices, compared to the state-of-the-art solutions -- scikit-learn, intel sklearn, and hummingbird. Our performance of CML and DL mixed pipelines achieves up to 3.04x speedup compared with cross-framework implementations. The project documents and source code are available at https://www.computercouncil.org/cmlcompiler. △ Less

Submitted 28 April, 2023; v1 submitted 31 January, 2023; originally announced January 2023.

Delayed Stochastic Algorithms for Distributed Weakly Convex Optimization

Authors: Wenzhi Gao, Qi Deng

Abstract: This paper studies delayed stochastic algorithms for weakly convex optimization in a distributed network with workers connected to a master node. Recently, Xu et al. 2022 showed that an inertial stochastic subgradient method converges at a rate of $\mathcal{O}(τ_{\text{max}}/\sqrt{K})$ which depends on the maximum information delay $τ_{\text{max}}$. In this work, we show that the delayed stochasti… ▽ More This paper studies delayed stochastic algorithms for weakly convex optimization in a distributed network with workers connected to a master node. Recently, Xu et al. 2022 showed that an inertial stochastic subgradient method converges at a rate of $\mathcal{O}(τ_{\text{max}}/\sqrt{K})$ which depends on the maximum information delay $τ_{\text{max}}$. In this work, we show that the delayed stochastic subgradient method ($\texttt{DSGD}$) obtains a tighter convergence rate which depends on the expected delay $\barτ$. Furthermore, for an important class of composition weakly convex problems, we develop a new delayed stochastic prox-linear ($\texttt{DSPL}$) method in which the delays only affect the high-order term in the complexity rate and hence, are negligible after a certain number of $\texttt{DSPL}$ iterations. In addition, we demonstrate the robustness of our proposed algorithms against arbitrary delays. By incorporating a simple safeguarding step in both methods, we achieve convergence rates that depend solely on the number of workers, eliminating the effect of the delay. Our numerical experiments further confirm the empirical superiority of our proposed methods. △ Less

Submitted 1 November, 2023; v1 submitted 30 January, 2023; originally announced January 2023.

Controlled synthetic chirality in macroscopic assemblies of carbon nanotubes

Authors: Jacques Doumani, Minhan Lou, Oliver Dewey, Nina Hong, Jichao Fan, Andrey Baydin, Yohei Yomogida, Kazuhiro Yanagi, Matteo Pasquali, Riichiro Saito, Junichiro Kono, Weilu Gao

Abstract: There is an emerging recognition that successful utilization of chiral degrees of freedom can bring new scientific and technological opportunities to diverse research areas. Hence, methods are being sought for creating artificial matter with controllable chirality in an uncomplicated and reproducible manner. Here, we report the development of two straightforward methods for fabricating wafer-scale… ▽ More There is an emerging recognition that successful utilization of chiral degrees of freedom can bring new scientific and technological opportunities to diverse research areas. Hence, methods are being sought for creating artificial matter with controllable chirality in an uncomplicated and reproducible manner. Here, we report the development of two straightforward methods for fabricating wafer-scale chiral architectures of ordered carbon nanotubes (CNTs) with tunable and giant circular dichroism (CD). Both methods employ simple approaches, (i) mechanical rotation and (ii) twist-stacking, based on controlled vacuum filtration and do not involve any sophisticated nanofabrication processes. We used a racemic mixture of CNTs as the starting material, so the intrinsic chirality of chiral CNTs is not responsible for the observed chirality. In particular, by controlling the stacking angle and handedness in (ii), we were able to maximize the CD response and achieve a record-high deep-ultraviolet ellipticity of 40 $\pm$ 1 mdeg/nm. Our theoretical simulations using the transfer matrix method reproduce the salient features of the experimentally observed CD spectra and further predict that a film of twist-stacked CNTs with an optimized thickness will exhibit an ellipticity as high as 150 mdeg/nm. The created wafer-scale objects represent a new class of synthetic chiral matter consisting of ordered quantum wires whose macroscopic properties are governed by nanoscopic electronic signatures such as van Hove singularities. These artificial structures with engineered chirality will not only provide playgrounds for uncovering new chiral phenomena but also open up new opportunities for developing high-performance chiral photonic and optoelectronic devices. △ Less

Submitted 28 January, 2023; originally announced January 2023.

Comments: Main text: 4 figures, 21 pages; SI: 9 figures, 1 table, 11 pages

Spectral fingerprint of quantum confinement in single CsPbBr$_3$ nanocrystals

Authors: Mohamed-Raouf Amara, Zakaria Said, Caixia Huo, Aurélie Pierret, Christophe Voisin, Weibo Gao, Qihua Xiong, Carole Diederichs

Abstract: Lead halide perovskite nanocrystals are promising materials for classical and quantum light emission. To understand these outstanding properties, a thorough analysis of the band-edge exciton emission is needed which is not reachable in ensemble and room temperature studies because of broadening effects. Here, we report on a cryogenic-temperature study of the photoluminescence of single CsPbBr$_3$… ▽ More Lead halide perovskite nanocrystals are promising materials for classical and quantum light emission. To understand these outstanding properties, a thorough analysis of the band-edge exciton emission is needed which is not reachable in ensemble and room temperature studies because of broadening effects. Here, we report on a cryogenic-temperature study of the photoluminescence of single CsPbBr$_3$ NCs in the intermediate quantum confinement regime. We reveal the size-dependence of the spectral features observed: the bright-triplet exciton energy splittings, the trion and biexciton binding energies as well as the optical phonon replica spectrum. In addition, we show that bright triplet energy splittings are consistent with a pure exchange model and that the variety of polarisation properties and spectra recorded can be rationalised simply by considering the orientation of the emitting dipoles and the populations of the emitting states. △ Less

Submitted 31 March, 2023; v1 submitted 23 January, 2023; originally announced January 2023.

Weighted Trace-Penalty Minimization for Full Configuration Interaction

Authors: Weiguo Gao, Yingzhou Li, Hanxiang Shen

Abstract: A novel unconstrained optimization model named weighted trace-penalty minimization (WTPM) is proposed to address the extreme eigenvalue problem arising from the Full Configuration Interaction (FCI) method. Theoretical analysis shows that the global minimizers of the WTPM objective function are the desired eigenvectors, rather than the eigenspace. Analyzing the condition number of the Hessian opera… ▽ More A novel unconstrained optimization model named weighted trace-penalty minimization (WTPM) is proposed to address the extreme eigenvalue problem arising from the Full Configuration Interaction (FCI) method. Theoretical analysis shows that the global minimizers of the WTPM objective function are the desired eigenvectors, rather than the eigenspace. Analyzing the condition number of the Hessian operator in detail contributes to the determination of a near-optimal weight matrix. With the sparse feature of FCI matrices in mind, the coordinate descent (CD) method is adapted to WTPM and results in WTPM-CD method. The reduction of computational and storage costs in each iteration shows the efficiency of the proposed algorithm. Finally, the numerical experiments demonstrate the capability to address large-scale FCI matrices. △ Less

Submitted 7 October, 2023; v1 submitted 17 January, 2023; originally announced January 2023.

MSC Class: 65F15

Learning to Compress Unmanned Aerial Vehicle (UAV) Captured Video: Benchmark and Analysis

Authors: Chuanmin Jia, Feng Ye, Huifang Sun, Siwei Ma, Wen Gao

Abstract: During the past decade, the Unmanned-Aerial-Vehicles (UAVs) have attracted increasing attention due to their flexible, extensive, and dynamic space-sensing capabilities. The volume of video captured by UAVs is exponentially growing along with the increased bitrate generated by the advancement of the sensors mounted on UAVs, bringing new challenges for on-device UAV storage and air-ground data tran… ▽ More During the past decade, the Unmanned-Aerial-Vehicles (UAVs) have attracted increasing attention due to their flexible, extensive, and dynamic space-sensing capabilities. The volume of video captured by UAVs is exponentially growing along with the increased bitrate generated by the advancement of the sensors mounted on UAVs, bringing new challenges for on-device UAV storage and air-ground data transmission. Most existing video compression schemes were designed for natural scenes without consideration of specific texture and view characteristics of UAV videos. In this work, we first contribute a detailed analysis of the current state of the field of UAV video coding. Then we propose to establish a novel task for learned UAV video coding and construct a comprehensive and systematic benchmark for such a task, present a thorough review of high quality UAV video datasets and benchmarks, and contribute extensive rate-distortion efficiency comparison of learned and conventional codecs after. Finally, we discuss the challenges of encoding UAV videos. It is expected that the benchmark will accelerate the research and development in video coding on drone platforms. △ Less

Submitted 15 January, 2023; originally announced January 2023.

Comments: MPAI End-to-end Video group progress report, DCC 2023

Unravelling the deterministic effect of the solid-state diffusion energy barrier for charge carrier on the self-discharge of supercapacitors

Authors: Xiaohui Yan, Yue He, Xuncheng Liu, Siqi Jing, Jiajian Guan, Wei Gao, Sudip Ray, Yige Xiong, Taibai Li, Xiang Ge

Abstract: The further development of fast electrochemical devices is hindered by self-discharge. Current strategies for suppressing self-discharge are mainly focused on the extrinsic and general mechanisms including faradaic reactions, charge redistribution, and ohmic leakage. However, the self-discharge process is still severe for conventional supercapacitors. Herein, we unravel the deterministic effect of… ▽ More The further development of fast electrochemical devices is hindered by self-discharge. Current strategies for suppressing self-discharge are mainly focused on the extrinsic and general mechanisms including faradaic reactions, charge redistribution, and ohmic leakage. However, the self-discharge process is still severe for conventional supercapacitors. Herein, we unravel the deterministic effect of solid-state diffusion energy barrier by constructing conjugately configured supercapacitors based on pairs of pre-lithiated niobium oxides with similar intercalation pseudocapacitive process but different phases. This device works with a single type of charge carrier while materials with various diffusion barriers can be implanted, thus serving as an ideal platform to illustrate the influence of the diffusion barrier. The results show that the comprehensive effect of solid-state diffusion energy barrier and extrinsic effects drives the self-discharge process. Noteworthy, the diffusion barrier presents with an exponential form, which governs the self-discharge of supercapacitors. This work is expected to unravel the deterministic effect of the solid-state diffusion energy barrier and provide a general guidance for suppressing self-discharge for supercapacitors. △ Less

Submitted 12 January, 2023; v1 submitted 9 January, 2023; originally announced January 2023.

Comments: 35 pages, 16 figures, 1 table

TWR-MCAE: A Data Augmentation Method for Through-the-Wall Radar Human Motion Recognition

Authors: Weicheng Gao, Xiaopeng Yang, Xiaodong Qu, Tian Lan

Abstract: To solve the problems of reduced accuracy and prolonging convergence time of through-the-wall radar (TWR) human motion due to wall attenuation, multipath effect, and system interference, we propose a multilink auto-encoding neural network (TWR-MCAE) data augmentation method. Specifically, the TWR-MCAE algorithm is jointly constructed by a singular value decomposition (SVD)-based data preprocessing… ▽ More To solve the problems of reduced accuracy and prolonging convergence time of through-the-wall radar (TWR) human motion due to wall attenuation, multipath effect, and system interference, we propose a multilink auto-encoding neural network (TWR-MCAE) data augmentation method. Specifically, the TWR-MCAE algorithm is jointly constructed by a singular value decomposition (SVD)-based data preprocessing module, an improved coordinate attention module, a compressed sensing learnable iterative shrinkage threshold reconstruction algorithm (LISTA) module, and an adaptive weight module. The data preprocessing module achieves wall clutter, human motion features, and noise subspaces separation. The improved coordinate attention module achieves clutter and noise suppression. The LISTA module achieves human motion feature enhancement. The adaptive weight module learns the weights and fuses the three subspaces. The TWR-MCAE can suppress the low-rank characteristics of wall clutter and enhance the sparsity characteristics in human motion at the same time. It can be linked before the classification step to improve the feature extraction capability without adding other prior knowledge or recollecting more data. Experiments show that the proposed algorithm gets a better peak signal-to-noise ratio (PSNR), which increases the recognition accuracy and speeds up the training process of the back-end classifiers. △ Less

Submitted 6 January, 2023; originally announced January 2023.

Comments: Publisher: IEEE Transactions on Geoscience and Remote Sensing (Volume: 60). Total Pages: 17. Total Figures: 17

Journal ref: in IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1-17, 2022, Art no. 5118617

Quality at the Tail of Machine Learning Inference

Authors: Zhengxin Yang, Wanling Gao, Chunjie Luo, Lei Wang, Fei Tang, Xu Wen, Jianfeng Zhan

Abstract: Machine learning inference should be subject to stringent inference time constraints while ensuring high inference quality, especially in safety-critical (e.g., autonomous driving) and mission-critical (e.g., emotion recognition) contexts. Neglecting either aspect can lead to severe consequences, such as loss of life and property damage. Many studies lack a comprehensive consideration of these met… ▽ More Machine learning inference should be subject to stringent inference time constraints while ensuring high inference quality, especially in safety-critical (e.g., autonomous driving) and mission-critical (e.g., emotion recognition) contexts. Neglecting either aspect can lead to severe consequences, such as loss of life and property damage. Many studies lack a comprehensive consideration of these metrics, leading to incomplete or misleading evaluations. The study unveils a counterintuitive revelation: deep learning inference quality exhibits fluctuations due to inference time. To depict this phenomenon, the authors coin a new term, "tail quality," providing a more comprehensive evaluation, and overcoming conventional metric limitations. Moreover, the research proposes an initial evaluation framework to analyze factors affecting quality fluctuations, facilitating the prediction of the potential distribution of inference quality. The effectiveness of the evaluation framework is validated through experiments conducted on deep learning models for three different tasks across four systems. △ Less

Submitted 26 February, 2024; v1 submitted 25 December, 2022; originally announced December 2022.

Comments: 10 pages, 4 figures, 4 tables

Differential Flatness of Lifting-Wing Quadcopters Subject to Drag and Lift for Accurate Tracking

Authors: Shuai Wang, Wenhan Gao, Quan Quan

Abstract: In this paper, we propose an effective unified control law for accurately tracking agile trajectories for lifting-wing quadcopters with different installation angles, which have the capability of vertical takeoff and landing (VTOL) as well as high-speed cruise flight. First, we derive a differential flatness transform for the lifting-wing dynamics with a nonlinear model under coordinated turn cond… ▽ More In this paper, we propose an effective unified control law for accurately tracking agile trajectories for lifting-wing quadcopters with different installation angles, which have the capability of vertical takeoff and landing (VTOL) as well as high-speed cruise flight. First, we derive a differential flatness transform for the lifting-wing dynamics with a nonlinear model under coordinated turn condition. To increase the tracking performance on agile trajectories, the proposed controller incorporates the state and input variables calculated from differential flatness as feedforward. In particular, the jerk, the 3-order derivative of the trajectory, is converted into angular velocity as a feedforward item, which significantly improves the system bandwidth. At the same time, feedback and feedforward outputs are combined to deal with external disturbances and model mismatch. The control algorithm has been thoroughly evaluated in the outdoor flight tests, which show that it can achieve accurate trajectory tracking. △ Less

Submitted 25 December, 2022; originally announced December 2022.

ToL: A Tensor of List-Based Unified Computation Model

Authors: Hongxiao Li, Wanling Gao, Lei Wang, Jianfeng Zhan

Abstract: Previous computation models either have equivalent abilities in representing all computations but fail to provide primitive operators for programming complex algorithms or lack generalized expression ability to represent newly-added computations. This article presents a unified computation model with generalized expression ability and a concise set of primitive operators for programming high-level… ▽ More Previous computation models either have equivalent abilities in representing all computations but fail to provide primitive operators for programming complex algorithms or lack generalized expression ability to represent newly-added computations. This article presents a unified computation model with generalized expression ability and a concise set of primitive operators for programming high-level algorithms. We propose a unified data abstraction -- Tensor of List, and offer a unified computation model based on Tensor of List, which we call the ToL model (in short, ToL). ToL introduces five atomic computations that can represent any elementary computation by finite composition, ensured with strict formal proof. Based on ToL, we design a pure-functional language -- ToLang. ToLang provides a concise set of primitive operators that can be used to program complex big data and AI algorithms. Our evaluations show ToL has generalized expression ability and a built-in performance indicator, born with a strictly defined computation metric -- elementary operation count (EOPs), consistent with FLOPs within a small error range. △ Less

Submitted 20 December, 2022; originally announced December 2022.

High fusion computers: The IoTs, edges, data centers, and humans-in-the-loop as a computer

Authors: Wanling Gao, Lei Wang, Mingyu Chen, Jin Xiong, Chunjie Luo, Wenli Zhang, Yunyou Huang, Weiping Li, Guoxin Kang, Chen Zheng, Biwei Xie, Shaopeng Dai, Qian He, Hainan Ye, Yungang Bao, Jianfeng Zhan

Abstract: Emerging and future applications rely heavily upon systems consisting of Internet of Things (IoT), edges, data centers, and humans-in-the-loop. Significantly different from warehouse-scale computers that serve independent concurrent user requests, this new class of computer systems directly interacts with the physical world, considering humans an essential part and performing safety-critical and m… ▽ More Emerging and future applications rely heavily upon systems consisting of Internet of Things (IoT), edges, data centers, and humans-in-the-loop. Significantly different from warehouse-scale computers that serve independent concurrent user requests, this new class of computer systems directly interacts with the physical world, considering humans an essential part and performing safety-critical and mission-critical operations; their computations have intertwined dependencies between not only adjacent execution loops but also actions or decisions triggered by IoTs, edge, datacenters, or humans-in-the-loop; the systems must first satisfy the accuracy metric in predicting, interpreting, or taking action before meeting the performance goal under different cases. This article argues we need a paradigm shift to reconstruct the IoTs, edges, data centers, and humans-in-the-loop as a computer rather than a distributed system. We coin a new term, high fusion computers (HFCs), to describe this class of systems. The fusion in the term has two implications: fusing IoTs, edges, data centers, and humans-in-the-loop as a computer, fusing the physical and digital worlds through HFC systems. HFC is a pivotal case of the open-source computer systems initiative. We laid out the challenges, plan, and call for uniting our community's wisdom and actions to address the HFC challenges. Everything, including the source code, will be publicly available from the project homepage: https://www.computercouncil.org/HFC/. △ Less

Submitted 18 November, 2022; originally announced December 2022.

Comments: This paper has been published in BenchCouncil Transactions on Benchmarks, Standards and Evaluations (TBench). Link: https://www.sciencedirect.com/science/article/pii/S277248592200062X

Journal ref: BenchCouncil Transactions on Benchmarks, Standards and Evaluations (2022)

Reinforced Genetic Algorithm for Structure-based Drug Design

Authors: Tianfan Fu, Wenhao Gao, Connor W. Coley, Jimeng Sun

Abstract: Structure-based drug design (SBDD) aims to discover drug candidates by finding molecules (ligands) that bind tightly to a disease-related protein (targets), which is the primary approach to computer-aided drug discovery. Recently, applying deep generative models for three-dimensional (3D) molecular design conditioned on protein pockets to solve SBDD has attracted much attention, but their formulat… ▽ More Structure-based drug design (SBDD) aims to discover drug candidates by finding molecules (ligands) that bind tightly to a disease-related protein (targets), which is the primary approach to computer-aided drug discovery. Recently, applying deep generative models for three-dimensional (3D) molecular design conditioned on protein pockets to solve SBDD has attracted much attention, but their formulation as probabilistic modeling often leads to unsatisfactory optimization performance. On the other hand, traditional combinatorial optimization methods such as genetic algorithms (GA) have demonstrated state-of-the-art performance in various molecular optimization tasks. However, they do not utilize protein target structure to inform design steps but rely on a random-walk-like exploration, which leads to unstable performance and no knowledge transfer between different tasks despite the similar binding physics. To achieve a more stable and efficient SBDD, we propose Reinforced Genetic Algorithm (RGA) that uses neural models to prioritize the profitable design steps and suppress random-walk behavior. The neural models take the 3D structure of the targets and ligands as inputs and are pre-trained using native complex structures to utilize the knowledge of the shared binding physics from different targets and then fine-tuned during optimization. We conduct thorough empirical studies on optimizing binding affinity to various disease targets and show that RGA outperforms the baselines in terms of docking scores and is more robust to random initializations. The ablation study also indicates that the training on different targets helps improve performance by leveraging the shared underlying physics of the binding processes. The code is available at https://github.com/futianfan/reinforced-genetic-algorithm. △ Less

Submitted 28 November, 2022; originally announced November 2022.

Understanding binary phase separation towards Cu-C nanocrystalline-amorphous composites

Authors: Jiajian Guan, Qin Jiang, Yue He, Xu Zhang, Bin Liao, Wei Gao, Lizhao Qin

Abstract: The nanocrystalline-amorphous textures are commonly observed in the coatings synthesized by energetic deposition.This work reports a theoretical study towards binary Cu-C phase separation.By performing a MD simulation using the LAMMPS instead of classical PFK methodology, we theoretically explained how the initial pressure and Cu concentration fundamentally determines Cu nanocrystalline's final mo… ▽ More The nanocrystalline-amorphous textures are commonly observed in the coatings synthesized by energetic deposition.This work reports a theoretical study towards binary Cu-C phase separation.By performing a MD simulation using the LAMMPS instead of classical PFK methodology, we theoretically explained how the initial pressure and Cu concentration fundamentally determines Cu nanocrystalline's final morphology and grain size,which gives a novel insight into binary phase separation and the evolution mechanism of nanocrystalline-amorphous structures during energetic deposition. △ Less

Submitted 16 July, 2023; v1 submitted 27 November, 2022; originally announced November 2022.

Comments: 12 pages, 4 figures

Supervised Pretraining for Molecular Force Fields and Properties Prediction

Authors: Xiang Gao, Weihao Gao, Wenzhi Xiao, Zhirui Wang, Chong Wang, Liang Xiang

Abstract: Machine learning approaches have become popular for molecular modeling tasks, including molecular force fields and properties prediction. Traditional supervised learning methods suffer from scarcity of labeled data for particular tasks, motivating the use of large-scale dataset for other relevant tasks. We propose to pretrain neural networks on a dataset of 86 millions of molecules with atom charg… ▽ More Machine learning approaches have become popular for molecular modeling tasks, including molecular force fields and properties prediction. Traditional supervised learning methods suffer from scarcity of labeled data for particular tasks, motivating the use of large-scale dataset for other relevant tasks. We propose to pretrain neural networks on a dataset of 86 millions of molecules with atom charges and 3D geometries as inputs and molecular energies as labels. Experiments show that, compared to training from scratch, fine-tuning the pretrained model can significantly improve the performance for seven molecular property prediction tasks and two force field tasks. We also demonstrate that the learned representations from the pretrained model contain adequate information about molecular structures, by showing that linear probing of the representations can predict many molecular information including atom types, interatomic distances, class of molecular scaffolds, and existence of molecular fragments. Our results show that supervised pretraining is a promising research direction in molecular modeling △ Less

Submitted 23 November, 2022; originally announced November 2022.

Comments: AI4Science Workshop at NeurIPS 2022

Learning Regularized Positional Encoding for Molecular Prediction

Authors: Xiang Gao, Weihao Gao, Wenzhi Xiao, Zhirui Wang, Chong Wang, Liang Xiang

Abstract: Machine learning has become a promising approach for molecular modeling. Positional quantities, such as interatomic distances and bond angles, play a crucial role in molecule physics. The existing works rely on careful manual design of their representation. To model the complex nonlinearity in predicting molecular properties in an more end-to-end approach, we propose to encode the positional quant… ▽ More Machine learning has become a promising approach for molecular modeling. Positional quantities, such as interatomic distances and bond angles, play a crucial role in molecule physics. The existing works rely on careful manual design of their representation. To model the complex nonlinearity in predicting molecular properties in an more end-to-end approach, we propose to encode the positional quantities with a learnable embedding that is continuous and differentiable. A regularization technique is employed to encourage embedding smoothness along the physical dimension. We experiment with a variety of molecular property and force field prediction tasks. Improved performance is observed for three different model architectures after plugging in the proposed positional encoding method. In addition, the learned positional encoding allows easier physics-based interpretation. We observe that tasks of similar physics have the similar learned positional encoding. △ Less

Submitted 23 November, 2022; originally announced November 2022.

Comments: AI4Science Workshop at NeurIPS 2022

Perceptual Video Coding for Machines via Satisfied Machine Ratio Modeling

Authors: Qi Zhang, Shanshe Wang, Xinfeng Zhang, Chuanmin Jia, Zhao Wang, Siwei Ma, Wen Gao

Abstract: Video Coding for Machines (VCM) aims to compress visual signals for machine analysis. However, existing methods only consider a few machines, neglecting the majority. Moreover, the machine's perceptual characteristics are not leveraged effectively, resulting in suboptimal compression efficiency. To overcome these limitations, this paper introduces Satisfied Machine Ratio (SMR), a metric that stati… ▽ More Video Coding for Machines (VCM) aims to compress visual signals for machine analysis. However, existing methods only consider a few machines, neglecting the majority. Moreover, the machine's perceptual characteristics are not leveraged effectively, resulting in suboptimal compression efficiency. To overcome these limitations, this paper introduces Satisfied Machine Ratio (SMR), a metric that statistically evaluates the perceptual quality of compressed images and videos for machines by aggregating satisfaction scores from them. Each score is derived from machine perceptual differences between original and compressed images. Targeting image classification and object detection tasks, we build two representative machine libraries for SMR annotation and create a large-scale SMR dataset to facilitate SMR studies. We then propose an SMR prediction model based on the correlation between deep feature differences and SMR. Furthermore, we introduce an auxiliary task to increase the prediction accuracy by predicting the SMR difference between two images in different quality. Extensive experiments demonstrate that SMR models significantly improve compression performance for machines and exhibit robust generalizability on unseen machines, codecs, datasets, and frame types. SMR enables perceptual coding for machines and propels VCM from specificity to generality. Code is available at https://github.com/ywwynm/SMR. △ Less

Submitted 9 January, 2024; v1 submitted 12 November, 2022; originally announced November 2022.

Robust Quantum Griffiths Singularity at above 1.5 Kelvin in Nitride Thin Films

Authors: Xiaoni Wang, Lijie Wang, Yixin Liu, Fan Chen, Wanpeng Gao, Yu Wu, Zulei Xu, Wei Peng, Zhen Wang, Zengfeng Di, Wei Li, Gang Mu, Zhirong Lin

Abstract: Quantum Griffiths singularity (QGS), which is closely correlated with the quenched disorder, is characterized by the divergence of the dynamical critical exponent and the presence of activated scaling behavior. Typically such a quantum phenomenon is rather rare and only observed in extremely low temperatures. Here we report the experimental observation of a robust QGS in nitride thin films, NbN, w… ▽ More Quantum Griffiths singularity (QGS), which is closely correlated with the quenched disorder, is characterized by the divergence of the dynamical critical exponent and the presence of activated scaling behavior. Typically such a quantum phenomenon is rather rare and only observed in extremely low temperatures. Here we report the experimental observation of a robust QGS in nitride thin films, NbN, which survives in a rather high temperature range. The electrical transport propertied were measured under the magnetic field up to 12 T. The field induced superconductor-metal transitions were observed with the continuously changed transition points with the decrease of temperature. The dynamical critical exponent based on the conventional power-law scaling reveals a divergent trend when approaching the the low temperature limit. Moreover, the temperature and field dependence of sheet resistance can be described by the activated scaling analysis in the temperature region up to 1.6 K $\leq T \leq$ 4.0 K. We argue that the robustness of QGS in the present system originates from the Pauli paramagnetic effect due to the strong coupling between the magnetic field and the spin degree of freedom. △ Less

Submitted 10 November, 2022; originally announced November 2022.

Comments: 6 pages, 4 figures

Journal ref: Phys. Rev. B 107, 094509 (2023)

Investigation of the Pauli paramagnetic effect in systematically tuned NbN thin films

Authors: Xiaoni Wang, Lijie Wang, Yixin Liu, Wanpeng Gao, Yu Wu, Zulei Xu, Hua Jin, Lu Zhang, Wei Peng, Zhen Wang, Wei Li, Gang Mu, Zhirong Lin

Abstract: Superconductivity and the normal-state properties of NbN films can be tuned in a wide range, supplying a suitable platform to investigate the systematical evolution of the superconducting performances. Herein, we report the upper critical field of NbN films in both the vertical ($B\perp$ film) and parallel ($B\parallel$ film) orientations over a wide temperature range. Eight samples with the super… ▽ More Superconductivity and the normal-state properties of NbN films can be tuned in a wide range, supplying a suitable platform to investigate the systematical evolution of the superconducting performances. Herein, we report the upper critical field of NbN films in both the vertical ($B\perp$ film) and parallel ($B\parallel$ film) orientations over a wide temperature range. Eight samples with the superconducting critical temperature $T_c$ ranging from 2.5 K to 9.8 K are studied. Meanwhile, the normal-state resistivity is tuned by more than six times by changing the conditions of the film growth. It is found that the magnitudes of the upper critical field in both field directions ($B_{c2}^{\perp}$ and $B_{c2}^{\parallel}$) exceed the paramagnetic limiting field $B_p$. The temperature dependent $B_{c2}^{\perp}$ can be described by the extended Werthamer--Helfand--Hohenberg (WHH) model considering the Pauli spin paramagnetism. Meanwhile, the $B_{c2}^{\parallel}$-$T$ data shows the feature of two-dimensional superconductivity in the temperature range near $T_c$. The evolution of the obtained Maki parameter with other parameters, such as the slope of the upper critical field near $T_c$ and the normal-state resistivity, are discussed to reveal the characteristics of the Pauli paramagnetic effect in this system. △ Less

Submitted 10 November, 2022; originally announced November 2022.

Comments: 7 pages, 6 figures

Journal ref: Physica C 606, 1354223 (2023)

Giant excitonic effects in bulk vacancy-ordered double perovskites

Authors: Fan Zhang, Weiwei Gao, Greis J. Cruz, Yi-yang Sun, Peihong Zhang, Jijun Zhao

Abstract: Using first-principles GW plus Bethe-Salpeter equation calculations, we identify anomalously strong excitonic effects in several vacancy-ordered double perovskites Cs2MX6 (M = Ti, Zr; X = I, Br). Giant exciton binding energies about 1 eV are found in these moderate-gap, inorganic bulk semiconductors, pushing the limit of our understanding of electron-hole (e-h) interaction and exciton formation in… ▽ More Using first-principles GW plus Bethe-Salpeter equation calculations, we identify anomalously strong excitonic effects in several vacancy-ordered double perovskites Cs2MX6 (M = Ti, Zr; X = I, Br). Giant exciton binding energies about 1 eV are found in these moderate-gap, inorganic bulk semiconductors, pushing the limit of our understanding of electron-hole (e-h) interaction and exciton formation in solids. Not only are the exciton binding energies extremely large compared with any other moderate-gap bulk semiconductors, but they are also larger than typical 2D semiconductors with comparable quasiparticle gaps. Our calculated lowest bright exciton energy agrees well with the experimental optical band gap. The low-energy excitons closely resemble the Frenkel excitons in molecular crystals, as they are highly localized in a single [MX6]2- octahedron and extended in the reciprocal space. The weak dielectric screening effects and the nearly flat frontier electronic bands, which are derived from the weakly bonded [MX6]2- units, together explain the significant excitonic effects. Spin-orbit coupling effects play a crucial role in red-shifting the lowest bright exciton by mixing up spin-singlet and spin-triplet excitons, while exciton-phonon coupling effects have minor impacts on the strong exciton binding energies. △ Less

Submitted 9 November, 2022; originally announced November 2022.

DialogConv: A Lightweight Fully Convolutional Network for Multi-view Response Selection

Authors: Yongkang Liu, Shi Feng, Wei Gao, Daling Wang, Yifei Zhang

Abstract: Current end-to-end retrieval-based dialogue systems are mainly based on Recurrent Neural Networks or Transformers with attention mechanisms. Although promising results have been achieved, these models often suffer from slow inference or huge number of parameters. In this paper, we propose a novel lightweight fully convolutional architecture, called DialogConv, for response selection. DialogConv is… ▽ More Current end-to-end retrieval-based dialogue systems are mainly based on Recurrent Neural Networks or Transformers with attention mechanisms. Although promising results have been achieved, these models often suffer from slow inference or huge number of parameters. In this paper, we propose a novel lightweight fully convolutional architecture, called DialogConv, for response selection. DialogConv is exclusively built on top of convolution to extract matching features of context and response. Dialogues are modeled in 3D views, where DialogConv performs convolution operations on embedding view, word view and utterance view to capture richer semantic information from multiple contextual views. On the four benchmark datasets, compared with state-of-the-art baselines, DialogConv is on average about 8.5x smaller in size, and 79.39x and 10.64x faster on CPU and GPU devices, respectively. At the same time, DialogConv achieves the competitive effectiveness of response selection. △ Less

Submitted 25 October, 2022; originally announced October 2022.

Comments: EMNLP2022

TetGAN: A Convolutional Neural Network for Tetrahedral Mesh Generation

Authors: William Gao, April Wang, Gal Metzer, Raymond A. Yeh, Rana Hanocka

Abstract: We present TetGAN, a convolutional neural network designed to generate tetrahedral meshes. We represent shapes using an irregular tetrahedral grid which encodes an occupancy and displacement field. Our formulation enables defining tetrahedral convolution, pooling, and upsampling operations to synthesize explicit mesh connectivity with variable topological genus. The proposed neural network layers… ▽ More We present TetGAN, a convolutional neural network designed to generate tetrahedral meshes. We represent shapes using an irregular tetrahedral grid which encodes an occupancy and displacement field. Our formulation enables defining tetrahedral convolution, pooling, and upsampling operations to synthesize explicit mesh connectivity with variable topological genus. The proposed neural network layers learn deep features over each tetrahedron and learn to extract patterns within spatial regions across multiple scales. We illustrate the capabilities of our technique to encode tetrahedral meshes into a semantically meaningful latent-space which can be used for shape editing and synthesis. Our project page is at https://threedle.github.io/tetGAN/. △ Less

Submitted 11 October, 2022; originally announced October 2022.

Comments: Accepted to BMVC2022

Physics-aware Differentiable Discrete Codesign for Diffractive Optical Neural Networks

Authors: Yingjie Li, Ruiyang Chen, Weilu Gao, Cunxi Yu

Abstract: Diffractive optical neural networks (DONNs) have attracted lots of attention as they bring significant advantages in terms of power efficiency, parallelism, and computational speed compared with conventional deep neural networks (DNNs), which have intrinsic limitations when implemented on digital platforms. However, inversely mapping algorithm-trained physical model parameters onto real-world opti… ▽ More Diffractive optical neural networks (DONNs) have attracted lots of attention as they bring significant advantages in terms of power efficiency, parallelism, and computational speed compared with conventional deep neural networks (DNNs), which have intrinsic limitations when implemented on digital platforms. However, inversely mapping algorithm-trained physical model parameters onto real-world optical devices with discrete values is a non-trivial task as existing optical devices have non-unified discrete levels and non-monotonic properties. This work proposes a novel device-to-system hardware-software codesign framework, which enables efficient physics-aware training of DONNs w.r.t arbitrary experimental measured optical devices across layers. Specifically, Gumbel-Softmax is employed to enable differentiable discrete mapping from real-world device parameters into the forward function of DONNs, where the physical parameters in DONNs can be trained by simply minimizing the loss function of the ML task. The results have demonstrated that our proposed framework offers significant advantages over conventional quantization-based methods, especially with low-precision optical devices. Finally, the proposed algorithm is fully verified with physical experimental optical systems in low-precision settings. △ Less

Submitted 28 September, 2022; originally announced September 2022.

Comments: International Conference on Computer-Aided Design (ICCAD'2022) To appear

Adaptive Dynamic Programming and Data-Driven Cooperative Optimal Output Regulation with Adaptive Observers

Authors: Omar Qasem, Khalid Jebari, Weinan Gao

Abstract: In this paper, a novel adaptive optimal control strategy is proposed to achieve the cooperative optimal output regulation of continuous-time linear multi-agent systems based on adaptive dynamic programming (ADP). The proposed method is different from those in the existing literature of ADP and cooperative output regulation in the sense that the knowledge of the exosystem dynamics is not required i… ▽ More In this paper, a novel adaptive optimal control strategy is proposed to achieve the cooperative optimal output regulation of continuous-time linear multi-agent systems based on adaptive dynamic programming (ADP). The proposed method is different from those in the existing literature of ADP and cooperative output regulation in the sense that the knowledge of the exosystem dynamics is not required in the design of the exostate observers for those agents with no direct access to the exosystem. Moreover, an optimal control policy is obtained without the prior knowledge of the modeling information of any agent while achieving the cooperative output regulation. Instead, we use the state/input information along the trajectories of the underlying dynamical systems and the estimated exostates to learn the optimal control policy. Simulation results show the efficacy of the proposed algorithm, where both estimation errors of exosystem matrix and exostates, and the tracking errors converge to zero in an optimal sense, which solves the cooperative optimal output regulation problem. △ Less

Submitted 25 September, 2022; originally announced September 2022.

Comments: This work has been accepted for presentation at IEEE CDC 2022

Improving Conversational Recommender System via Contextual and Time-Aware Modeling with Less Domain-Specific Knowledge

Authors: Lingzhi Wang, Shafiq Joty, Wei Gao, Xingshan Zeng, Kam-Fai Wong

Abstract: Conversational Recommender Systems (CRS) has become an emerging research topic seeking to perform recommendations through interactive conversations, which generally consist of generation and recommendation modules. Prior work on CRS tends to incorporate more external and domain-specific knowledge like item reviews to enhance performance. Despite the fact that the collection and annotation of the e… ▽ More Conversational Recommender Systems (CRS) has become an emerging research topic seeking to perform recommendations through interactive conversations, which generally consist of generation and recommendation modules. Prior work on CRS tends to incorporate more external and domain-specific knowledge like item reviews to enhance performance. Despite the fact that the collection and annotation of the external domain-specific information needs much human effort and degenerates the generalizability, too much extra knowledge introduces more difficulty to balance among them. Therefore, we propose to fully discover and extract internal knowledge from the context. We capture both entity-level and contextual-level representations to jointly model user preferences for the recommendation, where a time-aware attention is designed to emphasize the recently appeared items in entity-level representations. We further use the pre-trained BART to initialize the generation module to alleviate the data scarcity and enhance the context modeling. In addition to conducting experiments on a popular dataset (ReDial), we also include a multi-domain dataset (OpenDialKG) to show the effectiveness of our model. Experiments on both datasets show that our model achieves better performance on most evaluation metrics with less external knowledge and generalizes well to other domains. Additional analyses on the recommendation and generation tasks demonstrate the effectiveness of our model in different scenarios. △ Less

Submitted 22 September, 2022; originally announced September 2022.

Strong Dzyaloshinskii-Moriya Interaction in Monolayer CrI$_3$ on Metal Substrates

Authors: Fan Zhang, Xueao Li, Yabei Wu, Xiaolong Wang, Jijun Zhao, Weiwei Gao

Abstract: Dzyaloshinskii-Moriya interaction (DMI) is the primary mechanism for realizing real-space chiral spin textures, which are regarded as key components for the next-generation spintronics. However, DMI arises from a perturbation term of the spin-orbit interaction and is usually weak in pristine magnetic semiconductors. To date, large DMI and the resulting skyrmions are only realized in a few material… ▽ More Dzyaloshinskii-Moriya interaction (DMI) is the primary mechanism for realizing real-space chiral spin textures, which are regarded as key components for the next-generation spintronics. However, DMI arises from a perturbation term of the spin-orbit interaction and is usually weak in pristine magnetic semiconductors. To date, large DMI and the resulting skyrmions are only realized in a few materials under stringent conditions. Using first-principles calculations, we demonstrate that significant DMI occurs between nearest-neighbor Cr atoms in two-dimensional (2D) magnetic semiconductor CrI$_3$ on Au or Cu substrates. This exceptionally strong DMI is generated by the interfacial charge transfer and weak chemical interactions between chromium halides and metal substrates, which break the spatial inversion symmetry. These findings highlight the significance of substrate effects in 2D magnets and expand the inventory of feasible materials with strong DMI. △ Less

Submitted 17 September, 2022; originally announced September 2022.

Out-of-plane polarization and topological magnetic vortices in multiferroic CrPSe$_3$

Authors: Weiwei Gao, Jijun Zhao, James R. Chelikowsky

Abstract: Two-dimensional (2D) multiferroic materials are ideal systems for exploring new coupling mechanisms between different ferroic orders and producing novel quantum phenomena with potential applications. We employed first-principles density functional theory calculations to discover intrinsic ferroelectric and anti-ferroelectric phases of CrPSe$_3$, which show ferromagnetic order and compete with the… ▽ More Two-dimensional (2D) multiferroic materials are ideal systems for exploring new coupling mechanisms between different ferroic orders and producing novel quantum phenomena with potential applications. We employed first-principles density functional theory calculations to discover intrinsic ferroelectric and anti-ferroelectric phases of CrPSe$_3$, which show ferromagnetic order and compete with the centrosymmetric phase with an antiferromagnetic order. Our analysis show that the electrical dipoles of such type-I multiferroic phases come from the out-of-plane displacements of phosphorus ions due to the stereochemically active lone pairs. The coupling between polar and magnetic orders creates the opportunity for tunning the magnetic ground state by switching from the centrosymmetric to the ferroelectric phase using an out-of-plane electric field. In ferroelectric and antiferroelectric phases, the combination of easy-plane anisotropy and Dzyaloshinskii-Moriya interactions (DMI) indicate they can host topological magnetic vortices like meron pairs. △ Less

Submitted 17 September, 2022; originally announced September 2022.

Comments: 7 pages, 3 figures, and the supplementary material

Deep Lossy Plus Residual Coding for Lossless and Near-lossless Image Compression

Authors: Yuanchao Bai, Xianming Liu, Kai Wang, Xiangyang Ji, Xiaolin Wu, Wen Gao

Abstract: Lossless and near-lossless image compression is of paramount importance to professional users in many technical fields, such as medicine, remote sensing, precision engineering and scientific research. But despite rapidly growing research interests in learning-based image compression, no published method offers both lossless and near-lossless modes. In this paper, we propose a unified and powerful… ▽ More Lossless and near-lossless image compression is of paramount importance to professional users in many technical fields, such as medicine, remote sensing, precision engineering and scientific research. But despite rapidly growing research interests in learning-based image compression, no published method offers both lossless and near-lossless modes. In this paper, we propose a unified and powerful deep lossy plus residual (DLPR) coding framework for both lossless and near-lossless image compression. In the lossless mode, the DLPR coding system first performs lossy compression and then lossless coding of residuals. We solve the joint lossy and residual compression problem in the approach of VAEs, and add autoregressive context modeling of the residuals to enhance lossless compression performance. In the near-lossless mode, we quantize the original residuals to satisfy a given $\ell_\infty$ error bound, and propose a scalable near-lossless compression scheme that works for variable $\ell_\infty$ bounds instead of training multiple networks. To expedite the DLPR coding, we increase the degree of algorithm parallelization by a novel design of coding context, and accelerate the entropy coding with adaptive residual interval. Experimental results demonstrate that the DLPR coding system achieves both the state-of-the-art lossless and near-lossless image compression performance with competitive coding speed. △ Less

Submitted 10 January, 2024; v1 submitted 11 September, 2022; originally announced September 2022.

Comments: manuscript accepted by TPAMI, source code:https://github.com/BYchao100/Deep-Lossy-Plus-Residual-Coding

A mixed precision Jacobi SVD algorithm

Authors: Weiguo Gao, Yuxin Ma, Meiyue Shao

Abstract: We propose a mixed precision Jacobi algorithm for computing the singular value decomposition (SVD) of a dense matrix. After appropriate preconditioning, the proposed algorithm computes the SVD in a lower precision as an initial guess, and then performs one-sided Jacobi rotations in the working precision as iterative refinement. By carefully transforming a lower precision solution to a higher preci… ▽ More We propose a mixed precision Jacobi algorithm for computing the singular value decomposition (SVD) of a dense matrix. After appropriate preconditioning, the proposed algorithm computes the SVD in a lower precision as an initial guess, and then performs one-sided Jacobi rotations in the working precision as iterative refinement. By carefully transforming a lower precision solution to a higher precision one, our algorithm achieves about 2 times speedup on the x86-64 architecture compared to the usual one-sided Jacobi SVD algorithm in LAPACK, without sacrificing the accuracy. △ Less

Submitted 10 September, 2022; originally announced September 2022.

Comments: 23 pages, 12 figures

MSC Class: 65F15

Cooperative trajectory planning algorithm of USV-UAV with hull dynamic constraints

Authors: Tao Huang, Zhe Chen, Wang Gao, Zhenfeng Xue, Yong Liu

Abstract: Efficient trajectory generation in complex dynamic environments remains an open problem in the unmanned surface vehicle (USV). The perception of the USV is usually interfered with by the swing of the hull and the ambient weather, making it challenging to plan the optimal USV trajectories. In this paper, a cooperative trajectory planning algorithm for the coupled USV-UAV system is proposed to ensur… ▽ More Efficient trajectory generation in complex dynamic environments remains an open problem in the unmanned surface vehicle (USV). The perception of the USV is usually interfered with by the swing of the hull and the ambient weather, making it challenging to plan the optimal USV trajectories. In this paper, a cooperative trajectory planning algorithm for the coupled USV-UAV system is proposed to ensure that USV can execute a safe and smooth path in the process of autonomous advance in multi-obstacle maps. Specifically, the unmanned aerial vehicle (UAV) plays the role of a flight sensor, providing real-time global map and obstacle information with a lightweight semantic segmentation network and 3D projection transformation. And then, an initial obstacle avoidance trajectory is generated by a graph-based search method. Concerning the unique under-actuated kinematic characteristics of the USV, a numerical optimization method based on hull dynamic constraints is introduced to make the trajectory easier to be tracked for motion control. Finally, a motion control method based on NMPC with the lowest energy consumption constraint during execution is proposed. Experimental results verify the effectiveness of the whole system, and the generated trajectory is locally optimal for USV with considerable tracking accuracy. △ Less

Submitted 30 January, 2023; v1 submitted 7 September, 2022; originally announced September 2022.

Comments: 11 pages, 9 figures

Cross Modal Compression: Towards Human-comprehensible Semantic Compression

Authors: Jiguo Li, Chuanmin Jia, Xinfeng Zhang, Siwei Ma, Wen Gao

Abstract: Traditional image/video compression aims to reduce the transmission/storage cost with signal fidelity as high as possible. However, with the increasing demand for machine analysis and semantic monitoring in recent years, semantic fidelity rather than signal fidelity is becoming another emerging concern in image/video compression. With the recent advances in cross modal translation and generation,… ▽ More Traditional image/video compression aims to reduce the transmission/storage cost with signal fidelity as high as possible. However, with the increasing demand for machine analysis and semantic monitoring in recent years, semantic fidelity rather than signal fidelity is becoming another emerging concern in image/video compression. With the recent advances in cross modal translation and generation, in this paper, we propose the cross modal compression~(CMC), a semantic compression framework for visual data, to transform the high redundant visual data~(such as image, video, etc.) into a compact, human-comprehensible domain~(such as text, sketch, semantic map, attributions, etc.), while preserving the semantic. Specifically, we first formulate the CMC problem as a rate-distortion optimization problem. Secondly, we investigate the relationship with the traditional image/video compression and the recent feature compression frameworks, showing the difference between our CMC and these prior frameworks. Then we propose a novel paradigm for CMC to demonstrate its effectiveness. The qualitative and quantitative results show that our proposed CMC can achieve encouraging reconstructed results with an ultrahigh compression ratio, showing better compression performance than the widely used JPEG baseline. △ Less

Submitted 6 September, 2022; originally announced September 2022.

Comments: 10 pages, 4 figures

Decentralized Natural Policy Gradient with Variance Reduction for Collaborative Multi-Agent Reinforcement Learning

Authors: Jinchi Chen, Jie Feng, Weiguo Gao, Ke Wei

Abstract: This paper studies a policy optimization problem arising from collaborative multi-agent reinforcement learning in a decentralized setting where agents communicate with their neighbors over an undirected graph to maximize the sum of their cumulative rewards. A novel decentralized natural policy gradient method, dubbed Momentum-based Decentralized Natural Policy Gradient (MDNPG), is proposed, which… ▽ More This paper studies a policy optimization problem arising from collaborative multi-agent reinforcement learning in a decentralized setting where agents communicate with their neighbors over an undirected graph to maximize the sum of their cumulative rewards. A novel decentralized natural policy gradient method, dubbed Momentum-based Decentralized Natural Policy Gradient (MDNPG), is proposed, which incorporates natural gradient, momentum-based variance reduction, and gradient tracking into the decentralized stochastic gradient ascent framework. The $\mathcal{O}(n^{-1}ε^{-3})$ sample complexity for MDNPG to converge to an $ε$-stationary point has been established under standard assumptions, where $n$ is the number of agents. It indicates that MDNPG can achieve the optimal convergence rate for decentralized policy gradient methods and possesses a linear speedup in contrast to centralized optimization methods. Moreover, superior empirical performance of MDNPG over other state-of-the-art algorithms has been demonstrated by extensive numerical experiments. △ Less

Submitted 5 September, 2022; originally announced September 2022.

An Enhanced ADMM-based Interior Point Method for Linear and Conic Optimization

Authors: Qi Deng, Qing Feng, Wenzhi Gao, Dongdong Ge, Bo Jiang, Yuntian Jiang, Jingsong Liu, Tianhao Liu, Chenyu Xue, Yinyu Ye, Chuwen Zhang

Abstract: The ADMM-based interior point (ABIP, Lin et al. 2021) method is a hybrid algorithm that effectively combines interior point method (IPM) and first-order methods to achieve a performance boost in large-scale linear optimization. Different from traditional IPM that relies on computationally intensive Newton steps, the ABIP method applies the alternating direction method of multipliers (ADMM) to appr… ▽ More The ADMM-based interior point (ABIP, Lin et al. 2021) method is a hybrid algorithm that effectively combines interior point method (IPM) and first-order methods to achieve a performance boost in large-scale linear optimization. Different from traditional IPM that relies on computationally intensive Newton steps, the ABIP method applies the alternating direction method of multipliers (ADMM) to approximately solve the barrier penalized problem. However, similar to other first-order methods, this technique remains sensitive to condition number and inverse precision. In this paper, we provide an enhanced ABIP method with multiple improvements. Firstly, we develop an ABIP method to solve the general linear conic optimization and establish the associated iteration complexity. Secondly, inspired by some existing methods, we develop different implementation strategies for ABIP method, which substantially improve its performance in linear optimization. Finally, we conduct extensive numerical experiments in both synthetic and real-world datasets to demonstrate the empirical advantage of our developments. In particular, the enhanced ABIP method achieves a 5.8x reduction in the geometric mean of run time on $105$ selected LP instances from Netlib, and it exhibits advantages in certain structured problems such as SVM and PageRank. However, the enhanced ABIP method still falls behind commercial solvers in many benchmarks, especially when high accuracy is desired. We posit that it can serve as a complementary tool alongside well-established solvers. △ Less

Submitted 6 April, 2024; v1 submitted 5 September, 2022; originally announced September 2022.