Showing 51–100 of 158 results for author: Bi, J

Heterogeneous Graph Sparsification for Efficient Representation Learning

Authors: Chandan Chunduru, Chun Jiang Zhu, Blake Gains, Jinbo Bi

Abstract: Graph sparsification is a powerful tool to approximate an arbitrary graph and has been used in machine learning over homogeneous graphs. In heterogeneous graphs such as knowledge graphs, however, sparsification has not been systematically exploited to improve efficiency of learning tasks. In this work, we initiate the study on heterogeneous graph sparsification and develop sampling-based algorithm… ▽ More Graph sparsification is a powerful tool to approximate an arbitrary graph and has been used in machine learning over homogeneous graphs. In heterogeneous graphs such as knowledge graphs, however, sparsification has not been systematically exploited to improve efficiency of learning tasks. In this work, we initiate the study on heterogeneous graph sparsification and develop sampling-based algorithms for constructing sparsifiers that are provably sparse and preserve important information in the original graphs. We have performed extensive experiments to confirm that the proposed method can improve time and space complexities of representation learning while achieving comparable, or even better performance in subsequent graph learning tasks based on the learned embedding. △ Less

Submitted 14 November, 2022; originally announced November 2022.

Comments: Accepted and to appear in IEEE BIBM 2022 Workshop

Safety-Constrained Policy Transfer with Successor Features

Authors: Zeyu Feng, Bowen Zhang, Jianxin Bi, Harold Soh

Abstract: In this work, we focus on the problem of safe policy transfer in reinforcement learning: we seek to leverage existing policies when learning a new task with specified constraints. This problem is important for safety-critical applications where interactions are costly and unconstrained policies can lead to undesirable or dangerous outcomes, e.g., with physical robots that interact with humans. We… ▽ More In this work, we focus on the problem of safe policy transfer in reinforcement learning: we seek to leverage existing policies when learning a new task with specified constraints. This problem is important for safety-critical applications where interactions are costly and unconstrained policies can lead to undesirable or dangerous outcomes, e.g., with physical robots that interact with humans. We propose a Constrained Markov Decision Process (CMDP) formulation that simultaneously enables the transfer of policies and adherence to safety constraints. Our formulation cleanly separates task goals from safety considerations and permits the specification of a wide variety of constraints. Our approach relies on a novel extension of generalized policy improvement to constrained settings via a Lagrangian formulation. We devise a dual optimization algorithm that estimates the optimal dual variable of a target task, thus enabling safe transfer of policies derived from successor features learned on source tasks. Our experiments in simulated domains show that our approach is effective; it visits unsafe states less frequently and outperforms alternative state-of-the-art methods when taking safety constraints into account. △ Less

Submitted 10 November, 2022; originally announced November 2022.

Gap-opening Planets Make Dust Rings Wider

Authors: Jiaqing Bi, Min-Kai Lin, Ruobing Dong

Abstract: As one of the most commonly observed disk substructures, dust rings from high-resolution disk surveys appear to have different radial widths. Recent observations on PDS 70 and AB Aur reveal not only planets in the disk, but also the accompanying wide dust rings. We use three-dimensional dust-and-gas disk simulations to study whether gap-opening planets are responsible for the large ring width in d… ▽ More As one of the most commonly observed disk substructures, dust rings from high-resolution disk surveys appear to have different radial widths. Recent observations on PDS 70 and AB Aur reveal not only planets in the disk, but also the accompanying wide dust rings. We use three-dimensional dust-and-gas disk simulations to study whether gap-opening planets are responsible for the large ring width in disk observations. We find that gap-opening planets can widen rings of dust trapped at the pressure bump via planetary perturbations, even with the mid-plane dust-to-gas ratio approaching order unity and with the dust back-reaction accounted for. We show that the planet-related widening effect of dust rings can be quantified using diffusion-advection theory, and provide a generalized criterion for an equilibrated dust ring width in three-dimensional disk models. We also suggest that the ring width can be estimated using the gas turbulent viscosity $α_{\rm turb}$, but with cautions about the Schmidt number greater than order unity. △ Less

Submitted 20 October, 2022; originally announced October 2022.

Comments: 17 pages, 6+3 figures, 2 tables, accepted in ApJ

Learning Generalizable Models for Vehicle Routing Problems via Knowledge Distillation

Authors: Jieyi Bi, Yining Ma, Jiahai Wang, Zhiguang Cao, Jinbiao Chen, Yuan Sun, Yeow Meng Chee

Abstract: Recent neural methods for vehicle routing problems always train and test the deep models on the same instance distribution (i.e., uniform). To tackle the consequent cross-distribution generalization concerns, we bring the knowledge distillation to this field and propose an Adaptive Multi-Distribution Knowledge Distillation (AMDKD) scheme for learning more generalizable deep models. Particularly, o… ▽ More Recent neural methods for vehicle routing problems always train and test the deep models on the same instance distribution (i.e., uniform). To tackle the consequent cross-distribution generalization concerns, we bring the knowledge distillation to this field and propose an Adaptive Multi-Distribution Knowledge Distillation (AMDKD) scheme for learning more generalizable deep models. Particularly, our AMDKD leverages various knowledge from multiple teachers trained on exemplar distributions to yield a light-weight yet generalist student model. Meanwhile, we equip AMDKD with an adaptive strategy that allows the student to concentrate on difficult distributions, so as to absorb hard-to-master knowledge more effectively. Extensive experimental results show that, compared with the baseline neural methods, our AMDKD is able to achieve competitive results on both unseen in-distribution and out-of-distribution instances, which are either randomly synthesized or adopted from benchmark datasets (i.e., TSPLIB and CVRPLIB). Notably, our AMDKD is generic, and consumes less computational resources for inference. △ Less

Submitted 19 January, 2023; v1 submitted 14 October, 2022; originally announced October 2022.

Comments: Accepted at NeurIPS 2022

Auto-Encoding Goodness of Fit

Authors: Aaron Palmer, Zhiyi Chi, Derek Aguiar, Jinbo Bi

Abstract: For generative autoencoders to learn a meaningful latent representation for data generation, a careful balance must be achieved between reconstruction error and how close the distribution in the latent space is to the prior. However, this balance is challenging to achieve due to a lack of criteria that work both at the mini-batch (local) and aggregated posterior (global) level. Goodness of fit (Go… ▽ More For generative autoencoders to learn a meaningful latent representation for data generation, a careful balance must be achieved between reconstruction error and how close the distribution in the latent space is to the prior. However, this balance is challenging to achieve due to a lack of criteria that work both at the mini-batch (local) and aggregated posterior (global) level. Goodness of fit (GoF) hypothesis tests provide a measure of statistical indistinguishability between the latent distribution and a target distribution class. In this work, we develop the Goodness of Fit Autoencoder (GoFAE), which incorporates hypothesis tests at two levels. At the mini-batch level, it uses GoF test statistics as regularization objectives. At a more global level, it selects a regularization coefficient based on higher criticism, i.e., a test on the uniformity of the local GoF p-values. We justify the use of GoF tests by providing a relaxed $L_2$-Wasserstein bound on the distance between the latent distribution and target prior. We propose to use GoF tests and prove that optimization based on these tests can be done with stochastic gradient (SGD) descent on a compact Riemannian manifold. Empirically, we show that our higher criticism parameter selection procedure balances reconstruction and generation using mutual information and uniformity of p-values respectively. Finally, we show that GoFAE achieves comparable FID scores and mean squared errors with competing deep generative models while retaining statistical indistinguishability from Gaussian in the latent space based on a variety of hypothesis tests. △ Less

Submitted 12 October, 2022; originally announced October 2022.

Performances of Symmetric Loss for Private Data from Exponential Mechanism

Authors: Jing Bi, Vorapong Suppakitpaisarn

Abstract: This study explores the robustness of learning by symmetric loss on private data. Specifically, we leverage exponential mechanism (EM) on private labels. First, we theoretically re-discussed properties of EM when it is used for private learning with symmetric loss. Then, we propose numerical guidance of privacy budgets corresponding to different data scales and utility guarantees. Further, we cond… ▽ More This study explores the robustness of learning by symmetric loss on private data. Specifically, we leverage exponential mechanism (EM) on private labels. First, we theoretically re-discussed properties of EM when it is used for private learning with symmetric loss. Then, we propose numerical guidance of privacy budgets corresponding to different data scales and utility guarantees. Further, we conducted experiments on the CIFAR-10 dataset to present the traits of symmetric loss. Since EM is a more generic differential privacy (DP) technique, it being robust has the potential for it to be generalized, and to make other DP techniques more robust. △ Less

Submitted 8 October, 2022; originally announced October 2022.

Comments: 14th International Workshop on Parallel and Distributed Algorithms and Applications (PDAA2022)

SCALES: From Fairness Principles to Constrained Decision-Making

Authors: Sreejith Balakrishnan, Jianxin Bi, Harold Soh

Abstract: This paper proposes SCALES, a general framework that translates well-established fairness principles into a common representation based on the Constraint Markov Decision Process (CMDP). With the help of causal language, our framework can place constraints on both the procedure of decision making (procedural fairness) as well as the outcomes resulting from decisions (outcome fairness). Specifically… ▽ More This paper proposes SCALES, a general framework that translates well-established fairness principles into a common representation based on the Constraint Markov Decision Process (CMDP). With the help of causal language, our framework can place constraints on both the procedure of decision making (procedural fairness) as well as the outcomes resulting from decisions (outcome fairness). Specifically, we show that well-known fairness principles can be encoded either as a utility component, a non-causal component, or a causal component in a SCALES-CMDP. We illustrate SCALES using a set of case studies involving a simulated healthcare scenario and the real-world COMPAS dataset. Experiments demonstrate that our framework produces fair policies that embody alternative fairness principles in single-step and sequential decision-making scenarios. △ Less

Submitted 22 September, 2022; originally announced September 2022.

Comments: Accepted to the 2022 AAAI/ACM Conference on AI, Ethics, and Society (AIES '22), Updated version with additional citations, 14 pages

Search for relativistic fractionally charged particles in space

Authors: DAMPE Collaboration, F. Alemanno, C. Altomare, Q. An, P. Azzarello, F. C. T. Barbato, P. Bernardini, X. J. Bi, M. S. Cai, E. Casilli, E. Catanzani, J. Chang, D. Y. Chen, J. L. Chen, Z. F. Chen, M. Y. Cui, T. S. Cui, Y. X. Cui, H. T. Dai, A. De-Benedittis, I. De Mitri, F. de Palma, M. Deliyergiyev, A. Di Giovanni, M. Di Santo , et al. (126 additional authors not shown)

Abstract: More than a century after the performance of the oil drop experiment, the possible existence of fractionally charged particles FCP still remains unsettled. The search for FCPs is crucial for some extensions of the Standard Model in particle physics. Most of the previously conducted searches for FCPs in cosmic rays were based on experiments underground or at high altitudes. However, there have been… ▽ More More than a century after the performance of the oil drop experiment, the possible existence of fractionally charged particles FCP still remains unsettled. The search for FCPs is crucial for some extensions of the Standard Model in particle physics. Most of the previously conducted searches for FCPs in cosmic rays were based on experiments underground or at high altitudes. However, there have been few searches for FCPs in cosmic rays carried out in orbit other than AMS-01 flown by a space shuttle and BESS by a balloon at the top of the atmosphere. In this study, we conduct an FCP search in space based on on-orbit data obtained using the DArk Matter Particle Explorer (DAMPE) satellite over a period of five years. Unlike underground experiments, which require an FCP energy of the order of hundreds of GeV, our FCP search starts at only a few GeV. An upper limit of $6.2\times 10^{-10}~~\mathrm{cm^{-2}sr^{-1} s^{-1}}$ is obtained for the flux. Our results demonstrate that DAMPE exhibits higher sensitivity than experiments of similar types by three orders of magnitude that more stringently restricts the conditions for the existence of FCP in primary cosmic rays. △ Less

Submitted 9 September, 2022; originally announced September 2022.

Comments: 19 pages, 6 figures, accepted by PRD

Report number: 106, 063026

Journal ref: Physical Review D 106.6 (2022): 063026

A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining

Authors: Hongwu Peng, Shaoyi Huang, Shiyang Chen, Bingbing Li, Tong Geng, Ang Li, Weiwen Jiang, Wujie Wen, Jinbo Bi, Hang Liu, Caiwen Ding

Abstract: Transformers are considered one of the most important deep learning models since 2018, in part because it establishes state-of-the-art (SOTA) records and could potentially replace existing Deep Neural Networks (DNNs). Despite the remarkable triumphs, the prolonged turnaround time of Transformer models is a widely recognized roadblock. The variety of sequence lengths imposes additional computing ov… ▽ More Transformers are considered one of the most important deep learning models since 2018, in part because it establishes state-of-the-art (SOTA) records and could potentially replace existing Deep Neural Networks (DNNs). Despite the remarkable triumphs, the prolonged turnaround time of Transformer models is a widely recognized roadblock. The variety of sequence lengths imposes additional computing overhead where inputs need to be zero-padded to the maximum sentence length in the batch to accommodate the parallel computing platforms. This paper targets the field-programmable gate array (FPGA) and proposes a coherent sequence length adaptive algorithm-hardware co-design for Transformer acceleration. Particularly, we develop a hardware-friendly sparse attention operator and a length-aware hardware resource scheduling algorithm. The proposed sparse attention operator brings the complexity of attention-based models down to linear complexity and alleviates the off-chip memory traffic. The proposed length-aware resource hardware scheduling algorithm dynamically allocates the hardware resources to fill up the pipeline slots and eliminates bubbles for NLP tasks. Experiments show that our design has very small accuracy loss and has 80.2 $\times$ and 2.6 $\times$ speedup compared to CPU and GPU implementation, and 4 $\times$ higher energy efficiency than state-of-the-art GPU accelerator optimized via CUBLAS GEMM. △ Less

Submitted 20 August, 2022; v1 submitted 7 August, 2022; originally announced August 2022.

Comments: 2022 59th ACM/IEEE Design Automation Conference (DAC)

ACM Class: I.2; B.6; C.3

Flux Variations of Cosmic Ray Air Showers Detected by LHAASO-KM2A During a Thunderstorm on 10 June 2021

Authors: LHAASO Collaboration, F. Aharonian, Q. An, Axikegu, L. X. Bai, Y. X. Bai, Y. W. Bao, D. Bastieri, X. J. Bi, Y. J. Bi, J. T. Cai, Zhe Cao, Zhen Cao, J. Chang, J. F. Chang, E. S. Chen, Liang Chen, Liang Chen, Long Chen, M. J. Chen, M. L. Chen, S. H. Chen, S. Z. Chen, T. L. Chen, X. J. Chen , et al. (248 additional authors not shown)

Abstract: The Large High Altitude Air Shower Observatory (LHAASO) has three sub-arrays, KM2A, WCDA and WFCTA. The flux variations of cosmic ray air showers were studied by analyzing the KM2A data during the thunderstorm on 10 June 2021. The number of shower events that meet the trigger conditions increases significantly in atmospheric electric fields, with maximum fractional increase of 20%. The variations… ▽ More The Large High Altitude Air Shower Observatory (LHAASO) has three sub-arrays, KM2A, WCDA and WFCTA. The flux variations of cosmic ray air showers were studied by analyzing the KM2A data during the thunderstorm on 10 June 2021. The number of shower events that meet the trigger conditions increases significantly in atmospheric electric fields, with maximum fractional increase of 20%. The variations of trigger rates (increases or decreases) are found to be strongly dependent on the primary zenith angle. The flux of secondary particles increases significantly, following a similar trend with that of the shower events. To better understand the observed behavior, Monte Carlo simulations are performed with CORSIKA and G4KM2A (a code based on GEANT4). We find that the experimental data (in saturated negative fields) are in good agreement with simulations, assuming the presence of a uniform upward electric field of 700 V/cm with a thickness of 1500 m in the atmosphere above the observation level. Due to the acceleration/deceleration and deflection by the atmospheric electric field, the number of secondary particles with energy above the detector threshold is modified, resulting in the changes in shower detection rate. △ Less

Submitted 6 December, 2022; v1 submitted 25 July, 2022; originally announced July 2022.

Comments: 18 pages, 11 figures

Journal ref: Chinese Phys. C 47 015001 (2023)

ISTRBoost: Importance Sampling Transfer Regression using Boosting

Authors: Shrey Gupta, Jianzhao Bi, Yang Liu, Avani Wildani

Abstract: Current Instance Transfer Learning (ITL) methodologies use domain adaptation and sub-space transformation to achieve successful transfer learning. However, these methodologies, in their processes, sometimes overfit on the target dataset or suffer from negative transfer if the test dataset has a high variance. Boosting methodologies have been shown to reduce the risk of overfitting by iteratively r… ▽ More Current Instance Transfer Learning (ITL) methodologies use domain adaptation and sub-space transformation to achieve successful transfer learning. However, these methodologies, in their processes, sometimes overfit on the target dataset or suffer from negative transfer if the test dataset has a high variance. Boosting methodologies have been shown to reduce the risk of overfitting by iteratively re-weighing instances with high-residual. However, this balance is usually achieved with parameter optimization, as well as reducing the skewness in weights produced due to the size of the source dataset. While the former can be achieved, the latter is more challenging and can lead to negative transfer. We introduce a simpler and more robust fix to this problem by building upon the popular boosting ITL regression methodology, two-stage TrAdaBoost.R2. Our methodology,~\us{}, is a boosting and random-forest based ensemble methodology that utilizes importance sampling to reduce the skewness due to the source dataset. We show that~\us{}~performs better than competitive transfer learning methodologies $63\%$ of the time. It also displays consistency in its performance over diverse datasets with varying complexities, as opposed to the sporadic results observed for other transfer learning methodologies. △ Less

Submitted 25 April, 2022; originally announced April 2022.

Efficient route to achieve superconductivity improvement via substitutional La-Ce alloy superhydride at high pressure

Authors: Jingkai Bi, Yuki Nakamoto, Katsuya Shimizu, Mi Zhou, Hongbo Wang, Guangtao Liu, Yanming Ma

Abstract: The discovery of clathrate superhydrides has approached the long-standing dream of room-temperature superconductivity and thus inspired their prosperous research under high pressure. However, how to experimentally optimize these compelling superhydrides is still a formidable challenge. Here, we find that half of the Ce atoms in the recently discovered hexagonal close packed (hcp) CeH9 structure ca… ▽ More The discovery of clathrate superhydrides has approached the long-standing dream of room-temperature superconductivity and thus inspired their prosperous research under high pressure. However, how to experimentally optimize these compelling superhydrides is still a formidable challenge. Here, we find that half of the Ce atoms in the recently discovered hexagonal close packed (hcp) CeH9 structure can be randomly replaced by adjacent La, resulting in the formation of LaH9 unit that is impossible in a binary system. Our experiments show that hcp (La, Ce)H9 can be synthesized at ~110 GPa and possesses a maximum Tc of 178 K at higher pressure, which is evidenced by in-situ X-ray diffraction and electronic transport measurement where a sharp drop of resistivity to zero and a characteristic decrease of Tc under a magnetic field up to 9 T. More importantly, the Tc of (La, Ce)H9 is significantly increased by ~50-80 K compared to CeH9, showing the hitherto highest Tc at megabar pressure. Our experimental results not only verify the feasibility of improving the superconductivity of hydrides by introducing other suitable metals, but also provide important inspiration for finding high-Tc superconductors in various multinary superhydrides. △ Less

Submitted 10 April, 2022; originally announced April 2022.

Dust Dynamics in Transitional Disks: Clumping and Disk Recession

Authors: Jiaqing Bi, Jeffrey Fung

Abstract: The role of radiation pressure in dust migration and the opening of inner cavities in transitional disks is revisited in this paper. Dust dynamics including radiation pressure is often studied in axisymmetric models, but in this work, we show that highly non-axisymmetric features can arise from an instability at the inner disk edge. Dust grains clump into high density features there, allowing radi… ▽ More The role of radiation pressure in dust migration and the opening of inner cavities in transitional disks is revisited in this paper. Dust dynamics including radiation pressure is often studied in axisymmetric models, but in this work, we show that highly non-axisymmetric features can arise from an instability at the inner disk edge. Dust grains clump into high density features there, allowing radiation to leak around them and penetrate deeper into the disk, changing the course of dust migration. Our proof-of-concept, two-dimensional, vertically-averaged simulations show that the combination of radiation pressure, shadowing, and gas drag can produce a net outward migration, or recession, of the dust component of the disk. The recession speed of the inner disk edge is on the order of $10^{-5}$ times Keplerian speed in our parameter space, which is faster than the background viscous flow, assuming a Shakura & Sunyaev viscosity alpha $\lesssim 10^{-3}$. This speed, if sustained over the lifetime of the disk, can result in a dust cavity as large as tens of au. △ Less

Submitted 14 February, 2022; v1 submitted 4 February, 2022; originally announced February 2022.

Comments: 17 pages, 10 figures, accepted for publication in ApJ

CausalMTA: Eliminating the User Confounding Bias for Causal Multi-touch Attribution

Authors: Di Yao, Chang Gong, Lei Zhang, Sheng Chen, Jingping Bi

Abstract: Multi-touch attribution (MTA), aiming to estimate the contribution of each advertisement touchpoint in conversion journeys, is essential for budget allocation and automatically advertising. Existing methods first train a model to predict the conversion probability of the advertisement journeys with historical data and calculate the attribution of each touchpoint using counterfactual predictions. A… ▽ More Multi-touch attribution (MTA), aiming to estimate the contribution of each advertisement touchpoint in conversion journeys, is essential for budget allocation and automatically advertising. Existing methods first train a model to predict the conversion probability of the advertisement journeys with historical data and calculate the attribution of each touchpoint using counterfactual predictions. An assumption of these works is the conversion prediction model is unbiased, i.e., it can give accurate predictions on any randomly assigned journey, including both the factual and counterfactual ones. Nevertheless, this assumption does not always hold as the exposed advertisements are recommended according to user preferences. This confounding bias of users would lead to an out-of-distribution (OOD) problem in the counterfactual prediction and cause concept drift in attribution. In this paper, we define the causal MTA task and propose CausalMTA to eliminate the influence of user preferences. It systemically eliminates the confounding bias from both static and dynamic preferences to learn the conversion prediction model using historical data. We also provide a theoretical analysis to prove CausalMTA can learn an unbiased prediction model with sufficient data. Extensive experiments on both public datasets and the impression data in an e-commerce company show that CausalMTA not only achieves better prediction performance than the state-of-the-art method but also generates meaningful attribution credits across different advertising channels. △ Less

Submitted 21 July, 2022; v1 submitted 20 December, 2021; originally announced January 2022.

Comments: 11 pages, 5 figures This paper has been accepted in KDD 2022

Wafer-Scale Epitaxy of Flexible Nitride Films with Superior Plasmonic and Superconducting Performance

Authors: Ruyi Zhang, Xinyan Li, Fanqi Meng, Jiachang Bi, Shunda Zhang, Shaoqin Peng, Jie Sun, Xinming Wang, Liang Wu, Junxi Duan, Hongtao Cao, Qinghua Zhang, Lin Gu, Liang-Feng Huang, Yanwei Cao

Abstract: Transition-metal nitrides (e.g., TiN, ZrN, TaN) are incredible materials with excellent complementary-metal-oxide-semiconductor compatibility and remarkable performance in refractory plasmonics and superconducting quantum electronics. Epitaxial growth of flexible transition-metal nitride films, especially at wafer-scale, is fundamentally important for developing high-performance flexible photonics… ▽ More Transition-metal nitrides (e.g., TiN, ZrN, TaN) are incredible materials with excellent complementary-metal-oxide-semiconductor compatibility and remarkable performance in refractory plasmonics and superconducting quantum electronics. Epitaxial growth of flexible transition-metal nitride films, especially at wafer-scale, is fundamentally important for developing high-performance flexible photonics and superconducting electronics, but the study is rare thus far. This work reports the high-quality epitaxy of 2-inch titanium nitride (TiN) films on flexible fluorophlogopite-mica (F-mica) substrates via reactive magnetron sputtering. Combined measurements of spectroscopic ellipsometer and electrical transport reveal the superior plasmonic and superconducting performance of TiN/F-mica films owing to the high single crystallinity. More interestingly, the superconductivity of these flexible TiN films can be manipulated by the bending states, and enhanced superconducting critical temperature TC is observed in convex TiN films with in-plane tensile strain. Density-functional-theory calculations uncover that the strain can tune the electron-phonon interaction strength and resultant superconductivity of TiN films. This study provides a promising route towards integrating scalable single-crystalline conductive transition-metal nitride films with flexible electronics for high-performance plasmonics and superconducting electronics. △ Less

Submitted 6 December, 2021; originally announced December 2021.

Peta-electron volt gamma-ray emission from the Crab Nebula

Authors: The LHAASO Collaboration, Zhen Cao, F. Aharonian, Q. An, Axikegu, L. X. Bai, Y. X. Bai, Y. W. Bao, D. Bastieri, X. J. Bi, Y. J. Bi, H. Cai, J. T. Cai, Zhe Cao, J. Chang, J. F. Chang, B. M. Chen, E. S. Chen, J. Chen, Liang Chen, Liang Chen, Long Chen, M. J. Chen, M. L. Chen, Q. H. Chen , et al. (250 additional authors not shown)

Abstract: The Crab pulsar and the surrounding nebula powered by the pulsar's rotational energy through the formation and termination of a relativistic electron-positron wind is a bright source of gamma-rays carrying crucial information about this complex conglomerate. We report the detection of $γ$-rays with a spectrum showing gradual steepening over three energy decades, from $5\times 10^{-4}$ to $1.1$ pet… ▽ More The Crab pulsar and the surrounding nebula powered by the pulsar's rotational energy through the formation and termination of a relativistic electron-positron wind is a bright source of gamma-rays carrying crucial information about this complex conglomerate. We report the detection of $γ$-rays with a spectrum showing gradual steepening over three energy decades, from $5\times 10^{-4}$ to $1.1$ petaelectronvolt (PeV). The ultra-high-energy photons exhibit the presence of a PeV electron accelerator (a pevatron) with an acceleration rate exceeding 15% of the absolute theoretical limit. Assuming that unpulsed $γ$-rays are produced at the termination of the pulsar's wind, we constrain the pevatron's size, between $0.025$ and $0.1$ pc, and the magnetic field $\approx 110 μ$G. The production rate of PeV electrons, $2.5 \times 10^{36}$ erg $\rm s^{-1}$, constitutes 0.5% of the pulsar's spin-down luminosity, although we do not exclude a non-negligible contribution of PeV protons to the production of the highest energy $γ$-rays. △ Less

Submitted 11 November, 2021; originally announced November 2021.

Comments: 43 pages, 13 figures, 2 tables; Published in Science

Journal ref: Science, 2021, Vol 373, Issue 6553, pp. 425-430

Procedure Planning in Instructional Videos via Contextual Modeling and Model-based Policy Learning

Authors: Jing Bi, Jiebo Luo, Chenliang Xu

Abstract: Learning new skills by observing humans' behaviors is an essential capability of AI. In this work, we leverage instructional videos to study humans' decision-making processes, focusing on learning a model to plan goal-directed actions in real-life videos. In contrast to conventional action recognition, goal-directed actions are based on expectations of their outcomes requiring causal knowledge of… ▽ More Learning new skills by observing humans' behaviors is an essential capability of AI. In this work, we leverage instructional videos to study humans' decision-making processes, focusing on learning a model to plan goal-directed actions in real-life videos. In contrast to conventional action recognition, goal-directed actions are based on expectations of their outcomes requiring causal knowledge of potential consequences of actions. Thus, integrating the environment structure with goals is critical for solving this task. Previous works learn a single world model will fail to distinguish various tasks, resulting in an ambiguous latent space; planning through it will gradually neglect the desired outcomes since the global information of the future goal degrades quickly as the procedure evolves. We address these limitations with a new formulation of procedure planning and propose novel algorithms to model human behaviors through Bayesian Inference and model-based Imitation Learning. Experiments conducted on real-world instructional videos show that our method can achieve state-of-the-art performance in reaching the indicated goals. Furthermore, the learned contextual information presents interesting features for planning in a latent space. △ Less

Submitted 8 October, 2021; v1 submitted 4 October, 2021; originally announced October 2021.

Comments: ICCV 2021 Oral

GW Ori: circumtriple rings and planets

Authors: Jeremy L. Smallwood, Rebecca Nealon, Cheng Chen, Rebecca G. Martin, Jiaqing Bi, Ruobing Dong, Christophe Pinte

Abstract: GW Ori is a hierarchical triple star system with a misaligned circumtriple protoplanetary disc. Recent ALMA observations have identified three dust rings with a prominent gap at $100\, \rm au$ and misalignments between each of the rings. A break in the gas disc may be driven either by the torque from the triple star system or a planet that is massive enough to carve a gap in the disc. Once the dis… ▽ More GW Ori is a hierarchical triple star system with a misaligned circumtriple protoplanetary disc. Recent ALMA observations have identified three dust rings with a prominent gap at $100\, \rm au$ and misalignments between each of the rings. A break in the gas disc may be driven either by the torque from the triple star system or a planet that is massive enough to carve a gap in the disc. Once the disc is broken, the rings nodally precess on different timescales and become misaligned. We investigate the origins of the dust rings by means of $N$-body integrations and 3-dimensional hydrodynamic simulations. We find that for observationally-motivated parameters of protoplanetary discs, the disc does not break due to the torque from the star system. We suggest that the presence of a massive planet (or planets) in the disc separates the inner and outer disc. We conclude that the disc breaking in GW Ori is likely caused by undetected planets -- the first planet(s) in a circumtriple orbit. △ Less

Submitted 20 September, 2021; originally announced September 2021.

Comments: 17 pages, 16 figures, accepted to MNRAS

GhostShiftAddNet: More Features from Energy-Efficient Operations

Authors: Jia Bi, Jonathon Hare, Geoff V. Merrett

Abstract: Deep convolutional neural networks (CNNs) are computationally and memory intensive. In CNNs, intensive multiplication can have resource implications that may challenge the ability for effective deployment of inference on resource-constrained edge devices. This paper proposes GhostShiftAddNet, where the motivation is to implement a hardware-efficient deep network: a multiplication-free CNN with few… ▽ More Deep convolutional neural networks (CNNs) are computationally and memory intensive. In CNNs, intensive multiplication can have resource implications that may challenge the ability for effective deployment of inference on resource-constrained edge devices. This paper proposes GhostShiftAddNet, where the motivation is to implement a hardware-efficient deep network: a multiplication-free CNN with fewer redundant features. We introduce a new bottleneck block, GhostSA, that converts all multiplications in the block to cheap operations. The bottleneck uses an appropriate number of bit-shift filters to process intrinsic feature maps, then applies a series of transformations that consist of bit-wise shifts with addition operations to generate more feature maps that fully learn to capture information underlying intrinsic features. We schedule the number of bit-shift and addition operations for different hardware platforms. We conduct extensive experiments and ablation studies with desktop and embedded (Jetson Nano) devices for implementation and measurements. We demonstrate the proposed GhostSA block can replace bottleneck blocks in the backbone of state-of-the-art networks architectures and gives improved performance on image classification benchmarks. Further, our GhostShiftAddNet can achieve higher classification accuracy with fewer FLOPs and parameters (reduced by up to 3x) than GhostNet. When compared to GhostNet, inference latency on the Jetson Nano is improved by 1.3x and 2x on the GPU and CPU respectively. Code is available open-source on \url{https://github.com/JIABI/GhostShiftAddNet}. △ Less

Submitted 3 February, 2022; v1 submitted 20 September, 2021; originally announced September 2021.

Journal ref: The 32nd British Machine Vision Conference BMVC 2021

Optimizing FPGA-based Accelerator Design for Large-Scale Molecular Similarity Search

Authors: Hongwu Peng, Shiyang Chen, Zhepeng Wang, Junhuan Yang, Scott A. Weitze, Tong Geng, Ang Li, Jinbo Bi, Minghu Song, Weiwen Jiang, Hang Liu, Caiwen Ding

Abstract: Molecular similarity search has been widely used in drug discovery to identify structurally similar compounds from large molecular databases rapidly. With the increasing size of chemical libraries, there is growing interest in the efficient acceleration of large-scale similarity search. Existing works mainly focus on CPU and GPU to accelerate the computation of the Tanimoto coefficient in measurin… ▽ More Molecular similarity search has been widely used in drug discovery to identify structurally similar compounds from large molecular databases rapidly. With the increasing size of chemical libraries, there is growing interest in the efficient acceleration of large-scale similarity search. Existing works mainly focus on CPU and GPU to accelerate the computation of the Tanimoto coefficient in measuring the pairwise similarity between different molecular fingerprints. In this paper, we propose and optimize an FPGA-based accelerator design on exhaustive and approximate search algorithms. On exhaustive search using BitBound & folding, we analyze the similarity cutoff and folding level relationship with search speedup and accuracy, and propose a scalable on-the-fly query engine on FPGAs to reduce the resource utilization and pipeline interval. We achieve a 450 million compounds-per-second processing throughput for a single query engine. On approximate search using hierarchical navigable small world (HNSW), a popular algorithm with high recall and query speed. We propose an FPGA-based graph traversal engine to utilize a high throughput register array based priority queue and fine-grained distance calculation engine to increase the processing capability. Experimental results show that the proposed FPGA-based HNSW implementation has a 103385 query per second (QPS) on the Chembl database with 0.92 recall and achieves a 35x speedup than the existing CPU implementation on average. To the best of our knowledge, our FPGA-based implementation is the first attempt to accelerate molecular similarity search algorithms on FPGA and has the highest performance among existing approaches. △ Less

Submitted 13 September, 2021; originally announced September 2021.

Comments: ICCAD 2021

ACM Class: B.0; I.0

Potential PeVatron supernova remnant G106.3+2.7 seen in the highest-energy gamma rays

Authors: M. Amenomori, Y. W. Bao, X. J. Bi, D. Chen, T. L. Chen, W. Y. Chen, Xu Chen, Y. Chen, Cirennima, S. W. Cui, Danzengluobu, L. K. Ding, J. H. Fang, K. Fang, C. F. Feng, Zhaoyang Feng, Z. Y. Feng, Qi Gao, Q. B. Gou, Y. Q. Guo, Y. Y. Guo, H. H. He, Z. T. He, K. Hibino, N. Hotta , et al. (70 additional authors not shown)

Abstract: Cosmic rays (protons and other atomic nuclei) are believed to gain energies of petaelectronvolts (PeV) and beyond at astrophysical particle accelerators called 'PeVatrons' inside our Galaxy. Although a characteristic feature of a PeVatron is expected to be a hard gamma-ray energy spectrum that extends beyond 100 teraelectronvolts (TeV) without a cutoff, none of the currently known sources exhibits… ▽ More Cosmic rays (protons and other atomic nuclei) are believed to gain energies of petaelectronvolts (PeV) and beyond at astrophysical particle accelerators called 'PeVatrons' inside our Galaxy. Although a characteristic feature of a PeVatron is expected to be a hard gamma-ray energy spectrum that extends beyond 100 teraelectronvolts (TeV) without a cutoff, none of the currently known sources exhibits such a spectrum due to the low maximum energy of accelerated cosmic rays or insufficient detector sensitivity around 100 TeV. Here we report the observation of gamma-ray emission from the supernova remnant G106.3+2.7 above 10 TeV. This work provides flux data points up to and above 100 TeV and indicates that the very-high-energy gamma-ray emission above 10 TeV is well correlated with a molecular cloud rather than the pulsar PSR J2229+6114. Regarding the gamma-ray emission mechanism of G106.3+2.7, this morphological feature appears to favor a hadronic origin via the π0 decay caused by accelerated relativistic protons over a leptonic one via the inverse-Compton scattering by relativistic electrons. Furthermore, we point out that an X-ray flux upper limit on the synchrotron spectrum would provide important information to firmly establish the hadronic scenario as the mechanism of particle acceleration at the source. △ Less

Submitted 7 September, 2021; originally announced September 2021.

Comments: Published in Nature Astronomy

Journal ref: Nature Astronomy, 5, 460-464 (2021)

An Empirical Study on the Joint Impact of Feature Selection and Data Re-sampling on Imbalance Classification

Authors: Chongsheng Zhang, Paolo Soda, Jingjun Bi, Gaojuan Fan, George Almpanidis, Salvador Garcia

Abstract: In predictive tasks, real-world datasets often present different degrees of imbalanced (i.e., long-tailed or skewed) distributions. While the majority (the head) classes have sufficient samples, the minority (the tail) classes can be under-represented by a rather limited number of samples. Data pre-processing has been shown to be very effective in dealing with such problems. On one hand, data re-s… ▽ More In predictive tasks, real-world datasets often present different degrees of imbalanced (i.e., long-tailed or skewed) distributions. While the majority (the head) classes have sufficient samples, the minority (the tail) classes can be under-represented by a rather limited number of samples. Data pre-processing has been shown to be very effective in dealing with such problems. On one hand, data re-sampling is a common approach to tackling class imbalance. On the other hand, dimension reduction, which reduces the feature space, is a conventional technique for reducing noise and inconsistencies in a dataset. However, the possible synergy between feature selection and data re-sampling for high-performance imbalance classification has rarely been investigated before. To address this issue, we carry out a comprehensive empirical study on the joint influence of feature selection and re-sampling on two-class imbalance classification. Specifically, we study the performance of two opposite pipelines for imbalance classification by applying feature selection before or after data re-sampling. We conduct a large number of experiments, with a total of 9225 tests, on 52 publicly available datasets, using 9 feature selection methods, 6 re-sampling approaches for class imbalance learning, and 3 well-known classification algorithms. Experimental results show that there is no constant winner between the two pipelines; thus both of them should be considered to derive the best performing model for imbalance classification. We find that the performance of an imbalance classification model not only depends on the classifier adopted and the ratio between the number of majority and minority samples, but also depends on the ratio between the number of samples and features. Overall, this study should provide new reference value for researchers and practitioners in imbalance learning. △ Less

Submitted 13 September, 2021; v1 submitted 1 September, 2021; originally announced September 2021.

Comments: 25 pages, 12 figures; revision v1

Dynamic Transformer for Efficient Machine Translation on Embedded Devices

Authors: Hishan Parry, Lei Xun, Amin Sabet, Jia Bi, Jonathon Hare, Geoff V. Merrett

Abstract: The Transformer architecture is widely used for machine translation tasks. However, its resource-intensive nature makes it challenging to implement on constrained embedded devices, particularly where available hardware resources can vary at run-time. We propose a dynamic machine translation model that scales the Transformer architecture based on the available resources at any particular time. The… ▽ More The Transformer architecture is widely used for machine translation tasks. However, its resource-intensive nature makes it challenging to implement on constrained embedded devices, particularly where available hardware resources can vary at run-time. We propose a dynamic machine translation model that scales the Transformer architecture based on the available resources at any particular time. The proposed approach, 'Dynamic-HAT', uses a HAT SuperTransformer as the backbone to search for SubTransformers with different accuracy-latency trade-offs at design time. The optimal SubTransformers are sampled from the SuperTransformer at run-time, depending on latency constraints. The Dynamic-HAT is tested on the Jetson Nano and the approach uses inherited SubTransformers sampled directly from the SuperTransformer with a switching time of <1s. Using inherited SubTransformers results in a BLEU score loss of <1.5% because the SubTransformer configuration is not retrained from scratch after sampling. However, to recover this loss in performance, the dimensions of the design space can be reduced to tailor it to a family of target hardware. The new reduced design space results in a BLEU score increase of approximately 1% for sub-optimal models from the original design space, with a wide range for performance scaling between 0.356s - 1.526s for the GPU and 2.9s - 7.31s for the CPU. △ Less

Submitted 30 July, 2021; v1 submitted 17 July, 2021; originally announced July 2021.

Comments: Accepted at MLCAD 2021

Gamma-ray Observation of the Cygnus Region in the 100 TeV Energy Region

Authors: M. Amenomori, Y. W. Bao, X. J. Bi, D. Chen, T. L. Chen, W. Y. Chen, Xu Chen, Y. Chen, Cirennima, S. W. Cui, Danzengluobu, L. K. Ding, J. H. Fang, K. Fang, C. F. Feng, Zhaoyang Feng, Z. Y. Feng, Qi Gao, A. Gomi, Q. B. Gou, Y. Q. Guo, Y. Y. Guo, H. H. He, Z. T. He, K. Hibino , et al. (88 additional authors not shown)

Abstract: We report observations of gamma-ray emissions with energies in the 100 TeV energy region from the Cygnus region in our Galaxy. Two sources are significantly detected in the directions of the Cygnus OB1 and OB2 associations. Based on their positional coincidences, we associate one with a pulsar PSR J2032+4127 and the other mainly with a pulsar wind nebula PWN G75.2+0.1 with the pulsar moving away f… ▽ More We report observations of gamma-ray emissions with energies in the 100 TeV energy region from the Cygnus region in our Galaxy. Two sources are significantly detected in the directions of the Cygnus OB1 and OB2 associations. Based on their positional coincidences, we associate one with a pulsar PSR J2032+4127 and the other mainly with a pulsar wind nebula PWN G75.2+0.1 with the pulsar moving away from its original birthplace situated around the centroid of the observed gamma-ray emission. This work would stimulate further studies of particle acceleration mechanisms at these gamma-ray sources. △ Less

Submitted 2 July, 2021; originally announced July 2021.

Comments: Accepted for publication in the Physical Review Letters

Coherent epitaxy of trilayer nickelate (Nd0.8Sr0.2)4Ni3O10 films by high-pressure magnetron sputtering

Authors: Jiachang Bi, Yujuan Pei, Ruyi Zhang, Shaoqin Peng, Xinming Wang, Jie Sun, Jiagui Feng, Jingkai Yang, Yanwei Cao

Abstract: Rare-earth (R) nickelates (such as perovskite RNiO3, trilayer R4Ni3O10, and infinite layer RNiO2) have attracted tremendous interest very recently. However, unlike widely studied RNiO3 and RNiO2 films, the synthesis of trilayer nickelate R4Ni3O10 films is rarely reported. Here, single-crystalline (Nd0.8Sr0.2)4Ni3O10 epitaxial films were coherently grown on SrTiO3 substrates by high-pressure magnet… ▽ More Rare-earth (R) nickelates (such as perovskite RNiO3, trilayer R4Ni3O10, and infinite layer RNiO2) have attracted tremendous interest very recently. However, unlike widely studied RNiO3 and RNiO2 films, the synthesis of trilayer nickelate R4Ni3O10 films is rarely reported. Here, single-crystalline (Nd0.8Sr0.2)4Ni3O10 epitaxial films were coherently grown on SrTiO3 substrates by high-pressure magnetron sputtering. The crystal and electronic structures of (Nd0.8Sr0.2)4Ni3O10 films were characterized by high-resolution X-ray diffraction and X-ray photoemission spectroscopy, respectively. The electrical transport measurements reveal a metal-insulator transition near 82 K and negative magnetoresistance in (Nd0.8Sr0.2)4Ni3O10 films. Our work provides a novel route to synthesize high-quality trilayer nickelate R4Ni3O10 films. △ Less

Submitted 6 June, 2021; originally announced June 2021.

Comments: 9 pages, 4 figures

Measurement of the cosmic ray helium energy spectrum from 70 GeV to 80 TeV with the DAMPE space mission

Authors: F. Alemanno, Q. An, P. Azzarello, F. C. T. Barbato, P. Bernardini, X. J. Bi, M. S. Cai, E. Catanzani, J. Chang, D. Y. Chen, J. L. Chen, Z. F. Chen, M. Y. Cui, T. S. Cui, Y. X. Cui, H. T. Dai, A. D'Amone, A. De Benedittis, I. De Mitri, F. de Palma, M. Deliyergiyev, M. Di Santo, T. K. Dong, Z. X. Dong, G. Donvito , et al. (120 additional authors not shown)

Abstract: The measurement of the energy spectrum of cosmic ray helium nuclei from 70 GeV to 80 TeV using 4.5 years of data recorded by the DArk Matter Particle Explorer (DAMPE) is reported in this work. A hardening of the spectrum is observed at an energy of about 1.3 TeV, similar to previous observations. In addition, a spectral softening at about 34 TeV is revealed for the first time with large statistics… ▽ More The measurement of the energy spectrum of cosmic ray helium nuclei from 70 GeV to 80 TeV using 4.5 years of data recorded by the DArk Matter Particle Explorer (DAMPE) is reported in this work. A hardening of the spectrum is observed at an energy of about 1.3 TeV, similar to previous observations. In addition, a spectral softening at about 34 TeV is revealed for the first time with large statistics and well controlled systematic uncertainties, with an overall significance of $4.3σ$. The DAMPE spectral measurements of both cosmic protons and helium nuclei suggest a particle charge dependent softening energy, although with current uncertainties a dependence on the number of nucleons cannot be ruled out. △ Less

Submitted 21 May, 2021; v1 submitted 19 May, 2021; originally announced May 2021.

Comments: 11 pages, 13 figures, published in Phys. Rev. Lett. Add one more digit for first three columns in Table S2

Journal ref: Phys. Rev. Lett. 126, 201102 (2021)

One-step epitaxy of high-mobility La-doped BaSnO3 films by high-pressure magnetron sputtering

Authors: Ruyi Zhang, Xinyan Li, Jiachang Bi, Shunda Zhang, Shaoqin Peng, Yang Song, Qinghua Zhang, Lin Gu, Junxi Duan, Yanwei Cao

Abstract: As a unique perovskite transparent oxide semiconductor, high-mobility La-doped BaSnO3 films have been successfully synthesized by molecular beam epitaxy and pulsed laser deposition. However, it remains a big challenge for magnetron sputtering, a widely applied technique suitable for large-scale fabrication, to grow high-mobility La-doped BaSnO3 films. Here, we developed a method to synthesize high… ▽ More As a unique perovskite transparent oxide semiconductor, high-mobility La-doped BaSnO3 films have been successfully synthesized by molecular beam epitaxy and pulsed laser deposition. However, it remains a big challenge for magnetron sputtering, a widely applied technique suitable for large-scale fabrication, to grow high-mobility La-doped BaSnO3 films. Here, we developed a method to synthesize high-mobility epitaxial La-doped BaSnO3 films (mobility up to 121 cm2V-1s-1 at the carrier density ~ 4.0 x 10^20 cm-3 at room temperature) directly on SrTiO3 single crystal substrates using high-pressure magnetron sputtering. The structural and electrical properties of the La-doped BaSnO3 films were characterized by combined high-resolution X-ray diffraction, X-ray photoemission spectroscopy, and temperature-dependent electrical transport measurements. The room temperature electron mobility of La-doped BaSnO3 films in this work is 2 to 4 times higher than the reported values of the films grown by magnetron sputtering. Moreover, in the high carrier density range (n > 3 x 10^20 cm-3), the electron mobility value of 121 cm2V-1s-1 in our work is among the highest values for all reported doped BaSnO3 films. It is revealed that high argon pressure during sputtering plays a vital role in stabilizing the fully relaxed films and inducing oxygen vacancies, which benefit the high mobility at room temperature. Our work provides an easy and economical way to massively synthesize high-mobility transparent conducting films for transparent electronics. △ Less

Submitted 10 May, 2021; originally announced May 2021.

Comments: 15 pages, 6 figures

Dynamic-OFA: Runtime DNN Architecture Switching for Performance Scaling on Heterogeneous Embedded Platforms

Authors: Wei Lou, Lei Xun, Amin Sabet, Jia Bi, Jonathon Hare, Geoff V. Merrett

Abstract: Mobile and embedded platforms are increasingly required to efficiently execute computationally demanding DNNs across heterogeneous processing elements. At runtime, the available hardware resources to DNNs can vary considerably due to other concurrently running applications. The performance requirements of the applications could also change under different scenarios. To achieve the desired performa… ▽ More Mobile and embedded platforms are increasingly required to efficiently execute computationally demanding DNNs across heterogeneous processing elements. At runtime, the available hardware resources to DNNs can vary considerably due to other concurrently running applications. The performance requirements of the applications could also change under different scenarios. To achieve the desired performance, dynamic DNNs have been proposed in which the number of channels/layers can be scaled in real time to meet different requirements under varying resource constraints. However, the training process of such dynamic DNNs can be costly, since platform-aware models of different deployment scenarios must be retrained to become dynamic. This paper proposes Dynamic-OFA, a novel dynamic DNN approach for state-of-the-art platform-aware NAS models (i.e. Once-for-all network (OFA)). Dynamic-OFA pre-samples a family of sub-networks from a static OFA backbone model, and contains a runtime manager to choose different sub-networks under different runtime environments. As such, Dynamic-OFA does not need the traditional dynamic DNN training pipeline. Compared to the state-of-the-art, our experimental results using ImageNet on a Jetson Xavier NX show that the approach is up to 3.5x (CPU), 2.4x (GPU) faster for similar ImageNet Top-1 accuracy, or 3.8% (CPU), 5.1% (GPU) higher accuracy at similar latency. △ Less

Submitted 11 May, 2021; v1 submitted 8 May, 2021; originally announced May 2021.

Comments: Accepted at CVPR ECV Workshop 2021

An Efficient Algorithm for Deep Stochastic Contextual Bandits

Authors: Tan Zhu, Guannan Liang, Chunjiang Zhu, Haining Li, Jinbo Bi

Abstract: In stochastic contextual bandit (SCB) problems, an agent selects an action based on certain observed context to maximize the cumulative reward over iterations. Recently there have been a few studies using a deep neural network (DNN) to predict the expected reward for an action, and the DNN is trained by a stochastic gradient based method. However, convergence analysis has been greatly ignored to e… ▽ More In stochastic contextual bandit (SCB) problems, an agent selects an action based on certain observed context to maximize the cumulative reward over iterations. Recently there have been a few studies using a deep neural network (DNN) to predict the expected reward for an action, and the DNN is trained by a stochastic gradient based method. However, convergence analysis has been greatly ignored to examine whether and where these methods converge. In this work, we formulate the SCB that uses a DNN reward function as a non-convex stochastic optimization problem, and design a stage-wise stochastic gradient descent algorithm to optimize the problem and determine the action policy. We prove that with high probability, the action sequence chosen by this algorithm converges to a greedy action policy respecting a local optimal reward function. Extensive experiments have been performed to demonstrate the effectiveness and efficiency of the proposed algorithm on multiple real-world datasets. △ Less

Submitted 21 April, 2021; v1 submitted 12 April, 2021; originally announced April 2021.

Comments: Accepted by AAAI 2021 Appendix uploaded

First Detection of sub-PeV Diffuse Gamma Rays from the Galactic Disk: Evidence for Ubiquitous Galactic Cosmic Rays beyond PeV Energies

Authors: M. Amenomori, Y. W. Bao, X. J. Bi, D. Chen, T. L. Chen, W. Y. Chen, Xu Chen, Y. Chen, Cirennima, S. W. Cui, Danzengluobu, L. K. Ding, J. H. Fang, K. Fang, C. F. Feng, Zhaoyang Feng, Z. Y. Feng, Qi Gao, Q. B. Gou, Y. Q. Guo, Y. Y. Guo, H. H. He, Z. T. He, K. Hibino, N. Hotta , et al. (70 additional authors not shown)

Abstract: We report, for the first time, the long-awaited detection of diffuse gamma rays with energies between 100 TeV and 1 PeV in the Galactic disk. Particularly, all gamma rays above 398 TeV are observed apart from known TeV gamma-ray sources and compatible with expectations from the hadronic emission scenario in which gamma rays originate from the decay of $π^0$'s produced through the interaction of pr… ▽ More We report, for the first time, the long-awaited detection of diffuse gamma rays with energies between 100 TeV and 1 PeV in the Galactic disk. Particularly, all gamma rays above 398 TeV are observed apart from known TeV gamma-ray sources and compatible with expectations from the hadronic emission scenario in which gamma rays originate from the decay of $π^0$'s produced through the interaction of protons with the interstellar medium in the Galaxy. This is strong evidence that cosmic rays are accelerated beyond PeV energies in our Galaxy and spread over the Galactic disk. △ Less

Submitted 17 May, 2021; v1 submitted 11 April, 2021; originally announced April 2021.

Comments: Accepted for publication in the Physical Review Letters

Journal ref: Phys. Rev. Lett. 126, 141101 (2021)

Calibration of the Air Shower Energy Scale of the Water and Air Cherenkov Techniques in the LHAASO experiment

Authors: F. Aharonian, Q. An, Axikegu, L. X. Bai, Y. X. Bai, Y. W. Bao, D. Bastieri, X. J. Bi, Y. J. Bi, H. Cai, J. T. Cai, Z. Cao Z. Cao, J. Chang, J. F. Chang, X. C. Chang, B. M. Chen, J. Chen, L. Chen, L. Chen, L. Chen, M. J. Chen, M. L. Chen, Q. H. Chen, S. H. Chen, S. Z. Chen , et al. (233 additional authors not shown)

Abstract: The Wide Field-of-View Cherenkov Telescope Array (WFCTA) and the Water Cherenkov Detector Arrays (WCDA) of LHAASO are designed to work in combination for measuring the energy spectra of various cosmic ray species over a very wide energy range from a few TeV to 10 PeV. The energy calibration of WCDA can be achieved with a proven technique of measuring the westward shift of the Moon shadow of galact… ▽ More The Wide Field-of-View Cherenkov Telescope Array (WFCTA) and the Water Cherenkov Detector Arrays (WCDA) of LHAASO are designed to work in combination for measuring the energy spectra of various cosmic ray species over a very wide energy range from a few TeV to 10 PeV. The energy calibration of WCDA can be achieved with a proven technique of measuring the westward shift of the Moon shadow of galactic cosmic rays due to the geomagnetic field. This deflection angle $Δ$ is inversely proportional to the energy of the cosmic rays. The precise measurements of the shifts by WCDA allows us to calibrate its energy scale for energies as high as 35 TeV. The energy scale measured by WCDA can be used to cross calibrate the energy reconstructed by WFCTA, which spans the whole energy range up to 10 PeV. In this work, we will demonstrate the feasibility of the method using the data collected from April 2019 to January 2020 by the WFCTA array and WCDA-1 detector, the first of the three water Cherenkov ponds, already commissioned at LHAASO site. △ Less

Submitted 13 April, 2021; v1 submitted 11 April, 2021; originally announced April 2021.

Journal ref: Phys. Rev. D 104, 062007 (2021)

Puffed up Edges of Planet-opened Gaps in Protoplanetary Disks. I. hydrodynamic simulations

Authors: Jiaqing Bi, Min-Kai Lin, Ruobing Dong

Abstract: Dust gaps and rings appear ubiquitous in bright protoplanetary disks. Disk-planet interaction with dust-trapping at the edges of planet-induced gaps is one plausible explanation. However, the sharpness of some observed dust rings indicate that sub-mm-sized dust grains have settled to a thin layer in some systems. We test whether or not such dust around gas gaps opened by planets can remain settled… ▽ More Dust gaps and rings appear ubiquitous in bright protoplanetary disks. Disk-planet interaction with dust-trapping at the edges of planet-induced gaps is one plausible explanation. However, the sharpness of some observed dust rings indicate that sub-mm-sized dust grains have settled to a thin layer in some systems. We test whether or not such dust around gas gaps opened by planets can remain settled by performing three-dimensional, dust-plus-gas simulations of protoplanetary disks with an embedded planet. We find planets massive enough to open gas gaps stir small, sub-mm-sized dust grains to high disk elevations at the gap edges, where the dust scale-height can reach ~70% of the gas scale-height. We attribute this dust 'puff-up' to the planet-induced meridional gas flows previously identified by Fung & Chiang and others. We thus emphasize the importance of explicit 3D simulations to obtain the vertical distribution of sub-mm-sized grains around gas gaps opened by massive planets. We caution that the gas-gap-opening planet interpretation of well-defined dust rings is only self-consistent with large grains exceeding mm in size. △ Less

Submitted 16 March, 2021; originally announced March 2021.

Comments: 15 pages, 7+2 figures, accepted in ApJ

Escaping Saddle Points with Stochastically Controlled Stochastic Gradient Methods

Authors: Guannan Liang, Qianqian Tong, Chunjiang Zhu, Jinbo Bi

Abstract: Stochastically controlled stochastic gradient (SCSG) methods have been proved to converge efficiently to first-order stationary points which, however, can be saddle points in nonconvex optimization. It has been observed that a stochastic gradient descent (SGD) step introduces anistropic noise around saddle points for deep learning and non-convex half space learning problems, which indicates that S… ▽ More Stochastically controlled stochastic gradient (SCSG) methods have been proved to converge efficiently to first-order stationary points which, however, can be saddle points in nonconvex optimization. It has been observed that a stochastic gradient descent (SGD) step introduces anistropic noise around saddle points for deep learning and non-convex half space learning problems, which indicates that SGD satisfies the correlated negative curvature (CNC) condition for these problems. Therefore, we propose to use a separate SGD step to help the SCSG method escape from strict saddle points, resulting in the CNC-SCSG method. The SGD step plays a role similar to noise injection but is more stable. We prove that the resultant algorithm converges to a second-order stationary point with a convergence rate of $\tilde{O}( ε^{-2} log( 1/ε))$ where $ε$ is the pre-specified error tolerance. This convergence rate is independent of the problem dimension, and is faster than that of CNC-SGD. A more general framework is further designed to incorporate the proposed CNC-SCSG into any first-order method for the method to escape saddle points. Simulation studies illustrate that the proposed algorithm can escape saddle points in much fewer epochs than the gradient descent methods perturbed by either noise injection or a SGD step. △ Less

Submitted 23 April, 2021; v1 submitted 7 March, 2021; originally announced March 2021.

A Variance Controlled Stochastic Method with Biased Estimation for Faster Non-convex Optimization

Authors: Jia Bi, Steve R. Gunn

Abstract: In this paper, we proposed a new technique, {\em variance controlled stochastic gradient} (VCSG), to improve the performance of the stochastic variance reduced gradient (SVRG) algorithm. To avoid over-reducing the variance of gradient by SVRG, a hyper-parameter $λ$ is introduced in VCSG that is able to control the reduced variance of SVRG. Theory shows that the optimization method can converge by… ▽ More In this paper, we proposed a new technique, {\em variance controlled stochastic gradient} (VCSG), to improve the performance of the stochastic variance reduced gradient (SVRG) algorithm. To avoid over-reducing the variance of gradient by SVRG, a hyper-parameter $λ$ is introduced in VCSG that is able to control the reduced variance of SVRG. Theory shows that the optimization method can converge by using an unbiased gradient estimator, but in practice, biased gradient estimation can allow more efficient convergence to the vicinity since an unbiased approach is computationally more expensive. $λ$ also has the effect of balancing the trade-off between unbiased and biased estimations. Secondly, to minimize the number of full gradient calculations in SVRG, a variance-bounded batch is introduced to reduce the number of gradient calculations required in each iteration. For smooth non-convex functions, the proposed algorithm converges to an approximate first-order stationary point (i.e. $\mathbb{E}\|\nabla{f}(x)\|^{2}\leqε$) within $\mathcal{O}(min\{1/ε^{3/2},n^{1/4}/ε\})$ number of stochastic gradient evaluations, which improves the leading gradient complexity of stochastic gradient-based method SCS $(\mathcal{O}(min\{1/ε^{5/3},n^{2/3}/ε\})$. It is shown theoretically and experimentally that VCSG can be deployed to improve convergence. △ Less

Submitted 19 February, 2021; originally announced February 2021.

Crystal orientation-dependent oxidation of epitaxial TiN films with tunable plasmonics

Authors: Ruyi Zhang, Qian-Ying Ma, Haigang Liu, Tian-Yu Sun, Jiachang Bi, Yang Song, Shaoqin Peng, Lingyan Liang, Junhua Gao, Hongtao Cao, Liang-Feng Huang, Yanwei Cao

Abstract: Titanium nitride (TiN) is a paradigm of refractory transition metal nitrides with great potential in vast applications. Generally, the plasmonic performance of TiN can be tuned by oxidation, which was thought to be only temperature-, oxygen partial pressure-, and time-dependent. Regarding the role of crystallographic orientation in the oxidation and resultant optical properties of TiN films, littl… ▽ More Titanium nitride (TiN) is a paradigm of refractory transition metal nitrides with great potential in vast applications. Generally, the plasmonic performance of TiN can be tuned by oxidation, which was thought to be only temperature-, oxygen partial pressure-, and time-dependent. Regarding the role of crystallographic orientation in the oxidation and resultant optical properties of TiN films, little is known thus far. Here we reveal that both the oxidation resistance behavior and the plasmonic performance of epitaxial TiN films follow the order of (001) < (110) < (111). The effects of crystallographic orientation on the lattice constants, optical properties, and oxidation levels of epitaxial TiN films have been systematically studied by combined high-resolution X-ray diffraction, spectroscopic ellipsometry, X-ray absorption spectroscopy, and X-ray photoemission spectroscopy. To further understand the role of crystallographic orientation in the initial oxidation process of TiN films, density-functional-theory calculations are carried out, indicating the energy cost of oxidation is (001) < (110) < (111), consistent with the experiments. The superior endurance of the (111) orientation against mild oxidation can largely alleviate the previously stringent technical requirements for the growth of TiN films with high plasmonic performance. The crystallographic orientation can also offer an effective controlling parameter to design TiN-based plasmonic devices with desired peculiarity, e.g., superior chemical stability against mild oxidation or large optical tunability upon oxidation. △ Less

Submitted 17 February, 2021; originally announced February 2021.

Comments: 23 pages, 5 figures

Discrete Graph Structure Learning for Forecasting Multiple Time Series

Authors: Chao Shang, Jie Chen, Jinbo Bi

Abstract: Time series forecasting is an extensively studied subject in statistics, economics, and computer science. Exploration of the correlation and causation among the variables in a multivariate time series shows promise in enhancing the performance of a time series model. When using deep neural networks as forecasting models, we hypothesize that exploiting the pairwise information among multiple (multi… ▽ More Time series forecasting is an extensively studied subject in statistics, economics, and computer science. Exploration of the correlation and causation among the variables in a multivariate time series shows promise in enhancing the performance of a time series model. When using deep neural networks as forecasting models, we hypothesize that exploiting the pairwise information among multiple (multivariate) time series also improves their forecast. If an explicit graph structure is known, graph neural networks (GNNs) have been demonstrated as powerful tools to exploit the structure. In this work, we propose learning the structure simultaneously with the GNN if the graph is unknown. We cast the problem as learning a probabilistic graph model through optimizing the mean performance over the graph distribution. The distribution is parameterized by a neural network so that discrete graphs can be sampled differentiably through reparameterization. Empirical evaluations show that our method is simpler, more efficient, and better performing than a recently proposed bilevel learning approach for graph structure learning, as well as a broad array of forecasting models, either deep or non-deep learning based, and graph or non-graph based. △ Less

Submitted 20 April, 2021; v1 submitted 17 January, 2021; originally announced January 2021.

Comments: ICLR 2021. Code is available at https://github.com/chaoshangcs/GTS

Federated Nonconvex Sparse Learning

Authors: Qianqian Tong, Guannan Liang, Tan Zhu, Jinbo Bi

Abstract: Nonconvex sparse learning plays an essential role in many areas, such as signal processing and deep network compression. Iterative hard thresholding (IHT) methods are the state-of-the-art for nonconvex sparse learning due to their capability of recovering true support and scalability with large datasets. Theoretical analysis of IHT is currently based on centralized IID data. In realistic large-sca… ▽ More Nonconvex sparse learning plays an essential role in many areas, such as signal processing and deep network compression. Iterative hard thresholding (IHT) methods are the state-of-the-art for nonconvex sparse learning due to their capability of recovering true support and scalability with large datasets. Theoretical analysis of IHT is currently based on centralized IID data. In realistic large-scale situations, however, data are distributed, hardly IID, and private to local edge computing devices. It is thus necessary to examine the property of IHT in federated settings, which update in parallel on local devices and communicate with a central server only once in a while without sharing local data. In this paper, we propose two IHT methods: Federated Hard Thresholding (Fed-HT) and Federated Iterative Hard Thresholding (FedIter-HT). We prove that both algorithms enjoy a linear convergence rate and have strong guarantees to recover the optimal sparse estimator, similar to traditional IHT methods, but now with decentralized non-IID data. Empirical results demonstrate that the Fed-HT and FedIter-HT outperform their competitor - a distributed IHT, in terms of decreasing the objective values with lower requirements on communication rounds and bandwidth. △ Less

Submitted 31 December, 2020; originally announced January 2021.

Construction and On-site Performance of the LHAASO WFCTA Camera

Authors: F. Aharonian, Q. An, Axikegu, L. X. Bai, Y. X. Bai, Y. W. Bao, D. Bastieri, X. J. Bi, Y. J. Bi, H. Cai, J. T. Cai, Z. Cao, Z. Cao, J. Chang, J. F. Chang, X. C. Chang, B. M. Chen, J. Chen, L. Chen, L. Chen, L. Chen, M. J. Chen, M. L. Chen, Q. H. Chen, S. H. Chen , et al. (234 additional authors not shown)

Abstract: The focal plane camera is the core component of the Wide Field-of-view Cherenkov/fluorescence Telescope Array (WFCTA) of the Large High-Altitude Air Shower Observatory (LHAASO). Because of the capability of working under moonlight without aging, silicon photomultipliers (SiPM) have been proven to be not only an alternative but also an improvement to conventional photomultiplier tubes (PMT) in this… ▽ More The focal plane camera is the core component of the Wide Field-of-view Cherenkov/fluorescence Telescope Array (WFCTA) of the Large High-Altitude Air Shower Observatory (LHAASO). Because of the capability of working under moonlight without aging, silicon photomultipliers (SiPM) have been proven to be not only an alternative but also an improvement to conventional photomultiplier tubes (PMT) in this application. Eighteen SiPM-based cameras with square light funnels have been built for WFCTA. The telescopes have collected more than 100 million cosmic ray events and preliminary results indicate that these cameras are capable of working under moonlight. The characteristics of the light funnels and SiPMs pose challenges (e.g. dynamic range, dark count rate, assembly techniques). In this paper, we present the design features, manufacturing techniques and performances of these cameras. Finally, the test facilities, the test methods and results of SiPMs in the cameras are reported here. △ Less

Submitted 4 July, 2021; v1 submitted 29 December, 2020; originally announced December 2020.

Comments: 45 pages, 21 figures, article

Journal ref: Eur. Phys. J. C 81, 657 (2021)

Asynchronous Parallel Stochastic Quasi-Newton Methods

Authors: Qianqian Tong, Guannan Liang, Xingyu Cai, Chunjiang Zhu, Jinbo Bi

Abstract: Although first-order stochastic algorithms, such as stochastic gradient descent, have been the main force to scale up machine learning models, such as deep neural nets, the second-order quasi-Newton methods start to draw attention due to their effectiveness in dealing with ill-conditioned optimization problems. The L-BFGS method is one of the most widely used quasi-Newton methods. We propose an as… ▽ More Although first-order stochastic algorithms, such as stochastic gradient descent, have been the main force to scale up machine learning models, such as deep neural nets, the second-order quasi-Newton methods start to draw attention due to their effectiveness in dealing with ill-conditioned optimization problems. The L-BFGS method is one of the most widely used quasi-Newton methods. We propose an asynchronous parallel algorithm for stochastic quasi-Newton (AsySQN) method. Unlike prior attempts, which parallelize only the calculation for gradient or the two-loop recursion of L-BFGS, our algorithm is the first one that truly parallelizes L-BFGS with a convergence guarantee. Adopting the variance reduction technique, a prior stochastic L-BFGS, which has not been designed for parallel computing, reaches a linear convergence rate. We prove that our asynchronous parallel scheme maintains the same linear convergence rate but achieves significant speedup. Empirical evaluations in both simulations and benchmark datasets demonstrate the speedup in comparison with the non-parallel stochastic L-BFGS, as well as the better performance than first-order methods in solving ill-conditioned problems. △ Less

Submitted 1 November, 2020; originally announced November 2020.

Comments: Accepted by Parallel Computing Journal

The observation of the Crab Nebula with LHAASO-KM2A for the performance study

Authors: F. Aharonian, Q. An, Axikegu, L. X. Bai, Y. X. Bai, Y. W. Bao, D. Bastieri, X. J. Bi, Y. J. Bi, H. Cai, J. T. Cai, Z. Cao, Z. Cao, J. Chang, J. F. Chang, X. C. Chang, B. M. Chen, J. Chen, L. Chen, L. Chen, L. Chen, M. J. Chen, M. L. Chen, Q. H. Chen, S. H. Chen , et al. (234 additional authors not shown)

Abstract: As a sub-array of the Large High Altitude Air Shower Observatory (LHAASO), KM2A is mainly designed to cover a large fraction of the northern sky to hunt for gamma-ray sources at energies above 10 TeV. Even though the detector construction is still underway, a half of the KM2A array has been operating stably since the end of 2019. In this paper, we present the pipeline of KM2A data analysis and the… ▽ More As a sub-array of the Large High Altitude Air Shower Observatory (LHAASO), KM2A is mainly designed to cover a large fraction of the northern sky to hunt for gamma-ray sources at energies above 10 TeV. Even though the detector construction is still underway, a half of the KM2A array has been operating stably since the end of 2019. In this paper, we present the pipeline of KM2A data analysis and the first observation on the Crab Nebula, a standard candle in very high energy gamma-ray astronomy. We detect gamma-ray signals from the Crab Nebula in both energy ranges of 10$-$100 TeV and $>$100 TeV with high significance, by analyzing the KM2A data of 136 live days between December 2019 and May 2020. With the observations, we test the detector performance including angular resolution, pointing accuracy and cosmic ray background rejection power. The energy spectrum of the Crab Nebula in the energy range 10-250 TeV fits well with a single power-law function dN/dE =(1.13$\pm$0.05$_{stat}$$\pm$0.08$_{sys}$)$\times$10$^{-14}$$\cdot$(E/20TeV)$^{-3.09\pm0.06_{stat}\pm0.02_{sys}}$ cm$^{-2}$ s$^{-1}$ TeV$^{-1}$. It is consistent with previous measurements by other experiments. This opens a new window of gamma-ray astronomy above 0.1 PeV through which ultrahigh-energy gamma-ray new phenomena, such as cosmic PeVatrons, might be discovered. △ Less

Submitted 13 October, 2020; originally announced October 2020.

Comments: 13 pages, 15 figures,submitted to CPC

Journal ref: Chinese Phys. C 45 025002 (2021)

Cubic Spline Smoothing Compensation for Irregularly Sampled Sequences

Authors: Jing Shi, Jing Bi, Yingru Liu, Chenliang Xu

Abstract: The marriage of recurrent neural networks and neural ordinary differential networks (ODE-RNN) is effective in modeling irregularly-observed sequences. While ODE produces the smooth hidden states between observation intervals, the RNN will trigger a hidden state jump when a new observation arrives, thus cause the interpolation discontinuity problem. To address this issue, we propose the cubic splin… ▽ More The marriage of recurrent neural networks and neural ordinary differential networks (ODE-RNN) is effective in modeling irregularly-observed sequences. While ODE produces the smooth hidden states between observation intervals, the RNN will trigger a hidden state jump when a new observation arrives, thus cause the interpolation discontinuity problem. To address this issue, we propose the cubic spline smoothing compensation, which is a stand-alone module upon either the output or the hidden state of ODE-RNN and can be trained end-to-end. We derive its analytical solution and provide its theoretical interpolation error bound. Extensive experiments indicate its merits over both ODE-RNN and cubic spline interpolation. △ Less

Submitted 3 October, 2020; originally announced October 2020.

Effective Proximal Methods for Non-convex Non-smooth Regularized Learning

Authors: Guannan Liang, Qianqian Tong, Jiahao Ding, Miao Pan, Jinbo Bi

Abstract: Sparse learning is a very important tool for mining useful information and patterns from high dimensional data. Non-convex non-smooth regularized learning problems play essential roles in sparse learning, and have drawn extensive attentions recently. We design a family of stochastic proximal gradient methods by applying arbitrary sampling to solve the empirical risk minimization problem with a non… ▽ More Sparse learning is a very important tool for mining useful information and patterns from high dimensional data. Non-convex non-smooth regularized learning problems play essential roles in sparse learning, and have drawn extensive attentions recently. We design a family of stochastic proximal gradient methods by applying arbitrary sampling to solve the empirical risk minimization problem with a non-convex and non-smooth regularizer. These methods draw mini-batches of training examples according to an arbitrary probability distribution when computing stochastic gradients. A unified analytic approach is developed to examine the convergence and computational complexity of these methods, allowing us to compare the different sampling schemes. We show that the independent sampling scheme tends to improve performance over the commonly-used uniform sampling scheme. Our new analysis also derives a tighter bound on convergence speed for the uniform sampling than the best one available so far. Empirical evaluations demonstrate that the proposed algorithms converge faster than the state of the art. △ Less

Submitted 21 October, 2020; v1 submitted 14 September, 2020; originally announced September 2020.

Comments: Accepted by ICDM 2020, 24 pages

Effective Federated Adaptive Gradient Methods with Non-IID Decentralized Data

Authors: Qianqian Tong, Guannan Liang, Jinbo Bi

Abstract: Federated learning allows loads of edge computing devices to collaboratively learn a global model without data sharing. The analysis with partial device participation under non-IID and unbalanced data reflects more reality. In this work, we propose federated learning versions of adaptive gradient methods - Federated AGMs - which employ both the first-order and second-order momenta, to alleviate ge… ▽ More Federated learning allows loads of edge computing devices to collaboratively learn a global model without data sharing. The analysis with partial device participation under non-IID and unbalanced data reflects more reality. In this work, we propose federated learning versions of adaptive gradient methods - Federated AGMs - which employ both the first-order and second-order momenta, to alleviate generalization performance deterioration caused by dissimilarity of data population among devices. To further improve the test performance, we compare several schemes of calibration for the adaptive learning rate, including the standard Adam calibrated by $ε$, $p$-Adam, and one calibrated by an activation function. Our analysis provides the first set of theoretical results that the proposed (calibrated) Federated AGMs converge to a first-order stationary point under non-IID and unbalanced data settings for nonconvex optimization. We perform extensive experiments to compare these federated learning methods with the state-of-the-art FedAvg, FedMomentum and SCAFFOLD and to assess the different calibration schemes and the advantages of AGMs over the current federated learning methods. △ Less

Submitted 21 December, 2020; v1 submitted 14 September, 2020; originally announced September 2020.

Comments: 42 pages

Against Membership Inference Attack: Pruning is All You Need

Authors: Yijue Wang, Chenghong Wang, Zigeng Wang, Shanglin Zhou, Hang Liu, Jinbo Bi, Caiwen Ding, Sanguthevar Rajasekaran

Abstract: The large model size, high computational operations, and vulnerability against membership inference attack (MIA) have impeded deep learning or deep neural networks (DNNs) popularity, especially on mobile devices. To address the challenge, we envision that the weight pruning technique will help DNNs against MIA while reducing model storage and computational operation. In this work, we propose a pru… ▽ More The large model size, high computational operations, and vulnerability against membership inference attack (MIA) have impeded deep learning or deep neural networks (DNNs) popularity, especially on mobile devices. To address the challenge, we envision that the weight pruning technique will help DNNs against MIA while reducing model storage and computational operation. In this work, we propose a pruning algorithm, and we show that the proposed algorithm can find a subnetwork that can prevent privacy leakage from MIA and achieves competitive accuracy with the original DNNs. We also verify our theoretical insights with experiments. Our experimental results illustrate that the attack accuracy using model compression is up to 13.6% and 10% lower than that of the baseline and Min-Max game, accordingly. △ Less

Submitted 4 July, 2021; v1 submitted 27 August, 2020; originally announced August 2020.

Comments: Machine Learning (cs.LG); Cryptography and Security (cs.CR); Machine Learning (stat.ML)

Journal ref: IJCAI, 2021

Towards Plausible Differentially Private ADMM Based Distributed Machine Learning

Authors: Jiahao Ding, Jingyi Wang, Guannan Liang, Jinbo Bi, Miao Pan

Abstract: The Alternating Direction Method of Multipliers (ADMM) and its distributed version have been widely used in machine learning. In the iterations of ADMM, model updates using local private data and model exchanges among agents impose critical privacy concerns. Despite some pioneering works to relieve such concerns, differentially private ADMM still confronts many research challenges. For example, th… ▽ More The Alternating Direction Method of Multipliers (ADMM) and its distributed version have been widely used in machine learning. In the iterations of ADMM, model updates using local private data and model exchanges among agents impose critical privacy concerns. Despite some pioneering works to relieve such concerns, differentially private ADMM still confronts many research challenges. For example, the guarantee of differential privacy (DP) relies on the premise that the optimality of each local problem can be perfectly attained in each ADMM iteration, which may never happen in practice. The model trained by DP ADMM may have low prediction accuracy. In this paper, we address these concerns by proposing a novel (Improved) Plausible differentially Private ADMM algorithm, called PP-ADMM and IPP-ADMM. In PP-ADMM, each agent approximately solves a perturbed optimization problem that is formulated from its local private data in an iteration, and then perturbs the approximate solution with Gaussian noise to provide the DP guarantee. To further improve the model accuracy and convergence, an improved version IPP-ADMM adopts sparse vector technique (SVT) to determine if an agent should update its neighbors with the current perturbed solution. The agent calculates the difference of the current solution from that in the last iteration, and if the difference is larger than a threshold, it passes the solution to neighbors; or otherwise the solution will be discarded. Moreover, we propose to track the total privacy loss under the zero-concentrated DP (zCDP) and provide a generalization performance analysis. Experiments on real-world datasets demonstrate that under the same privacy guarantee, the proposed algorithms are superior to the state of the art in terms of model accuracy and convergence rate. △ Less

Submitted 10 August, 2020; originally announced August 2020.

Comments: Comments: Accepted for publication in CIKM'20

Temporal Gravity Model for Important Nodes Identification in Temporal Networks

Authors: Jialin Bi, Ji Jin, Cunquan Qu, Xiuxiu Zhan, Guanghui Wang

Abstract: Identifying important nodes is one of the central tasks in network science, which is crucial for analyzing the structure of a network and understanding the dynamical processes on a network. Most real-world systems are time-varying and can be well represented as temporal networks. Motivated by the classic gravity model in physics, we propose a temporal gravity model to identify influential nodes in… ▽ More Identifying important nodes is one of the central tasks in network science, which is crucial for analyzing the structure of a network and understanding the dynamical processes on a network. Most real-world systems are time-varying and can be well represented as temporal networks. Motivated by the classic gravity model in physics, we propose a temporal gravity model to identify influential nodes in temporal networks. Two critical elements in the gravity model are the masses of the objects and the distance between two objects. In the temporal gravity model, we treat nodes as the objects, basic node properties, such as static and temporal properties, as the nodes' masses. We define temporal distances, i.e., fastest arrival distance and temporal shortest distance, as the distance between two nodes in our model. We utilize our model as well as the baseline centrality methods on important nodes identification. Experimental results on ten real-world datasets show that the temporal gravity model outperforms the baseline methods in quantifying node structural influence. Moreover, when we use the temporal shortest distance as the distance between two nodes, our model is robust and performs the best in quantifying node spreading influence compared to the baseline methods. △ Less

Submitted 4 July, 2020; originally announced July 2020.

Comments: 34 pages, 6 pages

An Evolutional Algorithm for Automatic 2D Layer Segmentation in Laser-aided Additive Manufacturing

Authors: N. Liu, K. Ren, W. Zhang, Y. F. Zhang, Y. X. Chew, J. Y. H. Fuh, G. J. Bi

Abstract: Toolpath planning is an important task in laser aided additive manufacturing (LAAM) and other direct energy deposition (DED) processes. The deposition toolpaths for complex geometries with slender structures can be further optimized by partitioning the sliced 2D layers into sub-regions, and enable the design of appropriate infill toolpaths for different sub-regions. However, reported approaches fo… ▽ More Toolpath planning is an important task in laser aided additive manufacturing (LAAM) and other direct energy deposition (DED) processes. The deposition toolpaths for complex geometries with slender structures can be further optimized by partitioning the sliced 2D layers into sub-regions, and enable the design of appropriate infill toolpaths for different sub-regions. However, reported approaches for 2D layer segmentation generally require manual operations that are tedious and time-consuming. To increase segmentation efficiency, this paper proposes an autonomous approach based on evolutional computation for 2D layer segmentation. The algorithm works in an identify-and-segment manner. Specifically, the largest quasi-quadrilateral is identified and segmented from the target layer iteratively. Results from case studies have validated the effectiveness and efficacy of the developed algorithm. To further improve its performance, a roughing-finishing strategy is proposed. Via multi-processing, the strategy can remarkably increase the solution variety without affecting solution quality and search time, thus providing great application potential in LAAM toolpath planning. To the best of the authors knowledge, this work is the first to address automatic 2D layer segmentation problem in LAAM process. Therefore, it may be a valuable supplement to the state of the art in this area. △ Less

Submitted 26 June, 2020; v1 submitted 12 June, 2020; originally announced June 2020.

GW Ori: Interactions Between a Triple-star System and its Circumtriple Disk in Action

Authors: Jiaqing Bi, Nienke van der Marel, Ruobing Dong, Takayuki Muto, Rebecca G. Martin, Jeremy L. Smallwood, Jun Hashimoto, Hauyu Baobab Liu, Hideko Nomura, Yasuhiro Hasegawa, Michihiro Takami, Mihoko Konishi, Munetake Momose, Kazuhiro D. Kanagawa, Akimasa Kataoka, Tomohiro Ono, Michael L. Sitko, Sanemichi Z. Takahashi, Kengo Tomida, Takashi Tsukagoshi

Abstract: GW Ori is a hierarchical triple system which has a rare circumtriple disk. We present Atacama Large Millimeter/submillimeter Array (ALMA) observations of 1.3 mm dust continuum and 12CO J=2-1 molecular gas emission of the disk. For the first time, we identify three dust rings in the disk at ~46, 188, and 338 AU, with estimated dust mass of ~70-250 Earth masses, respectively. To our knowledge, the o… ▽ More GW Ori is a hierarchical triple system which has a rare circumtriple disk. We present Atacama Large Millimeter/submillimeter Array (ALMA) observations of 1.3 mm dust continuum and 12CO J=2-1 molecular gas emission of the disk. For the first time, we identify three dust rings in the disk at ~46, 188, and 338 AU, with estimated dust mass of ~70-250 Earth masses, respectively. To our knowledge, the outer ring in GW Ori is the largest dust ring ever found in protoplanetary disks. We use visibility modelling of dust continuum to show that the disk has misaligned parts and the innermost dust ring is eccentric. The disk misalignment is also suggested by the CO kinematics modelling. We interpret these substructures as evidence of ongoing dynamical interactions between the triple stars and the circumtriple disk. △ Less

Submitted 29 April, 2020; v1 submitted 7 April, 2020; originally announced April 2020.

Comments: 17 pages, 3+3 figures, 2 tables; accepted by ApJL on 2020-04-29

Creating and manipulating interfacial spin with giant magnetic response in 4$f$ antiferromagnets

Authors: Ruyi Zhang, Yujuan Pei, Yang Song, Jiachang Bi, Jingkai Yang, Junxi Duan, Yanwei Cao

Abstract: Creating and manipulating spin polarization in low-dimensional electron systems (such as two-dimensional electron gases) is fundamentally essential for spintronic applications, which is yet a challenge to date. In this work, we establish the metamagnetic phase diagram of 4$f$ antiferromagnetic TbScO$_3$ and reveal its giant magnetic response to sub-tesla magnetic field, which has not been reported… ▽ More Creating and manipulating spin polarization in low-dimensional electron systems (such as two-dimensional electron gases) is fundamentally essential for spintronic applications, which is yet a challenge to date. In this work, we establish the metamagnetic phase diagram of 4$f$ antiferromagnetic TbScO$_3$ and reveal its giant magnetic response to sub-tesla magnetic field, which has not been reported thus far. Utilizing this giant magnetic response, we demonstrate that the spin polarization of two-dimensional electron gas in SrTiO$_3$/LaTiO$_3$/TbScO$_3$ heterostructure can be manipulated successfully in aid of interfacial 3\textit{d}-4\textit{f} exchange interaction. Remarkably, the hysteretic magnetoresistances of two-dimensional electron gas at the SrTiO$_3$/LaTiO$_3$ interface are entirely determined by the metamagnetic phase transitions of the underlying TbScO$_3$ substrate. Our results pave a novel route to engineer the spin polarization of low-dimensional electron systems in 4$f$ antiferromagnet-based heterostructures. △ Less

Submitted 18 January, 2020; originally announced January 2020.

Comments: 5 pages, 4 figures

Learning from Interventions using Hierarchical Policies for Safe Learning

Authors: Jing Bi, Vikas Dhiman, Tianyou Xiao, Chenliang Xu

Abstract: Learning from Demonstrations (LfD) via Behavior Cloning (BC) works well on multiple complex tasks. However, a limitation of the typical LfD approach is that it requires expert demonstrations for all scenarios, including those in which the algorithm is already well-trained. The recently proposed Learning from Interventions (LfI) overcomes this limitation by using an expert overseer. The expert over… ▽ More Learning from Demonstrations (LfD) via Behavior Cloning (BC) works well on multiple complex tasks. However, a limitation of the typical LfD approach is that it requires expert demonstrations for all scenarios, including those in which the algorithm is already well-trained. The recently proposed Learning from Interventions (LfI) overcomes this limitation by using an expert overseer. The expert overseer only intervenes when it suspects that an unsafe action is about to be taken. Although LfI significantly improves over LfD, the state-of-the-art LfI fails to account for delay caused by the expert's reaction time and only learns short-term behavior. We address these limitations by 1) interpolating the expert's interventions back in time, and 2) by splitting the policy into two hierarchical levels, one that generates sub-goals for the future and another that generates actions to reach those desired sub-goals. This sub-goal prediction forces the algorithm to learn long-term behavior while also being robust to the expert's reaction time. Our experiments show that LfI using sub-goals in a hierarchical policy framework trains faster and achieves better asymptotic performance than typical LfD. △ Less

Submitted 4 December, 2019; originally announced December 2019.

Comments: Accepted for publication at the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20)