Showing 251–300 of 416 results for author: Yue, Y

Infall in massive clumps harboring bright infrared sources

Authors: Ying-Hua Yue, Sheng-Li Qin, Tie Liu, Meng-Yao Tang, Yuefang Wu, Ke Wang, Chao Zhang

Abstract: Thirty massive clumps associated with bright infrared sources were observed to detect the infall signatures and characterize infall properties in the envelope of the massive clumps by APEX telescope in CO(4-3) and C$^{17}$O(3-2) lines. Eighteen objects have "blue profile" in CO(4-3) line with virial parameters less than 2, suggesting that global collapse is taking place in these massive clumps. Th… ▽ More Thirty massive clumps associated with bright infrared sources were observed to detect the infall signatures and characterize infall properties in the envelope of the massive clumps by APEX telescope in CO(4-3) and C$^{17}$O(3-2) lines. Eighteen objects have "blue profile" in CO(4-3) line with virial parameters less than 2, suggesting that global collapse is taking place in these massive clumps. The CO(4-3) lines were fitted by the two-layer model in order to obtain infall velocities and mass infall rates. Derived mass infall rates are from 10$^{-3}$ to 10$^{-1}$ M$_{\odot}$yr$^{-1}$. A positive relationship between clump mass and infall rate appears to indicate that gravity plays a dominant role in the collapsing process. Higher luminosity clump has larger mass infall rate, implying that the clump with higher mass infall rate has higher star formation rate. △ Less

Submitted 20 October, 2020; originally announced October 2020.

Iterative Amortized Policy Optimization

Authors: Joseph Marino, Alexandre Piché, Alessandro Davide Ialongo, Yisong Yue

Abstract: Policy networks are a central feature of deep reinforcement learning (RL) algorithms for continuous control, enabling the estimation and sampling of high-value actions. From the variational inference perspective on RL, policy networks, when used with entropy or KL regularization, are a form of \textit{amortized optimization}, optimizing network parameters rather than the policy distributions direc… ▽ More Policy networks are a central feature of deep reinforcement learning (RL) algorithms for continuous control, enabling the estimation and sampling of high-value actions. From the variational inference perspective on RL, policy networks, when used with entropy or KL regularization, are a form of \textit{amortized optimization}, optimizing network parameters rather than the policy distributions directly. However, \textit{direct} amortized mappings can yield suboptimal policy estimates and restricted distributions, limiting performance and exploration. Given this perspective, we consider the more flexible class of \textit{iterative} amortized optimizers. We demonstrate that the resulting technique, iterative amortized policy optimization, yields performance improvements over direct amortization on benchmark continuous control tasks. △ Less

Submitted 22 October, 2021; v1 submitted 20 October, 2020; originally announced October 2020.

Comments: Advances in Neural Processing Systems (NeurIPS) 2021

Learning Better Representation for Tables by Self-Supervised Tasks

Authors: Liang Li, Can Ma, Yinliang Yue, Linjun Shou, Dayong Hu

Abstract: Table-to-text generation aims at automatically generating natural text to help people to conveniently obtain the important information in tables. Although neural models for table-to-text have achieved remarkable progress, some problems still overlooked. The first is that the values recorded in many tables are mostly numbers in practice. The existing approaches do not do special treatment for these… ▽ More Table-to-text generation aims at automatically generating natural text to help people to conveniently obtain the important information in tables. Although neural models for table-to-text have achieved remarkable progress, some problems still overlooked. The first is that the values recorded in many tables are mostly numbers in practice. The existing approaches do not do special treatment for these, and still regard these as words in natural language text. Secondly, the target texts in training dataset may contain redundant information or facts do not exist in the input tables. These may give wrong supervision signals to some methods based on content selection and planning and auxiliary supervision. To solve these problems, we propose two self-supervised tasks, Number Ordering and Significance Ordering, to help to learn better table representation. The former works on the column dimension to help to incorporate the size property of numbers into table representation. The latter acts on row dimension and help to learn a significance-aware table representation. We test our methods on the widely used dataset ROTOWIRE which consists of NBA game statistic and related news. The experimental results demonstrate that the model trained together with these two self-supervised tasks can generate text that contains more salient and well-organized facts, even without modeling context selection and planning. And we achieve the state-of-the-art performance on automatic metrics. △ Less

Submitted 30 March, 2021; v1 submitted 15 October, 2020; originally announced October 2020.

Comments: This article is writing messy, and some of the experiments are inadequate, which may mislead the reader about our work

Learning Calibrated Uncertainties for Domain Shift: A Distributionally Robust Learning Approach

Authors: Haoxuan Wang, Zhiding Yu, Yisong Yue, Anima Anandkumar, Anqi Liu, Junchi Yan

Abstract: We propose a framework for learning calibrated uncertainties under domain shifts, where the source (training) distribution differs from the target (test) distribution. We detect such domain shifts via a differentiable density ratio estimator and train it together with the task network, composing an adjusted softmax predictive form concerning domain shift. In particular, the density ratio estimatio… ▽ More We propose a framework for learning calibrated uncertainties under domain shifts, where the source (training) distribution differs from the target (test) distribution. We detect such domain shifts via a differentiable density ratio estimator and train it together with the task network, composing an adjusted softmax predictive form concerning domain shift. In particular, the density ratio estimation reflects the closeness of a target (test) sample to the source (training) distribution. We employ it to adjust the uncertainty of prediction in the task network. This idea of using the density ratio is based on the distributionally robust learning (DRL) framework, which accounts for the domain shift by adversarial risk minimization. We show that our proposed method generates calibrated uncertainties that benefit downstream tasks, such as unsupervised domain adaptation (UDA) and semi-supervised learning (SSL). On these tasks, methods like self-training and FixMatch use uncertainties to select confident pseudo-labels for re-training. Our experiments show that the introduction of DRL leads to significant improvements in cross-domain performance. We also show that the estimated density ratios align with human selection frequencies, suggesting a positive correlation with a proxy of human perceived uncertainties. △ Less

Submitted 5 February, 2024; v1 submitted 7 October, 2020; originally announced October 2020.

Comments: IJCAI 2023

RNN Training along Locally Optimal Trajectories via Frank-Wolfe Algorithm

Authors: Yun Yue, Ming Li, Venkatesh Saligrama, Ziming Zhang

Abstract: We propose a novel and efficient training method for RNNs by iteratively seeking a local minima on the loss surface within a small region, and leverage this directional vector for the update, in an outer-loop. We propose to utilize the Frank-Wolfe (FW) algorithm in this context. Although, FW implicitly involves normalized gradients, which can lead to a slow convergence rate, we develop a novel RNN… ▽ More We propose a novel and efficient training method for RNNs by iteratively seeking a local minima on the loss surface within a small region, and leverage this directional vector for the update, in an outer-loop. We propose to utilize the Frank-Wolfe (FW) algorithm in this context. Although, FW implicitly involves normalized gradients, which can lead to a slow convergence rate, we develop a novel RNN training method that, surprisingly, even with the additional cost, the overall training cost is empirically observed to be lower than back-propagation. Our method leads to a new Frank-Wolfe method, that is in essence an SGD algorithm with a restart scheme. We prove that under certain conditions our algorithm has a sublinear convergence rate of $O(1/ε)$ for $ε$ error. We then conduct empirical experiments on several benchmark datasets including those that exhibit long-term dependencies, and show significant performance improvement. We also experiment with deep RNN architectures and show efficient training performance. Finally, we demonstrate that our training method is robust to noisy data. △ Less

Submitted 15 October, 2020; v1 submitted 11 October, 2020; originally announced October 2020.

Temporal Vectorization for Stencils

Authors: Liang Yuan, Hang Cao, Yunquan Zhang, Kun Li, Pengqi Lu, Yue Yue

Abstract: Stencil computations represent a very common class of nested loops in scientific and engineering applications. Exploiting vector units in modern CPUs is crucial to achieving peak performance. Previous vectorization approaches often consider the data space, in particular the innermost unit-strided loop. It leads to the well-known data alignment conflict problem that vector loads are overlapped due… ▽ More Stencil computations represent a very common class of nested loops in scientific and engineering applications. Exploiting vector units in modern CPUs is crucial to achieving peak performance. Previous vectorization approaches often consider the data space, in particular the innermost unit-strided loop. It leads to the well-known data alignment conflict problem that vector loads are overlapped due to the data sharing between continuous stencil computations. This paper proposes a novel temporal vectorization scheme for stencils. It vectorizes the stencil computation in the iteration space and assembles points with different time coordinates in one vector. The temporal vectorization leads to a small fixed number of vector reorganizations that is irrelevant to the vector length, stencil order, and dimension. Furthermore, it is also applicable to Gauss-Seidel stencils, whose vectorization is not well-studied. The effectiveness of the temporal vectorization is demonstrated by various Jacobi and Gauss-Seidel stencils. △ Less

Submitted 9 October, 2020; originally announced October 2020.

Improving adversarial robustness of deep neural networks by using semantic information

Authors: Lina Wang, Rui Tang, Yawei Yue, Xingshu Chen, Wei Wang, Yi Zhu, Xuemei Zeng

Abstract: The vulnerability of deep neural networks (DNNs) to adversarial attack, which is an attack that can mislead state-of-the-art classifiers into making an incorrect classification with high confidence by deliberately perturbing the original inputs, raises concerns about the robustness of DNNs to such attacks. Adversarial training, which is the main heuristic method for improving adversarial robustnes… ▽ More The vulnerability of deep neural networks (DNNs) to adversarial attack, which is an attack that can mislead state-of-the-art classifiers into making an incorrect classification with high confidence by deliberately perturbing the original inputs, raises concerns about the robustness of DNNs to such attacks. Adversarial training, which is the main heuristic method for improving adversarial robustness and the first line of defense against adversarial attacks, requires many sample-by-sample calculations to increase training size and is usually insufficiently strong for an entire network. This paper provides a new perspective on the issue of adversarial robustness, one that shifts the focus from the network as a whole to the critical part of the region close to the decision boundary corresponding to a given class. From this perspective, we propose a method to generate a single but image-agnostic adversarial perturbation that carries the semantic information implying the directions to the fragile parts on the decision boundary and causes inputs to be misclassified as a specified target. We call the adversarial training based on such perturbations "region adversarial training" (RAT), which resembles classical adversarial training but is distinguished in that it reinforces the semantic information missing in the relevant regions. Experimental results on the MNIST and CIFAR-10 datasets show that this approach greatly improves adversarial robustness even using a very small dataset from the training data; moreover, it can defend against FGSM adversarial attacks that have a completely different pattern from the model seen during retraining. △ Less

Submitted 16 June, 2021; v1 submitted 18 August, 2020; originally announced August 2020.

Comments: 13 pages, 9 figures

ACM Class: I.2.6

Journal ref: [J]. Knowledge-Based Systems, 2021: 107141

A SLAM Map Restoration Algorithm Based on Submaps and an Undirected Connected Graph

Authors: Zongqian Zhan, Wenjie Jian, Yihui Li, Xin Wang, Yang Yue

Abstract: Many visual simultaneous localization and mapping (SLAM) systems have been shown to be accurate and robust, and have real-time performance capabilities on both indoor and ground datasets. However, these methods can be problematic when dealing with aerial frames captured by a camera mounted on an unmanned aerial vehicle (UAV) because the flight height of the UAV can be difficult to control and is e… ▽ More Many visual simultaneous localization and mapping (SLAM) systems have been shown to be accurate and robust, and have real-time performance capabilities on both indoor and ground datasets. However, these methods can be problematic when dealing with aerial frames captured by a camera mounted on an unmanned aerial vehicle (UAV) because the flight height of the UAV can be difficult to control and is easily affected by the environment.To cope with the case of lost tracking, many visual SLAM systems employ a relocalization strategy. This involves the tracking thread continuing the online working by inspecting the connections between the subsequent new frames and the generated map before the tracking was lost. To solve the missing map problem, which is an issue in many applications , after the tracking is lost, based on monocular visual SLAM, we present a method of reconstructing a complete global map of UAV datasets by sequentially merging the submaps via the corresponding undirected connected graph. Specifically, submaps are repeatedly generated, from the initialization process to the place where the tracking is lost, and a corresponding undirected connected graph is built by considering these submaps as nodes and the common map points within two submaps as edges. The common map points are then determined by the bag-of-words (BoW) method, and the submaps are merged if they are found to be connected with the online map in the undirect connected graph. To demonstrate the performance of the proposed method, we first investigated the performance on a UAV dataset, and the experimental results showed that, in the case of several tracking failures, the integrity of the mapping was significantly better than that of the current mainstream SLAM method. △ Less

Submitted 29 July, 2020; originally announced July 2020.

Learning Differentiable Programs with Admissible Neural Heuristics

Authors: Ameesh Shah, Eric Zhan, Jennifer J. Sun, Abhinav Verma, Yisong Yue, Swarat Chaudhuri

Abstract: We study the problem of learning differentiable functions expressed as programs in a domain-specific language. Such programmatic models can offer benefits such as composability and interpretability; however, learning them requires optimizing over a combinatorial space of program "architectures". We frame this optimization problem as a search in a weighted graph whose paths encode top-down derivati… ▽ More We study the problem of learning differentiable functions expressed as programs in a domain-specific language. Such programmatic models can offer benefits such as composability and interpretability; however, learning them requires optimizing over a combinatorial space of program "architectures". We frame this optimization problem as a search in a weighted graph whose paths encode top-down derivations of program syntax. Our key innovation is to view various classes of neural networks as continuous relaxations over the space of programs, which can then be used to complete any partial program. This relaxed program is differentiable and can be trained end-to-end, and the resulting training loss is an approximately admissible heuristic that can guide the combinatorial search. We instantiate our approach on top of the A-star algorithm and an iteratively deepened branch-and-bound search, and use these algorithms to learn programmatic classifiers in three sequence classification tasks. Our experiments show that the algorithms outperform state-of-the-art methods for program learning, and that they discover programmatic classifiers that yield natural interpretations and achieve competitive accuracy. △ Less

Submitted 27 March, 2021; v1 submitted 23 July, 2020; originally announced July 2020.

Comments: 9 pages, published in NeurIPS 2020

ACM Class: D.3.2; I.2.6

Active Learning under Label Shift

Authors: Eric Zhao, Anqi Liu, Animashree Anandkumar, Yisong Yue

Abstract: We address the problem of active learning under label shift: when the class proportions of source and target domains differ. We introduce a "medial distribution" to incorporate a tradeoff between importance weighting and class-balanced sampling and propose their combined usage in active learning. Our method is known as Mediated Active Learning under Label Shift (MALLS). It balances the bias from c… ▽ More We address the problem of active learning under label shift: when the class proportions of source and target domains differ. We introduce a "medial distribution" to incorporate a tradeoff between importance weighting and class-balanced sampling and propose their combined usage in active learning. Our method is known as Mediated Active Learning under Label Shift (MALLS). It balances the bias from class-balanced sampling and the variance from importance weighting. We prove sample complexity and generalization guarantees for MALLS which show active learning reduces asymptotic sample complexity even under arbitrary label shift. We empirically demonstrate MALLS scales to high-dimensional datasets and can reduce the sample complexity of active learning by 60% in deep active learning tasks. △ Less

Submitted 25 February, 2021; v1 submitted 16 July, 2020; originally announced July 2020.

Comments: 18 pages, 9 figures, to appear at the 2021 International Conference on Artificial Intelligence and Statistics (AIStats)

Implicit Distributional Reinforcement Learning

Authors: Yuguang Yue, Zhendong Wang, Mingyuan Zhou

Abstract: To improve the sample efficiency of policy-gradient based reinforcement learning algorithms, we propose implicit distributional actor-critic (IDAC) that consists of a distributional critic, built on two deep generator networks (DGNs), and a semi-implicit actor (SIA), powered by a flexible policy distribution. We adopt a distributional perspective on the discounted cumulative return and model it wi… ▽ More To improve the sample efficiency of policy-gradient based reinforcement learning algorithms, we propose implicit distributional actor-critic (IDAC) that consists of a distributional critic, built on two deep generator networks (DGNs), and a semi-implicit actor (SIA), powered by a flexible policy distribution. We adopt a distributional perspective on the discounted cumulative return and model it with a state-action-dependent implicit distribution, which is approximated by the DGNs that take state-action pairs and random noises as their input. Moreover, we use the SIA to provide a semi-implicit policy distribution, which mixes the policy parameters with a reparameterizable distribution that is not constrained by an analytic density function. In this way, the policy's marginal distribution is implicit, providing the potential to model complex properties such as covariance structure and skewness, but its parameter and entropy can still be estimated. We incorporate these features with an off-policy algorithm framework to solve problems with continuous action space and compare IDAC with state-of-the-art algorithms on representative OpenAI Gym environments. We observe that IDAC outperforms these baselines in most tasks. Python code is provided. △ Less

Submitted 19 October, 2020; v1 submitted 12 July, 2020; originally announced July 2020.

Comments: NeurIPS 2020

FC2RN: A Fully Convolutional Corner Refinement Network for Accurate Multi-Oriented Scene Text Detection

Authors: Xugong Qin, Yu Zhou, Dayan Wu, Yinliang Yue, Weiping Wang

Abstract: Recent scene text detection works mainly focus on curve text detection. However, in real applications, the curve texts are more scarce than the multi-oriented ones. Accurate detection of multi-oriented text with large variations of scales, orientations, and aspect ratios is of great significance. Among the multi-oriented detection methods, direct regression for the geometry of scene text shares a… ▽ More Recent scene text detection works mainly focus on curve text detection. However, in real applications, the curve texts are more scarce than the multi-oriented ones. Accurate detection of multi-oriented text with large variations of scales, orientations, and aspect ratios is of great significance. Among the multi-oriented detection methods, direct regression for the geometry of scene text shares a simple yet powerful pipeline and gets popular in academic and industrial communities, but it may produce imperfect detections, especially for long texts due to the limitation of the receptive field. In this work, we aim to improve this while keeping the pipeline simple. A fully convolutional corner refinement network (FC2RN) is proposed for accurate multi-oriented text detection, in which an initial corner prediction and a refined corner prediction are obtained at one pass. With a novel quadrilateral RoI convolution operation tailed for multi-oriented scene text, the initial quadrilateral prediction is encoded into the feature maps which can be further used to predict offset between the initial prediction and the ground-truth as well as output a refined confidence score. Experimental results on four public datasets including MSRA-TD500, ICDAR2017-RCTW, ICDAR2015, and COCO-Text demonstrate that FC2RN can outperform the state-of-the-art methods. The ablation study shows the effectiveness of corner refinement and scoring for accurate text localization. △ Less

Submitted 9 July, 2020; originally announced July 2020.

Graph Neural Networks for the Prediction of Substrate-Specific Organic Reaction Conditions

Authors: Serim Ryou, Michael R. Maser, Alexander Y. Cui, Travis J. DeLano, Yisong Yue, Sarah E. Reisman

Abstract: We present a systematic investigation using graph neural networks (GNNs) to model organic chemical reactions. To do so, we prepared a dataset collection of four ubiquitous reactions from the organic chemistry literature. We evaluate seven different GNN architectures for classification tasks pertaining to the identification of experimental reagents and conditions. We find that models are able to id… ▽ More We present a systematic investigation using graph neural networks (GNNs) to model organic chemical reactions. To do so, we prepared a dataset collection of four ubiquitous reactions from the organic chemistry literature. We evaluate seven different GNN architectures for classification tasks pertaining to the identification of experimental reagents and conditions. We find that models are able to identify specific graph features that affect reaction conditions and lead to accurate predictions. The results herein show great promise in advancing molecular machine learning. △ Less

Submitted 9 July, 2020; v1 submitted 8 July, 2020; originally announced July 2020.

Comments: 23 pages, 10 tables, 13 figures, to appear in the ICML 2020 Workshop on Graph Representation Learning and Beyond (GRLB)

Enhanced transport of spin-orbit coupled Bose gases in disordered potentials

Authors: Y. Yue, C. A. R. Sá de Melo, I. B. Spielman

Abstract: Anderson localization is a single particle localization phenomena in disordered media that is accompanied by an absence of diffusion. Spin-orbit coupling (SOC) describes an interaction between a particle's spin and its momentum that directly affects its energy dispersion, for example creating dispersion relations with gaps and multiple local minima. We show theoretically that combining one-dimensi… ▽ More Anderson localization is a single particle localization phenomena in disordered media that is accompanied by an absence of diffusion. Spin-orbit coupling (SOC) describes an interaction between a particle's spin and its momentum that directly affects its energy dispersion, for example creating dispersion relations with gaps and multiple local minima. We show theoretically that combining one-dimensional spin-orbit coupling with a transverse Zeeman field suppresses the effects of disorder, thereby increasing the localization length and conductivity. This increase results from a suppression of back scattering between states in the gap of the SOC dispersion relation. Here, we focus specifically on the interplay of disorder from an optical speckle potential and SOC generated by two-photon Raman processes in quasi-1D Bose-Einstein condensates. We first describe back-scattering using a Fermi's golden rule approach, and then numerically confirm this picture by solving the time-dependent 1D Gross Pitaevskii equation for a weakly interacting Bose-Einstein condensate with SOC and disorder. We find that on the 10's of millisecond time scale of typical cold atom experiments moving in harmonic traps, initial states with momentum in the zero-momentum SOC gap evolve with negligible back-scattering, while without SOC these same states rapidly localize. △ Less

Submitted 2 July, 2020; originally announced July 2020.

Comments: 12 pages, 7 figures

Journal ref: Phys. Rev. A 102, 033325 (2020)

Deep Bayesian Quadrature Policy Optimization

Authors: Akella Ravi Tej, Kamyar Azizzadenesheli, Mohammad Ghavamzadeh, Anima Anandkumar, Yisong Yue

Abstract: We study the problem of obtaining accurate policy gradient estimates using a finite number of samples. Monte-Carlo methods have been the default choice for policy gradient estimation, despite suffering from high variance in the gradient estimates. On the other hand, more sample efficient alternatives like Bayesian quadrature methods have received little attention due to their high computational co… ▽ More We study the problem of obtaining accurate policy gradient estimates using a finite number of samples. Monte-Carlo methods have been the default choice for policy gradient estimation, despite suffering from high variance in the gradient estimates. On the other hand, more sample efficient alternatives like Bayesian quadrature methods have received little attention due to their high computational complexity. In this work, we propose deep Bayesian quadrature policy gradient (DBQPG), a computationally efficient high-dimensional generalization of Bayesian quadrature, for policy gradient estimation. We show that DBQPG can substitute Monte-Carlo estimation in policy gradient methods, and demonstrate its effectiveness on a set of continuous control benchmarks. In comparison to Monte-Carlo estimation, DBQPG provides (i) more accurate gradient estimates with a significantly lower variance, (ii) a consistent improvement in the sample complexity and average return for several deep policy gradient algorithms, and, (iii) the uncertainty in gradient estimation that can be incorporated to further improve the performance. △ Less

Submitted 16 December, 2020; v1 submitted 28 June, 2020; originally announced June 2020.

Comments: Conference paper: AAAI-21. Code available at https://github.com/Akella17/Deep-Bayesian-Quadrature-Policy-Optimization

Average-case Complexity of Teaching Convex Polytopes via Halfspace Queries

Authors: Akash Kumar, Adish Singla, Yisong Yue, Yuxin Chen

Abstract: We examine the task of locating a target region among those induced by intersections of $n$ halfspaces in $\mathbb{R}^d$. This generic task connects to fundamental machine learning problems, such as training a perceptron and learning a $φ$-separable dichotomy. We investigate the average teaching complexity of the task, i.e., the minimal number of samples (halfspace queries) required by a teacher t… ▽ More We examine the task of locating a target region among those induced by intersections of $n$ halfspaces in $\mathbb{R}^d$. This generic task connects to fundamental machine learning problems, such as training a perceptron and learning a $φ$-separable dichotomy. We investigate the average teaching complexity of the task, i.e., the minimal number of samples (halfspace queries) required by a teacher to help a version-space learner in locating a randomly selected target. As our main result, we show that the average-case teaching complexity is $Θ(d)$, which is in sharp contrast to the worst-case teaching complexity of $Θ(n)$. If instead, we consider the average-case learning complexity, the bounds have a dependency on $n$ as $Θ(n)$ for \tt{i.i.d.} queries and $Θ(d \log(n))$ for actively chosen queries by the learner. Our proof techniques are based on novel insights from computational geometry, which allow us to count the number of convex polytopes and faces in a Euclidean space depending on the arrangement of halfspaces. Our insights allow us to establish a tight bound on the average-case complexity for $φ$-separable dichotomies, which generalizes the known $\mathcal{O}(d)$ bound on the average number of "extreme patterns" in the classical computational geometry literature (Cover, 1965). △ Less

Submitted 25 October, 2020; v1 submitted 25 June, 2020; originally announced June 2020.

Learning compositional functions via multiplicative weight updates

Authors: Jeremy Bernstein, Jiawei Zhao, Markus Meister, Ming-Yu Liu, Anima Anandkumar, Yisong Yue

Abstract: Compositionality is a basic structural feature of both biological and artificial neural networks. Learning compositional functions via gradient descent incurs well known problems like vanishing and exploding gradients, making careful learning rate tuning essential for real-world applications. This paper proves that multiplicative weight updates satisfy a descent lemma tailored to compositional fun… ▽ More Compositionality is a basic structural feature of both biological and artificial neural networks. Learning compositional functions via gradient descent incurs well known problems like vanishing and exploding gradients, making careful learning rate tuning essential for real-world applications. This paper proves that multiplicative weight updates satisfy a descent lemma tailored to compositional functions. Based on this lemma, we derive Madam -- a multiplicative version of the Adam optimiser -- and show that it can train state of the art neural network architectures without learning rate tuning. We further show that Madam is easily adapted to train natively compressed neural networks by representing their weights in a logarithmic number system. We conclude by drawing connections between multiplicative weight updates and recent findings about synapses in biology. △ Less

Submitted 8 January, 2021; v1 submitted 25 June, 2020; originally announced June 2020.

Competitive Policy Optimization

Authors: Manish Prajapat, Kamyar Azizzadenesheli, Alexander Liniger, Yisong Yue, Anima Anandkumar

Abstract: A core challenge in policy optimization in competitive Markov decision processes is the design of efficient optimization methods with desirable convergence and stability properties. To tackle this, we propose competitive policy optimization (CoPO), a novel policy gradient approach that exploits the game-theoretic nature of competitive games to derive policy updates. Motivated by the competitive gr… ▽ More A core challenge in policy optimization in competitive Markov decision processes is the design of efficient optimization methods with desirable convergence and stability properties. To tackle this, we propose competitive policy optimization (CoPO), a novel policy gradient approach that exploits the game-theoretic nature of competitive games to derive policy updates. Motivated by the competitive gradient optimization method, we derive a bilinear approximation of the game objective. In contrast, off-the-shelf policy gradient methods utilize only linear approximations, and hence do not capture interactions among the players. We instantiate CoPO in two ways:(i) competitive policy gradient, and (ii) trust-region competitive policy optimization. We theoretically study these methods, and empirically investigate their behavior on a set of comprehensive, yet challenging, competitive games. We observe that they provide stable optimization, convergence to sophisticated strategies, and higher scores when played against baseline policy gradient methods. △ Less

Submitted 18 June, 2020; originally announced June 2020.

Comments: 11 pages main paper, 6 pages references, and 31 pages appendix. 14 figures

The Power of Predictions in Online Control

Authors: Chenkai Yu, Guanya Shi, Soon-Jo Chung, Yisong Yue, Adam Wierman

Abstract: We study the impact of predictions in online Linear Quadratic Regulator control with both stochastic and adversarial disturbances in the dynamics. In both settings, we characterize the optimal policy and derive tight bounds on the minimum cost and dynamic regret. Perhaps surprisingly, our analysis shows that the conventional greedy MPC approach is a near-optimal policy in both stochastic and adver… ▽ More We study the impact of predictions in online Linear Quadratic Regulator control with both stochastic and adversarial disturbances in the dynamics. In both settings, we characterize the optimal policy and derive tight bounds on the minimum cost and dynamic regret. Perhaps surprisingly, our analysis shows that the conventional greedy MPC approach is a near-optimal policy in both stochastic and adversarial settings. Specifically, for length-$T$ problems, MPC requires only $O(\log T)$ predictions to reach $O(1)$ dynamic regret, which matches (up to lower-order terms) our lower bound on the required prediction horizon for constant regret. △ Less

Submitted 8 January, 2021; v1 submitted 13 June, 2020; originally announced June 2020.

A spatial template independent component analysis model for subject-level brain network estimation and inference

Authors: Amanda F. Mejia, David Bolin, Yu Ryan Yue, Jiongran Wang, Brian S. Caffo, Mary Beth Nebel

Abstract: Independent component analysis is commonly applied to functional magnetic resonance imaging (fMRI) data to extract independent components (ICs) representing functional brain networks. While ICA produces reliable group-level estimates, single-subject ICA often produces noisy results. Template ICA (tICA) is a hierarchical ICA model using empirical population priors to produce reliable subject-level… ▽ More Independent component analysis is commonly applied to functional magnetic resonance imaging (fMRI) data to extract independent components (ICs) representing functional brain networks. While ICA produces reliable group-level estimates, single-subject ICA often produces noisy results. Template ICA (tICA) is a hierarchical ICA model using empirical population priors to produce reliable subject-level IC estimates. However, this and other hierarchical ICA models assume unrealistically that subject effects are spatially independent. Here, we propose spatial template ICA (stICA), which incorporates spatial process priors into tICA. This results in greater estimation efficiency of ICs and subject effects. Additionally, the joint posterior distribution can be used to identify engaged areas using an excursions set approach. By leveraging spatial dependencies and avoiding massive multiple comparisons, stICA has high power to detect true effects. We derive an efficient expectation-maximization algorithm to obtain maximum likelihood estimates of the model parameters and posterior moments of the latent fields. Based on analysis of simulated data and fMRI data from the Human Connectome Project, we find that stICA produces estimates that are more accurate and reliable than benchmark approaches, and identifies larger and more reliable areas of engagement. The algorithm is quite tractable, achieving convergence within 7 hours in our fMRI analysis. △ Less

Submitted 4 June, 2020; v1 submitted 27 May, 2020; originally announced May 2020.

Comments: 32 pages, 16 figures

An in-depth investigation of 11 pulsars discovered by FAST

Authors: A. D. Cameron, D. Li, G. Hobbs, L. Zhang, C. C. Miao, J. B. Wang, M. Yuan, S. Wang, G. Jacobs Corban, M. Cruces, S. Dai, Y. Feng, J. Han, J. F. Kaczmarek, J. R. Nui, Z. C. Pan, L. Qian, Z. Z. Tao, P. Wang, S. Q. Wang, H. Xu, R. X. Xu, Y. L. Yue, S. B. Zhang, Q. J. Zhi , et al. (6 additional authors not shown)

Abstract: We present timing solutions and analyses of 11 pulsars discovered by the Five-hundred-meter Aperture Spherical radio Telescope (FAST). These pulsars were discovered using an ultra-wide bandwidth receiver in drift-scan observations made during the commissioning phase of FAST, and were then confirmed and timed using the 64-m Parkes Radio Telescope. Each pulsar has been observed over a span of at lea… ▽ More We present timing solutions and analyses of 11 pulsars discovered by the Five-hundred-meter Aperture Spherical radio Telescope (FAST). These pulsars were discovered using an ultra-wide bandwidth receiver in drift-scan observations made during the commissioning phase of FAST, and were then confirmed and timed using the 64-m Parkes Radio Telescope. Each pulsar has been observed over a span of at least one year. Highlighted discoveries include PSR J0344-0901, which displays mode-changing behaviour and may belong to the class of so-called `swooshing' pulsars (alongside PSRs B0919+06 and B1859+07); PSR J0803-0942, whose emission is almost completely linearly polarised; and PSRs J1900-0134 and J1945+1211, whose well defined polarisation angle curves place stringent constraints on their emission geometry. We further discuss the detectability of these pulsars by earlier surveys, and highlight lessons learned from our work in carrying out confirmation and monitoring observations of pulsars discovered by a highly sensitive telescope, many of which may be applicable to next-generation pulsar surveys. This paper marks one of the first major releases of FAST-discovered pulsars, and paves the way for future discoveries anticipated from the Commensal Radio Astronomy FAST Survey (CRAFTS). △ Less

Submitted 31 May, 2020; v1 submitted 18 May, 2020; originally announced May 2020.

Comments: 18 pages, 8 figures, 5 tables. Accepted for publication by MNRAS. Correcting error in the caption of Figure 8

Chance-Constrained Trajectory Optimization for Safe Exploration and Learning of Nonlinear Systems

Authors: Yashwanth Kumar Nakka, Anqi Liu, Guanya Shi, Anima Anandkumar, Yisong Yue, Soon-Jo Chung

Abstract: Learning-based control algorithms require data collection with abundant supervision for training. Safe exploration algorithms ensure the safety of this data collection process even when only partial knowledge is available. We present a new approach for optimal motion planning with safe exploration that integrates chance-constrained stochastic optimal control with dynamics learning and feedback con… ▽ More Learning-based control algorithms require data collection with abundant supervision for training. Safe exploration algorithms ensure the safety of this data collection process even when only partial knowledge is available. We present a new approach for optimal motion planning with safe exploration that integrates chance-constrained stochastic optimal control with dynamics learning and feedback control. We derive an iterative convex optimization algorithm that solves an \underline{Info}rmation-cost \underline{S}tochastic \underline{N}onlinear \underline{O}ptimal \underline{C}ontrol problem (Info-SNOC). The optimization objective encodes control cost for performance and exploration cost for learning, and the safety is incorporated as distributionally robust chance constraints. The dynamics are predicted from a robust regression model that is learned from data. The Info-SNOC algorithm is used to compute a sub-optimal pool of safe motion plans that aid in exploration for learning unknown residual dynamics under safety constraints. A stable feedback controller is used to execute the motion plan and collect data for model learning. We prove the safety of rollout from our exploration method and reduction in uncertainty over epochs, thereby guaranteeing the consistency of our learning method. We validate the effectiveness of Info-SNOC by designing and implementing a pool of safe trajectories for a planar robot. We demonstrate that our approach has higher success rate in ensuring safety when compared to a deterministic trajectory optimization approach. △ Less

Submitted 27 October, 2020; v1 submitted 9 May, 2020; originally announced May 2020.

Comments: Accepted IEEE Robotics and Automation Letters 2020

A Fast Radio Burst discovered in FAST drift scan survey

Authors: Weiwei Zhu, Di Li, Rui Luo, Chenchen Miao, Bing Zhang, Laura Spitler, Duncan Lorimer, Michael Kramer, David Champion, Youling Yue, Andrew Cameron, Marilyn Cruces, Ran Duan, Yi Feng, Jun Han, George Hobbs, Chenhui Niu, Jiarui Niu, Zhichen Pan, Lei Qian, Dai Shi, Ningyu Tang, Pei Wang, Hongfeng Wang, Mao Yuan , et al. (45 additional authors not shown)

Abstract: We report the discovery of a highly dispersed fast radio burst, FRB~181123, from an analysis of $\sim$1500~hr of drift-scan survey data taken using the Five-hundred-meter Aperture Spherical radio Telescope (FAST). The pulse has three distinct emission components, which vary with frequency across our 1.0--1.5~GHz observing band. We measure the peak flux density to be $>0.065$~Jy and the correspondi… ▽ More We report the discovery of a highly dispersed fast radio burst, FRB~181123, from an analysis of $\sim$1500~hr of drift-scan survey data taken using the Five-hundred-meter Aperture Spherical radio Telescope (FAST). The pulse has three distinct emission components, which vary with frequency across our 1.0--1.5~GHz observing band. We measure the peak flux density to be $>0.065$~Jy and the corresponding fluence $>0.2$~Jy~ms. Based on the observed dispersion measure of 1812~cm$^{-3}$~pc, we infer a redshift of $\sim 1.9$. From this, we estimate the peak luminosity and isotropic energy to be $\lesssim 2\times10^{43}$~erg~s$^{-1}$ and $\lesssim 2\times10^{40}$~erg, respectively. With only one FRB from the survey detected so far, our constraints on the event rate are limited. We derive a 95\% confidence lower limit for the event rate of 900 FRBs per day for FRBs with fluences $>0.025$~Jy~ms. We performed follow-up observations of the source with FAST for four hours and have not found a repeated burst. We discuss the implications of this discovery for our understanding of the physical mechanisms of FRBs. △ Less

Submitted 29 April, 2020; originally announced April 2020.

Comments: 7 pages, 3 figures, accepted for publication in ApJL

Absence of long-range order in an $XY$ pyrochlore antiferromagnet Er$_2$AlSbO$_7$

Authors: H. L. Che, Z. Y. Zhao, X. Rao, L. G. Chu, N. Li, W. J. Chu, P. Gao, X. Y. Yue, Y. Zhou, Q. J. Li, Q. Huang, E. S. Choi, Y. Y. Han, Z. Z. He, H. D. Zhou, X. Zhao, X. F. Sun

Abstract: Rare-earth pyrochlores are known to exhibit exotic magnetic phenomena. We report a study of crystal growth and characterizations of a new rare-earth compound Er$_2$AlSbO$_7$, in which Al$^{3+}$ and Sb$^{5+}$ ions share the same positions with a random distribution. The magnetism are studied by magnetic susceptibility, specific heat and thermal conductivity measurements at low temperatures down to… ▽ More Rare-earth pyrochlores are known to exhibit exotic magnetic phenomena. We report a study of crystal growth and characterizations of a new rare-earth compound Er$_2$AlSbO$_7$, in which Al$^{3+}$ and Sb$^{5+}$ ions share the same positions with a random distribution. The magnetism are studied by magnetic susceptibility, specific heat and thermal conductivity measurements at low temperatures down to several tens of milli-kelvin. Different from the other reported Er-based pyrochlores exhibiting distinct magnetically ordered states, a spin-freezing transition is detected in Er$_2$AlSbO$_7$ below 0.37 K, which is primarily ascribed to the inherent structural disorder. A cluster spin-glass state is proposed in view of the frequency dependence of the peak position in the ac susceptibility. In addition, the temperature and field dependence of thermal conductivity indicates rather strong spin fluctuations which is probably due to the phase competition. △ Less

Submitted 23 April, 2020; originally announced April 2020.

Comments: 10 pages, 8 figures, accepted for publication Phys. Rev. Materials

Journal ref: Phys. Rev. Materials 4, 054406 (2020)

Polar nano-clusters in nominally paraelectric ceramics demonstrating high microwave tunability for wireless communication

Authors: Hangfeng Zhang, Henry Giddens, Yajun Yue, Xinzhao Xu, Vicente Araullo-Peters, Vladimir Koval, Matteo Palma, Isaac Abrahams, Haixue Yan, Yang Hao

Abstract: Dielectric materials, with high tunability at microwave frequencies, are key components in the design of microwave communication systems. Dense Ba0.6Sr0.4TiO3 (BST) ceramics, with different grain sizes, were prepared in order to optimise the dielectric tunability via polar nano cluster effects. Dielectric permittivity and loss measurements were carried at both high and low frequencies and were sup… ▽ More Dielectric materials, with high tunability at microwave frequencies, are key components in the design of microwave communication systems. Dense Ba0.6Sr0.4TiO3 (BST) ceramics, with different grain sizes, were prepared in order to optimise the dielectric tunability via polar nano cluster effects. Dielectric permittivity and loss measurements were carried at both high and low frequencies and were supported by results from X-ray powder diffraction, scanning and transmission electron microscopies, Raman spectroscopy and piezoresponse force microscopy. The concentration of polar nano clusters, whose sizes are found to be in the range 20 to 50 nm, and the dielectric tunability increase with increasing grain size. A novel method for measurement of the microwave tunability in bulk dielectrics is presented. The highest tunability of 32% is achieved in ceramics with an average grain size of 10 um. The tunability of BST ceramics with applied DC field is demonstrated in a prototype small resonant antenna. △ Less

Submitted 14 April, 2020; originally announced April 2020.

Comments: 25pages, 6 figures

Journal ref: Journal of the European Ceramic Society,2020

Discovery of delayed spin-up behavior following two large glitches in the Crab pulsar, and the statistics of such processes

Authors: M. Y. Ge, S. N. Zhang, F. J. Lu, T. P. Li, J. P. Yuan, X. P. Zheng, Y. Huang, S. J. Zheng, Y. P. Chen, Z. Chang, Y. L. Tuo, Q. Cheng, C. Güngör, L. M. Song, Y. P. Xu, X. L. Cao, Y. Chen, C. Z. Liu, S. Zhang, J. L. Qu, Q. C. Bu, C. Cai, G. Chen, L. Chen, M. Z. Chen , et al. (111 additional authors not shown)

Abstract: Glitches correspond to sudden jumps of rotation frequency ($ν$) and its derivative ($\dotν$) of pulsars, the origin of which remains not well understood yet, partly because the jump processes of most glitches are not well time-resolved. There are three large glitches of the Crab pulsar, detected in 1989, 1996 and 2017, which were found to have delayed spin-up processes before the normal recovery p… ▽ More Glitches correspond to sudden jumps of rotation frequency ($ν$) and its derivative ($\dotν$) of pulsars, the origin of which remains not well understood yet, partly because the jump processes of most glitches are not well time-resolved. There are three large glitches of the Crab pulsar, detected in 1989, 1996 and 2017, which were found to have delayed spin-up processes before the normal recovery processes. Here we report two additional glitches of the Crab pulsar occurred in 2004 and 2011 for which we discovered delayed spin up processes, and present refined parameters of the largest glitch occurred in 2017. The initial rising time of the glitch is determined as $<0.48$ hour. We also carried out a statistical study of these five glitches with observed spin-up processes. The two glitches occurred in 2004 and 2011 have delayed spin-up time scales ($τ_{1}$) of $1.7\pm0.8$\,days and $1.6\pm0.4$\,days, respectively. We find that the $Δν$ vs. $|Δ{\dotν}|$ relation of these five glitches is similar to those with no detected delayed spin-up process, indicating that they are similar to the others in nature except that they have larger amplitudes. For these five glitches, the amplitudes of the delayed spin-up process ($|Δν_{\rm d1}|$) and recovery process ($Δν_{\rm d2}$), their time scales ($τ_{1}$, $τ_{2}$), and permanent changes in spin frequency ($Δν_{\rm p}$) and total frequency step ($Δν_{\rm g}$) have positive correlations. From these correlations, we suggest that the delayed spin-up processes are common for all glitches, but are too short and thus difficult to be detected for most glitches. △ Less

Submitted 1 April, 2020; originally announced April 2020.

Comments: 25 pages, 8 figures

A General Large Neighborhood Search Framework for Solving Integer Linear Programs

Authors: Jialin Song, Ravi Lanka, Yisong Yue, Bistra Dilkina

Abstract: This paper studies a strategy for data-driven algorithm design for large-scale combinatorial optimization problems that can leverage existing state-of-the-art solvers in general purpose ways. The goal is to arrive at new approaches that can reliably outperform existing solvers in wall-clock time. We focus on solving integer programs, and ground our approach in the large neighborhood search (LNS) p… ▽ More This paper studies a strategy for data-driven algorithm design for large-scale combinatorial optimization problems that can leverage existing state-of-the-art solvers in general purpose ways. The goal is to arrive at new approaches that can reliably outperform existing solvers in wall-clock time. We focus on solving integer programs, and ground our approach in the large neighborhood search (LNS) paradigm, which iteratively chooses a subset of variables to optimize while leaving the remainder fixed. The appeal of LNS is that it can easily use any existing solver as a subroutine, and thus can inherit the benefits of carefully engineered heuristic or complete approaches and their software implementations. We show that one can learn a good neighborhood selector using imitation and reinforcement learning techniques. Through an extensive empirical validation in bounded-time optimization, we demonstrate that our LNS framework can significantly outperform compared to state-of-the-art commercial solvers such as Gurobi. △ Less

Submitted 22 December, 2020; v1 submitted 29 March, 2020; originally announced April 2020.

Comments: NeurIPS 2020

Opportunities to Search for Extra-Terrestrial Intelligence with the Five-hundred-meter Aperture Spherical radio Telescope

Authors: Di Li, Vishal Gajjar, Pei Wang, Andrew Siemion, Zhisong Zhang, Haiyan Zhang, Youling Yue, Yan Zhu, Chengjin Jin, Shiyu Li, Sabrina Berger, Bryan Brzycki, Jeff Cobb, Steve Croft, Daniel Czech, David DeBoer, Julia DeMarines, Jamie Drew, J. Emilio Enriquez, Nectaria Gizani, Eric J. Korpela, Howard Isaacson, Matthew Lebofsky, Brian Lacki, David H. E. MacMahon , et al. (9 additional authors not shown)

Abstract: The discovery of ubiquitous habitable extrasolar planets, combined with revolutionary advances in instrumentation and observational capabilities, has ushered in a renaissance in the search for extra-terrestrial intelligence (SETI). Large scale SETI activities are now underway at numerous international facilities. The Five-hundred-meter Aperture Spherical radio Telescope (FAST) is the largest singl… ▽ More The discovery of ubiquitous habitable extrasolar planets, combined with revolutionary advances in instrumentation and observational capabilities, has ushered in a renaissance in the search for extra-terrestrial intelligence (SETI). Large scale SETI activities are now underway at numerous international facilities. The Five-hundred-meter Aperture Spherical radio Telescope (FAST) is the largest single-aperture radio telescope in the world, well positioned to conduct sensitive searches for radio emission indicative of exo-intelligence. SETI is one of the five key science goals specified in the original FAST project plan. A collaboration with the Breakthrough Listen Initiative has been initiated in 2016 with a joint statement signed both by Dr. Jun Yan, the then director of the National Astronomical Observatories, Chinese Academy of Sciences (NAOC), and Dr. Peter Worden, the Chairman of the Breakthrough Prize Foundation. In this paper, we highlight some of the unique features of FAST that will allow for novel SETI observations. We identify and describe three different signal types indicative of a technological source, namely, narrow-band, wide-band artificially dispersed, and modulated signals. We here propose observations with FAST to achieve sensitivities never before explored. △ Less

Submitted 24 March, 2020; v1 submitted 21 March, 2020; originally announced March 2020.

Comments: 14 pages, 8 figures, Accepted by Research in Astron. Astrophys. (FAST special issue)

A Control Barrier Perspective on Episodic Learning via Projection-to-State Safety

Authors: Andrew J. Taylor, Andrew Singletary, Yisong Yue, Aaron D. Ames

Abstract: In this paper we seek to quantify the ability of learning to improve safety guarantees endowed by Control Barrier Functions (CBFs). In particular, we investigate how model uncertainty in the time derivative of a CBF can be reduced via learning, and how this leads to stronger statements on the safe behavior of a system. To this end, we build upon the idea of Input-to-State Safety (ISSf) to define P… ▽ More In this paper we seek to quantify the ability of learning to improve safety guarantees endowed by Control Barrier Functions (CBFs). In particular, we investigate how model uncertainty in the time derivative of a CBF can be reduced via learning, and how this leads to stronger statements on the safe behavior of a system. To this end, we build upon the idea of Input-to-State Safety (ISSf) to define Projection-to-State Safety (PSSf), which characterizes degradation in safety in terms of a projected disturbance. This enables the direct quantification of both how learning can improve safety guarantees, and how bounds on learning error translate to bounds on degradation in safety. We demonstrate that a practical episodic learning approach can use PSSf to reduce uncertainty and improve safety guarantees in simulation and experimentally. △ Less

Submitted 17 March, 2020; originally announced March 2020.

Comments: 6 pages, 2 figures, submitted to L-CSS + CDC 2020

Human Preference-Based Learning for High-dimensional Optimization of Exoskeleton Walking Gaits

Authors: Maegan Tucker, Myra Cheng, Ellen Novoseller, Richard Cheng, Yisong Yue, Joel W. Burdick, Aaron D. Ames

Abstract: Optimizing lower-body exoskeleton walking gaits for user comfort requires understanding users' preferences over a high-dimensional gait parameter space. However, existing preference-based learning methods have only explored low-dimensional domains due to computational limitations. To learn user preferences in high dimensions, this work presents LineCoSpar, a human-in-the-loop preference-based fram… ▽ More Optimizing lower-body exoskeleton walking gaits for user comfort requires understanding users' preferences over a high-dimensional gait parameter space. However, existing preference-based learning methods have only explored low-dimensional domains due to computational limitations. To learn user preferences in high dimensions, this work presents LineCoSpar, a human-in-the-loop preference-based framework that enables optimization over many parameters by iteratively exploring one-dimensional subspaces. Additionally, this work identifies gait attributes that characterize broader preferences across users. In simulations and human trials, we empirically verify that LineCoSpar is a sample-efficient approach for high-dimensional preference optimization. Our analysis of the experimental data reveals a correspondence between human preferences and objective measures of dynamicity, while also highlighting differences in the utility functions underlying individual users' gait preferences. This result has implications for exoskeleton gait synthesis, an active field with applications to clinical use and patient rehabilitation. △ Less

Submitted 8 August, 2020; v1 submitted 13 March, 2020; originally announced March 2020.

Comments: 8 pages, 9 figures, 2 tables, to appear at IROS 2020

Neural-Swarm: Decentralized Close-Proximity Multirotor Control Using Learned Interactions

Authors: Guanya Shi, Wolfgang Hönig, Yisong Yue, Soon-Jo Chung

Abstract: In this paper, we present Neural-Swarm, a nonlinear decentralized stable controller for close-proximity flight of multirotor swarms. Close-proximity control is challenging due to the complex aerodynamic interaction effects between multirotors, such as downwash from higher vehicles to lower ones. Conventional methods often fail to properly capture these interaction effects, resulting in controllers… ▽ More In this paper, we present Neural-Swarm, a nonlinear decentralized stable controller for close-proximity flight of multirotor swarms. Close-proximity control is challenging due to the complex aerodynamic interaction effects between multirotors, such as downwash from higher vehicles to lower ones. Conventional methods often fail to properly capture these interaction effects, resulting in controllers that must maintain large safety distances between vehicles, and thus are not capable of close-proximity flight. Our approach combines a nominal dynamics model with a regularized permutation-invariant Deep Neural Network (DNN) that accurately learns the high-order multi-vehicle interactions. We design a stable nonlinear tracking controller using the learned model. Experimental results demonstrate that the proposed controller significantly outperforms a baseline nonlinear tracking controller with up to four times smaller worst-case height tracking errors. We also empirically demonstrate the ability of our learned model to generalize to larger swarm sizes. △ Less

Submitted 5 March, 2020; originally announced March 2020.

Comments: International Conference on Robotics and Automation (ICRA), 2020, https://youtu.be/v4j-9pH11Q8

GLAS: Global-to-Local Safe Autonomy Synthesis for Multi-Robot Motion Planning with End-to-End Learning

Authors: Benjamin Rivière, Wolfgang Hoenig, Yisong Yue, Soon-Jo Chung

Abstract: We present GLAS: Global-to-Local Autonomy Synthesis, a provably-safe, automated distributed policy generation for multi-robot motion planning. Our approach combines the advantage of centralized planning of avoiding local minima with the advantage of decentralized controllers of scalability and distributed computation. In particular, our synthesized policies only require relative state information… ▽ More We present GLAS: Global-to-Local Autonomy Synthesis, a provably-safe, automated distributed policy generation for multi-robot motion planning. Our approach combines the advantage of centralized planning of avoiding local minima with the advantage of decentralized controllers of scalability and distributed computation. In particular, our synthesized policies only require relative state information of nearby neighbors and obstacles, and compute a provably-safe action. Our approach has three major components: i) we generate demonstration trajectories using a global planner and extract local observations from them, ii) we use deep imitation learning to learn a decentralized policy that can run efficiently online, and iii) we introduce a novel differentiable safety module to ensure collision-free operation, thereby allowing for end-to-end policy training. Our numerical experiments demonstrate that our policies have a 20% higher success rate than optimal reciprocal collision avoidance, ORCA, across a wide range of robot and obstacle densities. We demonstrate our method on an aerial swarm, executing the policy on low-end microcontrollers in real-time. △ Less

Submitted 28 June, 2020; v1 submitted 26 February, 2020; originally announced February 2020.

Comments: Accepted at IEEE RA-L, see DOI below

The FAST discovery of an Eclipsing Binary Millisecond Pulsar in the Globular Cluster M92 (NGC 6341)

Authors: Zhichen Pan, Scott M. Ransom, Duncan R. Lorimer, William Fiore, Lei Qian, Lin Wang, Benjamin W. Stappers, George Hobbs, Weiwei Zhu, Youling Yue, Pei Wang, Jiguang Lu, Kuo Liu, Bo Peng, Lei Zhang, Di Li

Abstract: We report the discovery of an eclipsing binary millisecond pulsar in the globular cluster M92 (NGC6341) with the Five-hundred-meter Aperture Spherical radio Telescope (FAST). PSR J1717+4308A, or M92A, has a pulse frequency of 316.5~Hz (3.16~ms) and a dispersion measure of 35.45 pc cm$^{-3}$. The pulsar is a member of a binary system with an orbital period of 0.20~days around a low-mass companion w… ▽ More We report the discovery of an eclipsing binary millisecond pulsar in the globular cluster M92 (NGC6341) with the Five-hundred-meter Aperture Spherical radio Telescope (FAST). PSR J1717+4308A, or M92A, has a pulse frequency of 316.5~Hz (3.16~ms) and a dispersion measure of 35.45 pc cm$^{-3}$. The pulsar is a member of a binary system with an orbital period of 0.20~days around a low-mass companion which has a median mass of $\sim$0.18~\Ms. From observations so far, at least two eclipsing events have been observed in each orbit. The longer one lasted for ~5000~s in the orbital phase range 0.1--0.5. The other lasted for ~500~s and occurred between 1000--2000~s before or after the longer eclipsing event. The lengths of these two eclipsing events also change. These properties suggest that J1717+4308A is a ``red-back'' system with a low-mass main sequence or sub-giant companion. Timing observations of the pulsar and further searches of the data for additional pulsars are ongoing. △ Less

Submitted 24 February, 2020; originally announced February 2020.

Comments: 9 pages, 2 figures, submitted to ApJL, comments are welcomed!

Discovery and timing of pulsars in the globular cluster M13 with FAST

Authors: Lin Wang, Bo Peng, B. W. Stappers, Kuo Liu, M. J. Keith, A. G. Lyne, Jiguang Lu, Ye-Zhao Yu, Feifei Kou, Jun Yan, Peng Jiang, Chengjin Jin, Di Li, Qi Li, Lei Qian, Qiming Wang, Youling Yue, Haiyan Zhang, Shuxin Zhang, Yan Zhu

Abstract: We report the discovery of a binary millisecond pulsar (namely PSR J1641+3627F or M13F) in the globular cluster M13 (NGC 6205) and timing solutions of M13A to F using observations made with the Five-hundred-metre Aperture Spherical radio Telescope (FAST). PSR J1641+3627F has a spin period of 3.00 ms and an orbital period of 1.4 days. The most likely companion mass is 0.16 M$_{\odot}$. M13A to E al… ▽ More We report the discovery of a binary millisecond pulsar (namely PSR J1641+3627F or M13F) in the globular cluster M13 (NGC 6205) and timing solutions of M13A to F using observations made with the Five-hundred-metre Aperture Spherical radio Telescope (FAST). PSR J1641+3627F has a spin period of 3.00 ms and an orbital period of 1.4 days. The most likely companion mass is 0.16 M$_{\odot}$. M13A to E all have short spin periods and small period derivatives. We also confirm that the binary millisecond pulsar PSR J1641$+$3627E (also M13E) is a black widow with a companion mass around 0.02 M$_{\odot}$. We find that all the binary systems have low eccentricities compared to those typical for globular cluster pulsars and that they decrease with distance from the cluster core. This is consistent with what is expected as this cluster has a very low encounter rate per binary. △ Less

Submitted 14 February, 2020; originally announced February 2020.

Comments: 12 pages, 9 figures, accepted for publication in ApJ

Multiresolution Tensor Learning for Efficient and Interpretable Spatial Analysis

Authors: Jung Yeon Park, Kenneth Theo Carr, Stephan Zheng, Yisong Yue, Rose Yu

Abstract: Efficient and interpretable spatial analysis is crucial in many fields such as geology, sports, and climate science. Tensor latent factor models can describe higher-order correlations for spatial data. However, they are computationally expensive to train and are sensitive to initialization, leading to spatially incoherent, uninterpretable results. We develop a novel Multiresolution Tensor Learning… ▽ More Efficient and interpretable spatial analysis is crucial in many fields such as geology, sports, and climate science. Tensor latent factor models can describe higher-order correlations for spatial data. However, they are computationally expensive to train and are sensitive to initialization, leading to spatially incoherent, uninterpretable results. We develop a novel Multiresolution Tensor Learning (MRTL) algorithm for efficiently learning interpretable spatial patterns. MRTL initializes the latent factors from an approximate full-rank tensor model for improved interpretability and progressively learns from a coarse resolution to the fine resolution to reduce computation. We also prove the theoretical convergence and computational complexity of MRTL. When applied to two real-world datasets, MRTL demonstrates 4~5x speedup compared to a fixed resolution approach while yielding accurate and interpretable latent factors. △ Less

Submitted 14 August, 2020; v1 submitted 13 February, 2020; originally announced February 2020.

Comments: ICML 2020

Online Optimization with Memory and Competitive Control

Authors: Guanya Shi, Yiheng Lin, Soon-Jo Chung, Yisong Yue, Adam Wierman

Abstract: This paper presents competitive algorithms for a novel class of online optimization problems with memory. We consider a setting where the learner seeks to minimize the sum of a hitting cost and a switching cost that depends on the previous $p$ decisions. This setting generalizes Smoothed Online Convex Optimization. The proposed approach, Optimistic Regularized Online Balanced Descent, achieves a c… ▽ More This paper presents competitive algorithms for a novel class of online optimization problems with memory. We consider a setting where the learner seeks to minimize the sum of a hitting cost and a switching cost that depends on the previous $p$ decisions. This setting generalizes Smoothed Online Convex Optimization. The proposed approach, Optimistic Regularized Online Balanced Descent, achieves a constant, dimension-free competitive ratio. Further, we show a connection between online optimization with memory and online control with adversarial disturbances. This connection, in turn, leads to a new constant-competitive policy for a rich class of online control problems. △ Less

Submitted 8 January, 2021; v1 submitted 12 February, 2020; originally announced February 2020.

Comments: Neural Information Processing Systems (NeurIPS 2020)

Discrete Action On-Policy Learning with Action-Value Critic

Authors: Yuguang Yue, Yunhao Tang, Mingzhang Yin, Mingyuan Zhou

Abstract: Reinforcement learning (RL) in discrete action space is ubiquitous in real-world applications, but its complexity grows exponentially with the action-space dimension, making it challenging to apply existing on-policy gradient based deep RL algorithms efficiently. To effectively operate in multidimensional discrete action spaces, we construct a critic to estimate action-value functions, apply it on… ▽ More Reinforcement learning (RL) in discrete action space is ubiquitous in real-world applications, but its complexity grows exponentially with the action-space dimension, making it challenging to apply existing on-policy gradient based deep RL algorithms efficiently. To effectively operate in multidimensional discrete action spaces, we construct a critic to estimate action-value functions, apply it on correlated actions, and combine these critic estimated action values to control the variance of gradient estimation. We follow rigorous statistical analysis to design how to generate and combine these correlated actions, and how to sparsify the gradients by shutting down the contributions from certain dimensions. These efforts result in a new discrete action on-policy RL algorithm that empirically outperforms related on-policy algorithms relying on variance control techniques. We demonstrate these properties on OpenAI Gym benchmark tasks, and illustrate how discretizing the action space could benefit the exploration phase and hence facilitate convergence to a better local optimal solution thanks to the flexibility of discrete policy. △ Less

Submitted 21 February, 2020; v1 submitted 9 February, 2020; originally announced February 2020.

On the distance between two neural networks and the stability of learning

Authors: Jeremy Bernstein, Arash Vahdat, Yisong Yue, Ming-Yu Liu

Abstract: This paper relates parameter distance to gradient breakdown for a broad class of nonlinear compositional functions. The analysis leads to a new distance function called deep relative trust and a descent lemma for neural networks. Since the resulting learning rule seems to require little to no learning rate tuning, it may unlock a simpler workflow for training deeper and more complex neural network… ▽ More This paper relates parameter distance to gradient breakdown for a broad class of nonlinear compositional functions. The analysis leads to a new distance function called deep relative trust and a descent lemma for neural networks. Since the resulting learning rule seems to require little to no learning rate tuning, it may unlock a simpler workflow for training deeper and more complex neural networks. The Python code used in this paper is here: https://github.com/jxbz/fromage. △ Less

Submitted 8 January, 2021; v1 submitted 9 February, 2020; originally announced February 2020.

The Fundamental Performance of FAST with 19-beam Receiver at L Band

Authors: Peng Jiang, Ning-Yu Tang, Li-Gang Hou, Meng-Ting Liu, Marko Krco, Lei Qian, Jing-Hai Sun, Tao-Chung Ching, Bin Liu, Yan Duan, You-Ling Yue, Heng-Qian Gan, Rui Yao, Hui Li, Gao-Feng Pan, Dong-Jun Yu, Hong-Fei Liu, Di Li, Bo Peng, Jun Yan, FAST Collaboration

Abstract: The Five-hundred-meter Aperture Spherical radio Telescope (FAST) passed national acceptance and is taking pilot cycle of 'Shared-Risk' observations. The 19-beam receiver covering 1.05-1.45 GHz was used for most of these observations. The electronics gain fluctuation of the system is better than 1\% over 3.5 hours, enabling enough stability for observations. Pointing accuracy, aperture efficiency a… ▽ More The Five-hundred-meter Aperture Spherical radio Telescope (FAST) passed national acceptance and is taking pilot cycle of 'Shared-Risk' observations. The 19-beam receiver covering 1.05-1.45 GHz was used for most of these observations. The electronics gain fluctuation of the system is better than 1\% over 3.5 hours, enabling enough stability for observations. Pointing accuracy, aperture efficiency and system temperature are three key parameters of FAST. The measured standard deviation of pointing accuracy is 7.9$''$, which satisfies the initial design of FAST. When zenith angle is less than 26.4$^\circ$, the aperture efficiency and system temperature around 1.4 GHz are $\sim$ 0.63 and less than 24 K for central beam, respectively. The measured value of these two parameters are better than designed value of 0.6 and 25 K, respectively. The sensitivity and stability of the 19-beam backend are confirmed to satisfy expectation by spectral HI observations toward N672 and polarization observations toward 3C286. The performance allows FAST to take sensitive observations in various scientific goals, from studies of pulsar to galaxy evolution. △ Less

Submitted 5 February, 2020; originally announced February 2020.

Comments: 29 pages, 28 figures, accepted by Research in Astronomy and Astrophysics

A PRESTO-based Parallel Pulsar Search Pipeline Used for FAST Drift Scan Data

Authors: Qiuyu Yu, Zhichen Pan, Lei Qian, Shen Wang, Youling Yue, Menglin Huang, Qiaoli Hao, Shanping You, Bo Peng, Yan Zhu, Lei Zhang, Zhijie Liu

Abstract: We developed a pulsar search pipeline based on PRESTO (PulsaR Exploration and Search Toolkit). This pipeline simply runs dedispersion, FFT (Fast Fourier Transformation), and acceleration search in process-level parallel to shorten the processing time. With two parallel strategies, the pipeline can highly shorten the processing time in both the normal searches or acceleration searches. This pipelin… ▽ More We developed a pulsar search pipeline based on PRESTO (PulsaR Exploration and Search Toolkit). This pipeline simply runs dedispersion, FFT (Fast Fourier Transformation), and acceleration search in process-level parallel to shorten the processing time. With two parallel strategies, the pipeline can highly shorten the processing time in both the normal searches or acceleration searches. This pipeline was first tested with PMPS (Parkes Multibeam Pulsar Survery) data and discovered two new faint pulsars. Then, it was successfully used in processing the FAST (Five-hundred-meter Aperture Spherical radio Telescope) drift scan data with tens of new pulsar discoveries up to now. The pipeline is only CPU-based and can be easily and quickly deployed in computing nodes for testing purposes or data processes. △ Less

Submitted 29 December, 2019; originally announced December 2019.

Comments: 10 pages, 2 figures, accepted by Research in Astronomy and Astrophysics (RAA)

Algebraic representation of continuous lattices via the open filter monad, revisited

Authors: Wei Yao, Yueli Yue

Abstract: In [A. Day, Filter monads, continuous lattices and closure systems, Can. J. Math. 27 (1975) 50--59], Day showed that continuous lattices are precisely the algebras of the open filter monad over the category of $T_0$ spaces. The aim of this paper is to give a clean and clear version of the whole process of Day's approach. In [A. Day, Filter monads, continuous lattices and closure systems, Can. J. Math. 27 (1975) 50--59], Day showed that continuous lattices are precisely the algebras of the open filter monad over the category of $T_0$ spaces. The aim of this paper is to give a clean and clear version of the whole process of Day's approach. △ Less

Submitted 15 December, 2019; originally announced December 2019.

Dynamic manipulation of friction in smart textile composites of liquid-crystal elastomers

Authors: Takuya Ohzono, Mohand O. Saed, Youfeng Yue, Yasuo Norikane, Eugene M. Terentjev

Abstract: Smart surfaces that reversibly change the interfacial friction coefficients in response to external stimuli enable a wide range of applications, such as grips, seals, brake pads, packaging films, and fabrics. Here a new concept of such a smart frictional system is reported: a composite film of a plain-weave polyester textile sheet, and a thermo-responsive nematic liquid crystalline elastomer (LCE)… ▽ More Smart surfaces that reversibly change the interfacial friction coefficients in response to external stimuli enable a wide range of applications, such as grips, seals, brake pads, packaging films, and fabrics. Here a new concept of such a smart frictional system is reported: a composite film of a plain-weave polyester textile sheet, and a thermo-responsive nematic liquid crystalline elastomer (LCE). The composite is deployed with retractable micro-undulations of the elastomer inside each weave mesh, enabling dramatic changes of the contact interface with the opposing surface on LCE actuation, which is induced e.g. by a change in temperature (T). At ambient T, the protruding viscoelastic parts of LCE in the nematic phase make contact with the opposing flat surface, resulting in a very high friction. At an elevated T (50C, isotropic phase), the undulations of LCE surface are retracted within the thickness of the textile, and the contacts are limited to small regions around overlapping textile fibers, lowering the friction dramatically. This effect is fully reversible on heating/cooling cycles. The surface undulations are spontaneous, i.e. fabricated without any lithographic or alignment techniques. The present composite opens a new way to practical uses of sheets/films with switchable friction enabled by stimuli-responsive LCEs. △ Less

Submitted 23 December, 2019; originally announced December 2019.

Journal ref: Adv. Mater. Interfaces 2020, 7, 1901996

Anomalous Muon Magnetic Moment in the Inverse Seesaw Extended Next-to-Minimal Supersymmetric Standard Model

Authors: Junjie Cao, Jingwei Lian, Lei Meng, Yuanfang Yue, Pengxuan Zhu

Abstract: The present work investigates the possibility that both dark matter and the anomalous magnetic moment of the muon may be explained within the context of the inverse seesaw extended Next-to-Minimal Supersymmetric Standard Model (ISS-NMSSM). In ISS-NMSSM, the newly introduced Higgs-neutrino Yukawa coupling $Y_ν$ provides additional Higgsino-sneutrino loop contribution to $(g-2)_μ$. If the deviation… ▽ More The present work investigates the possibility that both dark matter and the anomalous magnetic moment of the muon may be explained within the context of the inverse seesaw extended Next-to-Minimal Supersymmetric Standard Model (ISS-NMSSM). In ISS-NMSSM, the newly introduced Higgs-neutrino Yukawa coupling $Y_ν$ provides additional Higgsino-sneutrino loop contribution to $(g-2)_μ$. If the deviation between the experimental observations and the Standard Model predictions of the anomalous muon magnetic moment is confirmed by the further experimental and theoretical studies, it can be explained naturally within the ISS-NMSSM framework without conflicting with the current stringent limits on the direct detection of dark matter and Large Hadron Collider searches. △ Less

Submitted 24 April, 2020; v1 submitted 21 December, 2019; originally announced December 2019.

Comments: 18 pages, 4 figures

Journal ref: Phys. Rev. D 101, 095009 (2020)

Learning for Safety-Critical Control with Control Barrier Functions

Authors: Andrew Taylor, Andrew Singletary, Yisong Yue, Aaron Ames

Abstract: Modern nonlinear control theory seeks to endow systems with properties of stability and safety, and have been deployed successfully in multiple domains. Despite this success, model uncertainty remains a significant challenge in synthesizing safe controllers, leading to degradation in the properties provided by the controllers. This paper develops a machine learning framework utilizing Control Barr… ▽ More Modern nonlinear control theory seeks to endow systems with properties of stability and safety, and have been deployed successfully in multiple domains. Despite this success, model uncertainty remains a significant challenge in synthesizing safe controllers, leading to degradation in the properties provided by the controllers. This paper develops a machine learning framework utilizing Control Barrier Functions (CBFs) to reduce model uncertainty as it impact the safe behavior of a system. This approach iteratively collects data and updates a controller, ultimately achieving safe behavior. We validate this method in simulation and experimentally on a Segway platform. △ Less

Submitted 20 December, 2019; originally announced December 2019.

Comments: Extended version (12 Pages), Short version submitted to Learning for Dynamics & Control (L4DC) 2020 Conference

Algebraic representation of L-valued continuous lattices via the open filter monad

Authors: Wei Yao, Yueli Yue, Bin Pang

Abstract: With a complete Heyting algebra $L$ as the truth value table, we prove that the collections of open filters of stratified $L$-valued topological spaces form a monad. By means of $L$-Scott topology and the specialization $L$-order, we get that the algebras of open filter monad are precisely $L$-continuous lattices. With a complete Heyting algebra $L$ as the truth value table, we prove that the collections of open filters of stratified $L$-valued topological spaces form a monad. By means of $L$-Scott topology and the specialization $L$-order, we get that the algebras of open filter monad are precisely $L$-continuous lattices. △ Less

Submitted 7 December, 2019; originally announced December 2019.

Comments: 5 figures

Pilot HI Survey of Planck Galactic Cold Clumps with FAST

Authors: Ningyu Tang, Di Li, Pei Zuo, Lei Qian, Tie Liu, Yuefang Wu, Marko Krčo, Mengting Liu, Youling Yue, Yan Zhu, Hongfei Liu, Dongjun Yu, Jinghai Sun, Peng Jiang, Gaofeng Pan, Hui Li, Hengqian Gan, Rui Yao, Shu Liu, FAST Collaboration

Abstract: We present a pilot HI survey of 17 Planck Galactic Cold Clumps (PGCCs) with the Five-hundred-meter Aperture Spherical radio Telescope (FAST). HI Narrow Self-Absorption (HINSA) is an effective method to detect cold HI being mixed with molecular hydrogen H$_2$ and improves our understanding of the atomic to molecular transition in the interstellar medium. HINSA was found in 58\% PGCCs that we observ… ▽ More We present a pilot HI survey of 17 Planck Galactic Cold Clumps (PGCCs) with the Five-hundred-meter Aperture Spherical radio Telescope (FAST). HI Narrow Self-Absorption (HINSA) is an effective method to detect cold HI being mixed with molecular hydrogen H$_2$ and improves our understanding of the atomic to molecular transition in the interstellar medium. HINSA was found in 58\% PGCCs that we observed. The column density of HINSA was found to have an intermediate correlation with that of $^{13}$CO, following $\rm log( N(HINSA)) = (0.52\pm 0.26) log(N_{^{13}CO}) + (10 \pm 4.1) $. HI abundance relative to total hydrogen [HI]/[H] has an average value of $4.4\times 10^{-3}$, which is about 2.8 times of the average value of previous HINSA surveys toward molecular clouds. For clouds with total column density N$\rm_H >5 \times 10^{20}$ cm$^{-2}$, an inverse correlation between HINSA abundance and total hydrogen column density is found, confirming the depletion of cold HI gas during molecular gas formation in more massive clouds. Nonthermal line width of $^{13}$CO is about 0-0.5 km s$^{-1}$ larger than that of HINSA. One possible explanation of narrower nonthermal width of HINSA is that HINSA region is smaller than that of $^{13}$CO. Based on an analytic model of H$_2$ formation and H$_2$ dissociation by cosmic ray, we found the cloud ages to be within 10$^{6.7}$-10$^{7.0}$ yr for five sources. △ Less

Submitted 2 December, 2019; originally announced December 2019.

Comments: 11 pages, 11 figures, accepted by the Research in Astronomy and Astrophysics (RAA)

Possible itinerant excitations and quantum spin state transitions in the effective spin-1/2 triangular-lattice antiferromagnet Na$_2$BaCo(PO$_4$)$_2$

Authors: N. Li, Q. Huang, X. Y. Yue, W. J. Chu, Q. Chen, E. S. Choi, X. Zhao, H. D. Zhou, X. F. Sun

Abstract: The most fascinating feature of certain two-dimensional (2D) gapless quantum spin liquid (QSL) is that their spinon excitations behave like the fermionic carriers of a paramagnetic metal. The spinon Fermi surface is then expected to produce a linear increase of the thermal conductivity with temperature that should manifest via a residual value ($κ_0/T$) in the zero-temperature limit. However, this… ▽ More The most fascinating feature of certain two-dimensional (2D) gapless quantum spin liquid (QSL) is that their spinon excitations behave like the fermionic carriers of a paramagnetic metal. The spinon Fermi surface is then expected to produce a linear increase of the thermal conductivity with temperature that should manifest via a residual value ($κ_0/T$) in the zero-temperature limit. However, this linear in T behavior has been reported for very few QSL candidates. Here, we studied the ultralow-temperature thermal conductivity of an effective spin-1/2 triangular QSL candidate Na$_2$BaCo(PO$_4$)$_2$, which has an antiferromagnetic order at very low temperature ($T_N \sim$ 148 mK), and observed a finite $κ_0/T$ extrapolated from the data above $T_N$. Moreover, while approaching zero temperature, it exhibits series of quantum spin state transitions with applied field along the $c$ axis. These observations indicate that Na$_2$BaCo(PO$_4$)$_2$ possibly behaves as a gapless QSL with itinerant spin excitations above $T_N$ and its strong quantum spin fluctuations persist below $T_N$. △ Less

Submitted 16 September, 2020; v1 submitted 25 November, 2019; originally announced November 2019.

Comments: 24 pages, 5 figures, with Supplementary Information

Journal ref: Nature Commun. 11, 4216 (2020)

Empirical Study of Off-Policy Policy Evaluation for Reinforcement Learning

Authors: Cameron Voloshin, Hoang M. Le, Nan Jiang, Yisong Yue

Abstract: We offer an experimental benchmark and empirical study for off-policy policy evaluation (OPE) in reinforcement learning, which is a key problem in many safety critical applications. Given the increasing interest in deploying learning-based methods, there has been a flurry of recent proposals for OPE method, leading to a need for standardized empirical analyses. Our work takes a strong focus on div… ▽ More We offer an experimental benchmark and empirical study for off-policy policy evaluation (OPE) in reinforcement learning, which is a key problem in many safety critical applications. Given the increasing interest in deploying learning-based methods, there has been a flurry of recent proposals for OPE method, leading to a need for standardized empirical analyses. Our work takes a strong focus on diversity of experimental design to enable stress testing of OPE methods. We provide a comprehensive benchmarking suite to study the interplay of different attributes on method performance. We distill the results into a summarized set of guidelines for OPE in practice. Our software package, the Caltech OPE Benchmarking Suite (COBS), is open-sourced and we invite interested researchers to further contribute to the benchmark. △ Less

Submitted 27 November, 2021; v1 submitted 15 November, 2019; originally announced November 2019.

Triply Robust Off-Policy Evaluation

Authors: Anqi Liu, Hao Liu, Anima Anandkumar, Yisong Yue

Abstract: We propose a robust regression approach to off-policy evaluation (OPE) for contextual bandits. We frame OPE as a covariate-shift problem and leverage modern robust regression tools. Ours is a general approach that can be used to augment any existing OPE method that utilizes the direct method. When augmenting doubly robust methods, we call the resulting method Triply Robust. We prove upper bounds o… ▽ More We propose a robust regression approach to off-policy evaluation (OPE) for contextual bandits. We frame OPE as a covariate-shift problem and leverage modern robust regression tools. Ours is a general approach that can be used to augment any existing OPE method that utilizes the direct method. When augmenting doubly robust methods, we call the resulting method Triply Robust. We prove upper bounds on the resulting bias and variance, as well as derive novel minimax bounds based on robust minimax analysis for covariate shift. Our robust regression method is compatible with deep learning, and is thus applicable to complex OPE settings that require powerful function approximators. Finally, we demonstrate superior empirical performance across the standard OPE benchmarks, especially in the case where the logging policy is unknown and must be estimated from data. △ Less

Submitted 15 November, 2019; v1 submitted 13 November, 2019; originally announced November 2019.

Comments: Preliminary Work

Big Data Challenges of FAST

Authors: Youling Yue, Di Li

Abstract: We present the big-data challenges posed by the science operation of the Five-hundred-meter Aperture Spherical radio Telescope (FAST). Unlike the common usage of the word `big-data', which tend to emphasize both quantity and diversity, the main characteristics of FAST data stream is its single-source data rate at more than 6 GB/s and the resulting data volume at about 20 PB per year. We describe h… ▽ More We present the big-data challenges posed by the science operation of the Five-hundred-meter Aperture Spherical radio Telescope (FAST). Unlike the common usage of the word `big-data', which tend to emphasize both quantity and diversity, the main characteristics of FAST data stream is its single-source data rate at more than 6 GB/s and the resulting data volume at about 20 PB per year. We describe here the main culprit of such a high data rate and large volume, namely pulsar search, and our solution. △ Less

Submitted 12 November, 2019; originally announced November 2019.

Comments: 4 pages, 1 figure

Journal ref: Big Scientific Data Management. BigSDM 2018. Lecture Notes in Computer Science, vol 11473. Springer, Cham