Showing 101–150 of 273 results for author: Tao, M

Insights from an Industrial Collaborative Assembly Project: Lessons in Research and Collaboration

Authors: Tan Chen, Zhe Huang, James Motes, Junyi Geng, Quang Minh Ta, Holly Dinkel, Hameed Abdul-Rashid, Jessica Myers, Ye-Ji Mun, Wei-che Lin, Yuan-yung Huang, Sizhe Liu, Marco Morales, Nancy M. Amato, Katherine Driggs-Campbell, Timothy Bretl

Abstract: Significant progress in robotics reveals new opportunities to advance manufacturing. Next-generation industrial automation will require both integration of distinct robotic technologies and their application to challenging industrial environments. This paper presents lessons from a collaborative assembly project between three academic research groups and an industry partner. The goal of the projec… ▽ More Significant progress in robotics reveals new opportunities to advance manufacturing. Next-generation industrial automation will require both integration of distinct robotic technologies and their application to challenging industrial environments. This paper presents lessons from a collaborative assembly project between three academic research groups and an industry partner. The goal of the project is to develop a flexible, safe, and productive manufacturing cell for sub-centimeter precision assembly. Solving this problem in a high-mix, low-volume production line motivates multiple research thrusts in robotics. This work identifies new directions in collaborative robotics for industrial applications and offers insight toward strengthening collaborations between institutions in academia and industry on the development of new technologies. △ Less

Submitted 28 May, 2022; originally announced May 2022.

Comments: Spotlight presentation at ICRA 2022 Workshop on Collaborative Robots and the Work of the Future (ICRA 2022 CoR-WotF); see the spotlight presentation at https://sites.google.com/view/icra22ws-cor-wotf/accepted-papers?authuser=0

Momentum Stiefel Optimizer, with Applications to Suitably-Orthogonal Attention, and Optimal Transport

Authors: Lingkai Kong, Yuqing Wang, Molei Tao

Abstract: The problem of optimization on Stiefel manifold, i.e., minimizing functions of (not necessarily square) matrices that satisfy orthogonality constraints, has been extensively studied. Yet, a new approach is proposed based on, for the first time, an interplay between thoughtfully designed continuous and discrete dynamics. It leads to a gradient-based optimizer with intrinsically added momentum. This… ▽ More The problem of optimization on Stiefel manifold, i.e., minimizing functions of (not necessarily square) matrices that satisfy orthogonality constraints, has been extensively studied. Yet, a new approach is proposed based on, for the first time, an interplay between thoughtfully designed continuous and discrete dynamics. It leads to a gradient-based optimizer with intrinsically added momentum. This method exactly preserves the manifold structure but does not require additional operation to keep momentum in the changing (co)tangent space, and thus has low computational cost and pleasant accuracy. Its generalization to adaptive learning rates is also demonstrated. Notable performances are observed in practical tasks. For instance, we found that placing orthogonal constraints on attention heads of trained-from-scratch Vision Transformer [Dosovitskiy et al. 2022] could markedly improve its performance, when our optimizer is used, and it is better that each head is made orthogonal within itself but not necessarily to other heads. This optimizer also makes the useful notion of Projection Robust Wasserstein Distance [Paty & Cuturi 2019; Lin et al. 2020] for high-dim. optimal transport even more effective. △ Less

Submitted 2 March, 2023; v1 submitted 27 May, 2022; originally announced May 2022.

Comments: Code: https://github.com/konglk1203/VariationalStiefelOptimizer

Journal ref: ICLR 2023

Ergodic Achievable Rate Maximization of RIS-assisted Millimeter-Wave MIMO-OFDM Communication Systems

Authors: Renwang Li, Shu Sun, Meixia Tao

Abstract: Reconfigurable intelligent surface (RIS) has attracted extensive attention in recent years. However, most research focuses on the scenario of the narrowband and/or instantaneous channel state information (CSI), while wide bandwidth with the use of millimeter-wave (mmWave) (including sub-Terahertz) spectrum is a major trend in next-generation wireless communications, and statistical CSI is more pra… ▽ More Reconfigurable intelligent surface (RIS) has attracted extensive attention in recent years. However, most research focuses on the scenario of the narrowband and/or instantaneous channel state information (CSI), while wide bandwidth with the use of millimeter-wave (mmWave) (including sub-Terahertz) spectrum is a major trend in next-generation wireless communications, and statistical CSI is more practical to obtain in realistic systems. Thus, we {consider} the ergodic achievable rate of RIS-assisted mmWave multiple-input multiple-output orthogonal frequency division multiplexing communication systems. The widely used Saleh-Valenzuela channel model is adopted to characterize the mmWave channels and only the statistical CSI is available. We first derive the approximations of the ergodic achievable rate by means of the majorization theory and Jensen's inequality. Then, an alternating optimization based algorithm is proposed to maximize the ergodic achievable rate by jointly designing the transmit covariance matrix at the base station and the reflection coefficients at the RIS. Specifically, the design of the transmit covariance matrix is transformed into a power allocation problem and solved by spatial-frequency water-filling. The reflection coefficients are optimized by the Riemannian conjugate gradient algorithm. Simulation results corroborate the effectiveness of the proposed algorithms. △ Less

Submitted 3 November, 2022; v1 submitted 26 May, 2022; originally announced May 2022.

Comments: submitted for possible publication. in IEEE Transactions on Wireless Communications, 2022

Hybrid Spherical- and Planar-Wave Channel Modeling and Estimation for Terahertz Integrated UM-MIMO and IRS Systems

Authors: Yuhang Chen, Renwang Li, Chong Han, Shu Sun, Meixia Tao

Abstract: Integrated ultra-massive multiple-input multiple-output (UM-MIMO) and intelligent reflecting surface (IRS) systems are promising for 6G and beyond Terahertz (0.1-10 THz) communications, to effectively bypass the barriers of limited coverage and line-of-sight blockage. However, excessive dimensions of UM-MIMO and IRS enlarge the near-field region, while strong THz channel sparsity in far-field is d… ▽ More Integrated ultra-massive multiple-input multiple-output (UM-MIMO) and intelligent reflecting surface (IRS) systems are promising for 6G and beyond Terahertz (0.1-10 THz) communications, to effectively bypass the barriers of limited coverage and line-of-sight blockage. However, excessive dimensions of UM-MIMO and IRS enlarge the near-field region, while strong THz channel sparsity in far-field is detrimental to spatial multiplexing. Moreover, channel estimation (CE) requires recovering the large-scale channel from severely compressed observations due to limited RF-chains. To tackle these challenges, a hybrid spherical- and planar-wave channel model (HSPM) is developed for the cascaded channel of the integrated system. The spatial multiplexing gains under near-field and far-field regions are analyzed, which are found to be limited by the segmented channel with a lower rank. Furthermore, a compressive sensing-based CE framework is developed, including a sparse channel representation method, a separate-side estimation (SSE) and a dictionary-shrinkage estimation (DSE) algorithms. Numerical results verify the effectiveness of the HSPM, the capacity of which is only $5\times10^{-4}$ bits/s/Hz deviated from that obtained by the ground-truth spherical-wave-model, with 256 elements. While the SSE achieves improved accuracy for CE than benchmark algorithms, the DSE is more attractive in noisy environments, with 0.8 dB lower normalized-mean-square-error than SSE. △ Less

Submitted 25 May, 2022; originally announced May 2022.

Double-Sided Information Aided Temporal-Correlated Massive Access

Authors: Weifeng Zhu, Meixia Tao, Yunfeng Guan

Abstract: This letter considers temporal-correlated massive access, where each device, once activated, is likely to transmit continuously over several consecutive frames. Motivated by that the device activity at each frame is correlated to not only its previous frame but also its next frame, we propose a double-sided information (DSI) aided joint activity detection and channel estimation algorithm based on… ▽ More This letter considers temporal-correlated massive access, where each device, once activated, is likely to transmit continuously over several consecutive frames. Motivated by that the device activity at each frame is correlated to not only its previous frame but also its next frame, we propose a double-sided information (DSI) aided joint activity detection and channel estimation algorithm based on the approximate message passing (AMP) framework. The DSI is extracted from the estimation results in a sliding window that contains the target detection frame and its previous and next frames. The proposed algorithm demonstrates superior performance over the state-of-the-art methods. △ Less

Submitted 16 May, 2022; originally announced May 2022.

Comments: 6 pages, 5 figures

Deep Learning-Enabled Semantic Communication Systems with Task-Unaware Transmitter and Dynamic Data

Authors: Hongwei Zhang, Shuo Shao, Meixia Tao, Xiaoyan Bi, Khaled B. Letaief

Abstract: Existing deep learning-enabled semantic communication systems often rely on shared background knowledge between the transmitter and receiver that includes empirical data and their associated semantic information. In practice, the semantic information is defined by the pragmatic task of the receiver and cannot be known to the transmitter. The actual observable data at the transmitter can also have… ▽ More Existing deep learning-enabled semantic communication systems often rely on shared background knowledge between the transmitter and receiver that includes empirical data and their associated semantic information. In practice, the semantic information is defined by the pragmatic task of the receiver and cannot be known to the transmitter. The actual observable data at the transmitter can also have non-identical distribution with the empirical data in the shared background knowledge library. To address these practical issues, this paper proposes a new neural network-based semantic communication system for image transmission, where the task is unaware at the transmitter and the data environment is dynamic. The system consists of two main parts, namely the semantic coding (SC) network and the data adaptation (DA) network. The SC network learns how to extract and transmit the semantic information using a receiver-leading training process. By using the domain adaptation technique from transfer learning, the DA network learns how to convert the data observed into a similar form of the empirical data that the SC network can process without retraining. Numerical experiments show that the proposed method can be adaptive to observable datasets while keeping high performance in terms of both data recovery and task execution. △ Less

Submitted 17 October, 2022; v1 submitted 30 April, 2022; originally announced May 2022.

Spatial Likelihood Voting with Self-Knowledge Distillation for Weakly Supervised Object Detection

Authors: Ze Chen, Zhihang Fu, Jianqiang Huang, Mingyuan Tao, Rongxin Jiang, Xiang Tian, Yaowu Chen, Xian-sheng Hua

Abstract: Weakly supervised object detection (WSOD), which is an effective way to train an object detection model using only image-level annotations, has attracted considerable attention from researchers. However, most of the existing methods, which are based on multiple instance learning (MIL), tend to localize instances to the discriminative parts of salient objects instead of the entire content of all ob… ▽ More Weakly supervised object detection (WSOD), which is an effective way to train an object detection model using only image-level annotations, has attracted considerable attention from researchers. However, most of the existing methods, which are based on multiple instance learning (MIL), tend to localize instances to the discriminative parts of salient objects instead of the entire content of all objects. In this paper, we propose a WSOD framework called the Spatial Likelihood Voting with Self-knowledge Distillation Network (SLV-SD Net). In this framework, we introduce a spatial likelihood voting (SLV) module to converge region proposal localization without bounding box annotations. Specifically, in every iteration during training, all the region proposals in a given image act as voters voting for the likelihood of each category in the spatial dimensions. After dilating the alignment on the area with large likelihood values, the voting results are regularized as bounding boxes, which are then used for the final classification and localization. Based on SLV, we further propose a self-knowledge distillation (SD) module to refine the feature representations of the given image. The likelihood maps generated by the SLV module are used to supervise the feature learning of the backbone network, encouraging the network to attend to wider and more diverse areas of the image. Extensive experiments on the PASCAL VOC 2007/2012 and MS-COCO datasets demonstrate the excellent performance of SLV-SD Net. In addition, SLV-SD Net produces new state-of-the-art results on these benchmarks. △ Less

Submitted 14 April, 2022; originally announced April 2022.

Comments: arXiv admin note: text overlap with arXiv:2006.12884

Journal ref: Image and Vision Computing, Volume 116, 2021, 104314, ISSN 0262-8856

Dynamic Supervisor for Cross-dataset Object Detection

Authors: Ze Chen, Zhihang Fu, Jianqiang Huang, Mingyuan Tao, Shengyu Li, Rongxin Jiang, Xiang Tian, Yaowu Chen, Xian-sheng Hua

Abstract: The application of cross-dataset training in object detection tasks is complicated because the inconsistency in the category range across datasets transforms fully supervised learning into semi-supervised learning. To address this problem, recent studies focus on the generation of high-quality missing annotations. In this study, we first point out that it is not enough to generate high-quality ann… ▽ More The application of cross-dataset training in object detection tasks is complicated because the inconsistency in the category range across datasets transforms fully supervised learning into semi-supervised learning. To address this problem, recent studies focus on the generation of high-quality missing annotations. In this study, we first point out that it is not enough to generate high-quality annotations using a single model, which only looks once for annotations. Through detailed experimental analyses, we further conclude that hard-label training is conducive to generating high-recall annotations, while soft-label training tends to obtain high-precision annotations. Inspired by the aspects mentioned above, we propose a dynamic supervisor framework that updates the annotations multiple times through multiple-updated submodels trained using hard and soft labels. In the final generated annotations, both recall and precision improve significantly through the integration of hard-label training with soft-label training. Extensive experiments conducted on various dataset combination settings support our analyses and demonstrate the superior performance of the proposed dynamic supervisor. △ Less

Submitted 31 March, 2022; originally announced April 2022.

Journal ref: Neurocomputing, Volume 469, 2022, Pages 310-320, ISSN 0925-2312

Global Correlation and Local Information Flows in Controllable Non-Markovian Open Quantum Dynamics

Authors: Xin-Yu Chen, Na-Na Zhang, Wan-Ting He, Xiang-Yu Kong, Ming-Jie Tao, Fu-Guo Deng, Qing Ai, Gui-Lu Long

Abstract: In a fully-controllable experiment platform for studying non-Markovian open quantum dynamics, we show that the non-Markovianity could be investigated from the global and local aspects. By mixing random unitary dynamics, we demonstrate non-Markovian and Markovian open quantum dynamics. From the global point of view, by tuning the base frequency we demonstrate the transition from the Markovianity to… ▽ More In a fully-controllable experiment platform for studying non-Markovian open quantum dynamics, we show that the non-Markovianity could be investigated from the global and local aspects. By mixing random unitary dynamics, we demonstrate non-Markovian and Markovian open quantum dynamics. From the global point of view, by tuning the base frequency we demonstrate the transition from the Markovianity to the non-Markovianity as measured by the quantum mutual information (QMI). In a Markovian open quantum process, the QMI decays monotonically, while it may rise temporarily in a non-Markovian process. However, under some circumstances, it is not sufficient to globally investigate the non-Markovianity of the open quantum dynamics. As an essential supplement, we further utilize the quantum Fisher information (QFI) flow to locally characterize the non-Markovianity in different channels. We demonstrate that the QMI in combination with the QFI flow are capable of measuring the non-Markovianity for a multi-channel open quantum dynamics. △ Less

Submitted 28 February, 2022; originally announced February 2022.

Comments: 15 pages, 4 figures, to be published in npj Quantum Inf

Journal ref: npj Quantum Information (2022) 8:22

Ergodic Achievable Rate Analysis and Optimization of RIS-assisted Millimeter-Wave MIMO Communication Systems

Authors: Renwang Li, Shu Sun, Yuhang Chen, Chong Han, Meixia Tao

Abstract: Reconfigurable intelligent surfaces (RISs) have emerged as a prospective technology for next-generation wireless networks due to their potential in coverage and capacity enhancement. Previous works on achievable rate analysis of RIS-assisted communication systems have mainly focused on the rich-scattering environment where Rayleigh and Rician channel models can be applied. This work studies the er… ▽ More Reconfigurable intelligent surfaces (RISs) have emerged as a prospective technology for next-generation wireless networks due to their potential in coverage and capacity enhancement. Previous works on achievable rate analysis of RIS-assisted communication systems have mainly focused on the rich-scattering environment where Rayleigh and Rician channel models can be applied. This work studies the ergodic achievable rate of RIS-assisted multiple-input multiple-output communication systems in millimeter-wave band with limited scattering under the Saleh-Valenzuela channel model. Firstly, we derive an upper bound of the ergodic achievable rate by means of majorization theory and Jensen's inequality. The upper bound shows that the ergodic achievable rate increases logarithmically with the number of antennas at the base station (BS) and user, the number of the reflection units at the RIS, and the eigenvalues of the steering matrices associated with the BS, user and RIS. Then, we aim to maximize the ergodic achievable rate by jointly optimizing the transmit covariance matrix at the BS and the reflection coefficients at the RIS. Specifically, the transmit covariance matrix is optimized by the water-filling algorithm and the reflection coefficients are optimized using the Riemannian conjugate gradient algorithm. Simulation results validate the effectiveness of the proposed optimization algorithms. △ Less

Submitted 20 September, 2022; v1 submitted 14 February, 2022; originally announced February 2022.

Comments: 30 pages, 11 figures

Journal ref: IEEE Transactions on Wireless Communications, 2022

Joint Design of Hybrid Beamforming and Reflection Coefficients in RIS-aided mmWave MIMO Systems

Authors: Renwang Li, Bei Guo, Meixia Tao, Ya-Feng Liu, Wei Yu

Abstract: This paper considers a reconfigurable intelligent surface (RIS)-aided millimeter wave (mmWave) downlink communication system where hybrid analog-digital beamforming is employed at the base station (BS). We formulate a power minimization problem by jointly optimizing hybrid beamforming at the BS and the response matrix at the RIS, under the signal-to-interference-plus-noise ratio (SINR) constraints… ▽ More This paper considers a reconfigurable intelligent surface (RIS)-aided millimeter wave (mmWave) downlink communication system where hybrid analog-digital beamforming is employed at the base station (BS). We formulate a power minimization problem by jointly optimizing hybrid beamforming at the BS and the response matrix at the RIS, under the signal-to-interference-plus-noise ratio (SINR) constraints at all users. The problem is highly challenging to solve due to the non-convex SINR constraints as well as the unit-modulus phase shift constraints for both the RIS reflection coefficients and the analog beamformer. A two-layer penalty-based algorithm is proposed to decouple variables in SINR constraints, and manifold optimization is adopted to handle the non-convex unit-modulus constraints. {We also propose a low-complexity sequential optimization method, which optimizes the RIS reflection coefficients, the analog beamformer, and the digital beamformer sequentially without iteration.} Furthermore, the relationship between the power minimization problem and the max-min fairness (MMF) problem is discussed. Simulation results show that the proposed penalty-based algorithm outperforms the state-of-the-art semidefinite relaxation (SDR)-based algorithm. Results also demonstrate that the RIS plays an important role in the power reduction. △ Less

Submitted 14 February, 2022; originally announced February 2022.

Comments: arXiv admin note: text overlap with arXiv:2101.09980

PANet: Perspective-Aware Network with Dynamic Receptive Fields and Self-Distilling Supervision for Crowd Counting

Authors: Xiaoshuang Chen, Yiru Zhao, Yu Qin, Fei Jiang, Mingyuan Tao, Xiansheng Hua, Hongtao Lu

Abstract: Crowd counting aims to learn the crowd density distributions and estimate the number of objects (e.g. persons) in images. The perspective effect, which significantly influences the distribution of data points, plays an important role in crowd counting. In this paper, we propose a novel perspective-aware approach called PANet to address the perspective problem. Based on the observation that the siz… ▽ More Crowd counting aims to learn the crowd density distributions and estimate the number of objects (e.g. persons) in images. The perspective effect, which significantly influences the distribution of data points, plays an important role in crowd counting. In this paper, we propose a novel perspective-aware approach called PANet to address the perspective problem. Based on the observation that the size of the objects varies greatly in one image due to the perspective effect, we propose the dynamic receptive fields (DRF) framework. The framework is able to adjust the receptive field by the dilated convolution parameters according to the input image, which helps the model to extract more discriminative features for each local region. Different from most previous works which use Gaussian kernels to generate the density map as the supervised information, we propose the self-distilling supervision (SDS) training method. The ground-truth density maps are refined from the first training stage and the perspective information is distilled to the model in the second stage. The experimental results on ShanghaiTech Part_A and Part_B, UCF_QNRF, and UCF_CC_50 datasets demonstrate that our proposed PANet outperforms the state-of-the-art methods by a large margin. △ Less

Submitted 18 August, 2022; v1 submitted 31 October, 2021; originally announced November 2021.

Comments: The paper is under consideration at Computer Vision and Image Understanding

Large Learning Rate Tames Homogeneity: Convergence and Balancing Effect

Authors: Yuqing Wang, Minshuo Chen, Tuo Zhao, Molei Tao

Abstract: Recent empirical advances show that training deep models with large learning rate often improves generalization performance. However, theoretical justifications on the benefits of large learning rate are highly limited, due to challenges in analysis. In this paper, we consider using Gradient Descent (GD) with a large learning rate on a homogeneous matrix factorization problem, i.e.,… ▽ More Recent empirical advances show that training deep models with large learning rate often improves generalization performance. However, theoretical justifications on the benefits of large learning rate are highly limited, due to challenges in analysis. In this paper, we consider using Gradient Descent (GD) with a large learning rate on a homogeneous matrix factorization problem, i.e., $\min_{X, Y} \|A - XY^\top\|_{\sf F}^2$. We prove a convergence theory for constant large learning rates well beyond $2/L$, where $L$ is the largest eigenvalue of Hessian at the initialization. Moreover, we rigorously establish an implicit bias of GD induced by such a large learning rate, termed 'balancing', meaning that magnitudes of $X$ and $Y$ at the limit of GD iterations will be close even if their initialization is significantly unbalanced. Numerical experiments are provided to support our theory. △ Less

Submitted 26 February, 2022; v1 submitted 7 October, 2021; originally announced October 2021.

Comments: 45 pages

Journal ref: ICLR 2022

Parametric resonance for enhancing the rate of metastable transition

Authors: Ying Chao, Molei Tao

Abstract: This work is devoted to quantifying how periodic perturbation can change the rate of metastable transition in stochastic mechanical systems with weak noises. A closed-form explicit expression for approximating the rate change is provided, and the corresponding transition mechanism can also be approximated. Unlike the majority of existing relevant works, these results apply to kinetic Langevin equa… ▽ More This work is devoted to quantifying how periodic perturbation can change the rate of metastable transition in stochastic mechanical systems with weak noises. A closed-form explicit expression for approximating the rate change is provided, and the corresponding transition mechanism can also be approximated. Unlike the majority of existing relevant works, these results apply to kinetic Langevin equations with high-dimensional potentials and nonlinear perturbations. They are obtained based on a higher-order Hamiltonian formalism and perturbation analysis for the Freidlin-Wentzell action functional. This tool allowed us to show that parametric excitation at a resonant frequency can significantly enhance the rate of metastable transitions. Numerical experiments for both low-dimensional toy models and a molecular cluster are also provided. For the latter, we show that vibrating a material appropriately can help heal its defect, and our theory provides the appropriate vibration. △ Less

Submitted 29 September, 2021; originally announced September 2021.

The Mirror Langevin Algorithm Converges with Vanishing Bias

Authors: Ruilin Li, Molei Tao, Santosh S. Vempala, Andre Wibisono

Abstract: The technique of modifying the geometry of a problem from Euclidean to Hessian metric has proved to be quite effective in optimization, and has been the subject of study for sampling. The Mirror Langevin Diffusion (MLD) is a sampling analogue of mirror flow in continuous time, and it has nice convergence properties under log-Sobolev or Poincare inequalities relative to the Hessian metric, as shown… ▽ More The technique of modifying the geometry of a problem from Euclidean to Hessian metric has proved to be quite effective in optimization, and has been the subject of study for sampling. The Mirror Langevin Diffusion (MLD) is a sampling analogue of mirror flow in continuous time, and it has nice convergence properties under log-Sobolev or Poincare inequalities relative to the Hessian metric, as shown by Chewi et al. (2020). In discrete time, a simple discretization of MLD is the Mirror Langevin Algorithm (MLA) studied by Zhang et al. (2020), who showed a biased convergence bound with a non-vanishing bias term (does not go to zero as step size goes to zero). This raised the question of whether we need a better analysis or a better discretization to achieve a vanishing bias. Here we study the basic Mirror Langevin Algorithm and show it indeed has a vanishing bias. We apply mean-square analysis based on Li et al. (2019) and Li et al. (2021) to show the mixing time bound for MLA under the modified self-concordance condition introduced by Zhang et al. (2020). △ Less

Submitted 11 October, 2021; v1 submitted 24 September, 2021; originally announced September 2021.

Sqrt(d) Dimension Dependence of Langevin Monte Carlo

Authors: Ruilin Li, Hongyuan Zha, Molei Tao

Abstract: This article considers the popular MCMC method of unadjusted Langevin Monte Carlo (LMC) and provides a non-asymptotic analysis of its sampling error in 2-Wasserstein distance. The proof is based on a refinement of mean-square analysis in Li et al. (2019), and this refined framework automates the analysis of a large class of sampling algorithms based on discretizations of contractive SDEs. Using th… ▽ More This article considers the popular MCMC method of unadjusted Langevin Monte Carlo (LMC) and provides a non-asymptotic analysis of its sampling error in 2-Wasserstein distance. The proof is based on a refinement of mean-square analysis in Li et al. (2019), and this refined framework automates the analysis of a large class of sampling algorithms based on discretizations of contractive SDEs. Using this framework, we establish an $\tilde{O}(\sqrt{d}/ε)$ mixing time bound for LMC, without warm start, under the common log-smooth and log-strongly-convex conditions, plus a growth condition on the 3rd-order derivative of the potential of target measures. This bound improves the best previously known $\tilde{O}(d/ε)$ result and is optimal (in terms of order) in both dimension $d$ and accuracy tolerance $ε$ for target measures satisfying the aforementioned assumptions. Our theoretical analysis is further validated by numerical experiments. △ Less

Submitted 20 February, 2022; v1 submitted 8 September, 2021; originally announced September 2021.

Comments: v1 submitted on May 28, 2021 (NeurIPS 2021 deadline); v2 added an important reference and discussions; v3 is the camera ready version

Journal ref: ICLR 2022

Computational Design of Active 3D-Printed Multi-State Structures for Shape Morphing

Authors: Thomas S. Lumpe, Michael Tao, Kristina Shea, David I. W. Levin

Abstract: Active structures have the ability to change their shape, properties, and functionality as a response to changing operational conditions, which makes them more versatile than their static counterparts. However, most active structures currently lack the capability to achieve multiple, different target states with a single input actuation or require a tedious material programming step. Furthermore,… ▽ More Active structures have the ability to change their shape, properties, and functionality as a response to changing operational conditions, which makes them more versatile than their static counterparts. However, most active structures currently lack the capability to achieve multiple, different target states with a single input actuation or require a tedious material programming step. Furthermore, the systematic design and fabrication of active structures is still a challenge as many structures are designed by hand in a trial and error process and thus are limited by engineers' knowledge and experience. In this work, a computational design and fabrication framework is proposed to generate structures with multiple target states for one input actuation that don't require a separate training step. A material dithering scheme based on multi-material 3D printing is combined with locally applied copper coil heating elements and sequential heating patterns to control the thermo-mechanical properties of the structures and switch between the different deformation modes. A novel topology optimization approach based on power diagrams is used to encode the different target states in the structure while ensuring the fabricability of the structures and the compatibility with the drop-in heating elements. The versatility of the proposed framework is demonstrated for four different example structures from engineering and computer graphics. The numerical and experimental results show that the optimization framework can produce structures that show the desired motion, but experimental accuracy is limited by current fabrication methods. The generality of the proposed method makes it suitable for the development of structures for applications in many different fields from aerospace to robotics to animated fabrication in computer graphics. △ Less

Submitted 30 August, 2021; originally announced August 2021.

Unified Analysis on L1 over L2 Minimization for signal recovery

Authors: Min Tao, Xiao-Ping Zhang

Abstract: In this paper, we carry out a unified study for $L_1$ over $L_2$ sparsity promoting models, which are widely used in the regime of coherent dictionaries for recovering sparse nonnegative/arbitrary signals. First, we provide a unified theoretical analysis on the existence of the global solutions of the constrained and the unconstrained $L_{1}/L_{2}$ models. Second, we analyze the sparse property of… ▽ More In this paper, we carry out a unified study for $L_1$ over $L_2$ sparsity promoting models, which are widely used in the regime of coherent dictionaries for recovering sparse nonnegative/arbitrary signals. First, we provide a unified theoretical analysis on the existence of the global solutions of the constrained and the unconstrained $L_{1}/L_{2}$ models. Second, we analyze the sparse property of any local minimizer of these $L_{1}/L_{2}$ models which serves as a certificate to rule out the nonlocal-minimizer stationary solutions. Third, we derive an analytical solution for the proximal operator of the $L_{1} / L_{2}$ with nonnegative constraint. Equipped with this, we apply the alternating direction method of multipliers to the unconstrained model with nonnegative constraint in a particular splitting way, referred to as ADMM$_p^+$. We establish its global convergence to a d-stationary solution (sharpest stationary) without the Kurdyka-Łojasiewicz assumption. Extensive numerical simulations confirm the superior of ADMM$_p^+$ over the state-of-the-art methods in sparse recovery. In particular, ADMM$_p^+$ reduces computational time by about $95\%\sim99\%$ while achieving a much higher accuracy than the commonly used scaled gradient projection method for the wavelength misalignment problem. △ Less

Submitted 22 January, 2023; v1 submitted 2 August, 2021; originally announced August 2021.

Comments: 21 pages, 2 figures

MSC Class: 90C26; 90C90

Adversarial Visual Robustness by Causal Intervention

Authors: Kaihua Tang, Mingyuan Tao, Hanwang Zhang

Abstract: Adversarial training is the de facto most promising defense against adversarial examples. Yet, its passive nature inevitably prevents it from being immune to unknown attackers. To achieve a proactive defense, we need a more fundamental understanding of adversarial examples, beyond the popular bounded threat model. In this paper, we provide a causal viewpoint of adversarial vulnerability: the cause… ▽ More Adversarial training is the de facto most promising defense against adversarial examples. Yet, its passive nature inevitably prevents it from being immune to unknown attackers. To achieve a proactive defense, we need a more fundamental understanding of adversarial examples, beyond the popular bounded threat model. In this paper, we provide a causal viewpoint of adversarial vulnerability: the cause is the spurious correlation ubiquitously existing in learning, i.e., the confounding effect, where attackers are precisely exploiting these effects. Therefore, a fundamental solution for adversarial robustness is by causal intervention. As these visual confounders are imperceptible in general, we propose to use the instrumental variable that achieves causal intervention without the need for confounder observation. We term our robust training method as Causal intervention by instrumental Variable (CiiV). It's a causal regularization that 1) augments the image with multiple retinotopic centers and 2) encourages the model to learn causal features, rather than local confounding patterns, by favoring features linearly responding to spatial interpolations. Extensive experiments on a wide spectrum of attackers and settings applied in CIFAR-10, CIFAR-100, and mini-ImageNet demonstrate that CiiV is robust to adaptive attacks, including the recent AutoAttack. Besides, as a general causal regularization, it can be easily plugged into other methods to further boost the robustness. △ Less

Submitted 6 October, 2021; v1 submitted 17 June, 2021; originally announced June 2021.

Comments: Codes are available at the following Github project: https://github.com/KaihuaTang/CiiV-Adversarial-Robustness.pytorch

Broadened phonon-assisted wide-band radiation and subsequent low-threshold self-absorption coherent modulation in the high-entropy glass system doped with Nd3+ ions

Authors: Linde Zhang, Jingyuan Zhang, Xiang Wang, Meng Tao, Gangtao Dai, Jing Wu, Zhangwang Miao, Shifei Han, Haijuan Yu, Xuechun Lin

Abstract: For crystalline materials with long-range orders, the phonon modes involved in the phonon-assisted radiation process are generally involving one or several phonons with specific vibration frequencies 1-4. In some glassy material, the phonon modes broaden in a short range 5-7. However, the locally distinct chemical environments or mass disorder in high-entropy systems can induce an anharmonic phono… ▽ More For crystalline materials with long-range orders, the phonon modes involved in the phonon-assisted radiation process are generally involving one or several phonons with specific vibration frequencies 1-4. In some glassy material, the phonon modes broaden in a short range 5-7. However, the locally distinct chemical environments or mass disorder in high-entropy systems can induce an anharmonic phonon-phonon coupling and composition disorder, which leads to significant phonon broadening 8,9.The terminology of high-entropy comes from the high configuration entropy larger than 1.5R (R is the gas constant), which results from randomly distributed multiple nearly equal components in a crystal lattice 10,11. Inspired by the high-entropy strategy, we deployed a high-entropy glass system (HEGS) doped with neodymium ions, which exhibits a complex structure with tetrahedral voids filled by different ions, including, Li+, Zn2+, Si4+, P5+, S6+, etc. Phonon spectral broadening up to thousands of wavenumbers in the HEGS allows strong wide-band absorption in both the near-infrared and mid-infrared ranges and assists the system radiation, i.e., broadened phonon-assisted wide-band radiation (BPAWR). The subsequent low-threshold self-absorption coherence modulation (SACM) was also observed in the HEGS, modulated by changing excitation wavelengths, sample size, and doping concentrations. The time delay of the BPAWR signal is up to 1.66 ns relative to the zero-delay signal, while the time delay of the Raman process is typically in the order of fs to ps, rarely up to 10 ps 12-15.The BPAWR-SACM can be applied to realize signal amplification of the centered non-absorption band when dual-wavelength lasers pump the HEGS sample, and signal amplification can be up to 26.02 dB. The spectral characteristics of the BPAWR and the dynamics of the energy distribution of the excited species are investigated in detail. △ Less

Submitted 9 May, 2021; originally announced May 2021.

Half-Real Half-Fake Distillation for Class-Incremental Semantic Segmentation

Authors: Zilong Huang, Wentian Hao, Xinggang Wang, Mingyuan Tao, Jianqiang Huang, Wenyu Liu, Xian-Sheng Hua

Abstract: Despite their success for semantic segmentation, convolutional neural networks are ill-equipped for incremental learning, \ie, adapting the original segmentation model as new classes are available but the initial training data is not retained. Actually, they are vulnerable to catastrophic forgetting problem. We try to address this issue by "inverting" the trained segmentation network to synthesize… ▽ More Despite their success for semantic segmentation, convolutional neural networks are ill-equipped for incremental learning, \ie, adapting the original segmentation model as new classes are available but the initial training data is not retained. Actually, they are vulnerable to catastrophic forgetting problem. We try to address this issue by "inverting" the trained segmentation network to synthesize input images starting from random noise. To avoid setting detailed pixel-wise segmentation maps as the supervision manually, we propose the SegInversion to synthesize images using the image-level labels. To increase the diversity of synthetic images, the Scale-Aware Aggregation module is integrated into SegInversion for controlling the scale (the number of pixels) of synthetic objects. Along with real images of new classes, the synthesized images will be fed into the distillation-based framework to train the new segmentation model which retains the information about previously learned classes, whilst updating the current model to learn the new ones. The proposed method significantly outperforms other incremental learning methods and obtains state-of-the-art performance on the PASCAL VOC 2012 and ADE20K datasets. The code and models will be made publicly available. △ Less

Submitted 1 April, 2021; originally announced April 2021.

Variational Symplectic Accelerated Optimization on Lie Groups

Authors: Taeyoung Lee, Molei Tao, Melvin Leok

Abstract: There has been significant interest in generalizations of the Nesterov accelerated gradient descent algorithm due to its improved performance guarantee compared to the standard gradient descent algorithm, and its applicability to large scale optimization problems arising in deep learning. A particularly fruitful approach is based on numerical discretizations of differential equations that describe… ▽ More There has been significant interest in generalizations of the Nesterov accelerated gradient descent algorithm due to its improved performance guarantee compared to the standard gradient descent algorithm, and its applicability to large scale optimization problems arising in deep learning. A particularly fruitful approach is based on numerical discretizations of differential equations that describe the continuous time limit of the Nesterov algorithm, and a generalization involving time-dependent Bregman Lagrangian and Hamiltonian dynamics that converges at an arbitrarily fast rate to the minimum. We develop a Lie group variational discretization based on an extended path space formulation of the Bregman Lagrangian on Lie groups, and analyze its computational properties with two examples in attitude determination and vision-based localization. △ Less

Submitted 25 March, 2021; originally announced March 2021.

GRIT: a package for structure-preserving simulations of gravitationally interacting rigid-bodies

Authors: Renyi Chen, Gongjie Li, Molei Tao

Abstract: Spin-orbit coupling of planetary systems plays an important role in the dynamics and habitability of planets. However, symplectic integrators that can accurately simulate not only how orbit affects spin but also how spin affects orbit have not been constructed for general systems. Thus, we develop symplectic Lie-group integrators to simulate systems consisting gravitationally interacting rigid bod… ▽ More Spin-orbit coupling of planetary systems plays an important role in the dynamics and habitability of planets. However, symplectic integrators that can accurately simulate not only how orbit affects spin but also how spin affects orbit have not been constructed for general systems. Thus, we develop symplectic Lie-group integrators to simulate systems consisting gravitationally interacting rigid bodies. A user friendly package (GRIT \url{https://github.com/GRIT-RBSim/GRIT}) is provided and external forcings such as tidal interactions are also included. As a demonstration, this package is applied to Trappist-I. It shows that the differences in transit timing variations due to spin-orbit coupling could reach a few min in ten year measurements, and strong planetary perturbations can push Trappist-I f, g and h out of the synchronized states. △ Less

Submitted 6 July, 2021; v1 submitted 23 March, 2021; originally announced March 2021.

Comments: 18 pages, 12 figures. Accepted for publication in ApJ

Data-driven Prediction of General Hamiltonian Dynamics via Learning Exactly-Symplectic Maps

Authors: Renyi Chen, Molei Tao

Abstract: We consider the learning and prediction of nonlinear time series generated by a latent symplectic map. A special case is (not necessarily separable) Hamiltonian systems, whose solution flows give such symplectic maps. For this special case, both generic approaches based on learning the vector field of the latent ODE and specialized approaches based on learning the Hamiltonian that generates the ve… ▽ More We consider the learning and prediction of nonlinear time series generated by a latent symplectic map. A special case is (not necessarily separable) Hamiltonian systems, whose solution flows give such symplectic maps. For this special case, both generic approaches based on learning the vector field of the latent ODE and specialized approaches based on learning the Hamiltonian that generates the vector field exist. Our method, however, is different as it does not rely on the vector field nor assume its existence; instead, it directly learns the symplectic evolution map in discrete time. Moreover, we do so by representing the symplectic map via a generating function, which we approximate by a neural network (hence the name GFNN). This way, our approximation of the evolution map is always \emph{exactly} symplectic. This additional geometric structure allows the local prediction error at each step to accumulate in a controlled fashion, and we will prove, under reasonable assumptions, that the global prediction error grows at most \emph{linearly} with long prediction time, which significantly improves an otherwise exponential growth. In addition, as a map-based and thus purely data-driven method, GFNN avoids two additional sources of inaccuracies common in vector-field based approaches, namely the error in approximating the vector field by finite difference of the data, and the error in numerical integration of the vector field for making predictions. Numerical experiments further demonstrate our claims. △ Less

Submitted 12 June, 2021; v1 submitted 9 March, 2021; originally announced March 2021.

Comments: 17 pages, 7 figures. ICML 2021

Accurate and Efficient Simulations of Hamiltonian Mechanical Systems with Discontinuous Potentials

Authors: Molei Tao, Shi Jin

Abstract: This article considers Hamiltonian mechanical systems with potential functions admitting jump discontinuities. The focus is on accurate and efficient numerical approximations of their solutions, which will be defined via the laws of reflection and refraction. Despite of the success of symplectic integrators for smooth mechanical systems, their construction for the discontinuous ones is nontrivial,… ▽ More This article considers Hamiltonian mechanical systems with potential functions admitting jump discontinuities. The focus is on accurate and efficient numerical approximations of their solutions, which will be defined via the laws of reflection and refraction. Despite of the success of symplectic integrators for smooth mechanical systems, their construction for the discontinuous ones is nontrivial, and numerical convergence order can be impaired too. Several rather-usable numerical methods are proposed, including: a first-order symplectic integrator for general problems, a third-order symplectic integrator for problems with only one linear interface, arbitrarily high-order reversible integrators for general problems (no longer symplectic), and an adaptive time-stepping version of the previous high-order method. Interestingly, whether symplecticity leads to favorable long time performance is no longer clear due to discontinuity, as traditional Hamiltonian backward error analysis does not apply any more. Therefore, at this stage, our recommended default method is the last one. Various numerical evidence, on the order of convergence, long time performance, momentum map conservation, and consistency with the computationally-expensive penalty method, are supplied. A complex problem, namely the Sauteed Mushroom, is also proposed and numerically investigated, for which multiple bifurcations between trapped and ergodic dynamics are observed. △ Less

Submitted 26 January, 2021; originally announced January 2021.

Comments: Questions and comments are welcome

Two-Way Passive Beamforming Design for RIS-Aided FDD Communication Systems

Authors: Bei Guo, Chenhao Sun, Meixia Tao

Abstract: Reconfigurable intelligent surfaces (RISs) are able to provide passive beamforming gain via low-cost reflecting elements and hence improve wireless link quality. This work considers two-way passive beamforming design in RIS-aided frequency division duplexing (FDD) systems where the RIS reflection coefficients are the same for downlink and uplink and should be optimized for both directions simultan… ▽ More Reconfigurable intelligent surfaces (RISs) are able to provide passive beamforming gain via low-cost reflecting elements and hence improve wireless link quality. This work considers two-way passive beamforming design in RIS-aided frequency division duplexing (FDD) systems where the RIS reflection coefficients are the same for downlink and uplink and should be optimized for both directions simultaneously. We formulate a joint optimization of the transmit/receive beamformers at the base station (BS) and the RIS reflection coefficients. The objective is to maximize the weighted sum of the downlink and uplink rates, where the weighting parameter is adjustable to obtain different achievable downlink-uplink rate pairs. We develop an efficient manifold optimization algorithm to obtain a stationary solution. For comparison, we also introduce two heuristic designs based on one-way optimization, namely, time-sharing and phase-averaging. Simulation results show that the proposed manifold-based two-way optimization design significantly enlarges the achievable downlink-uplink rate region compared with the two heuristic designs. It is also shown that phase-averaging is superior to time-sharing when the number of RIS elements is large. △ Less

Submitted 25 January, 2021; originally announced January 2021.

Comments: To appear in IEEE WCNC 2021

Joint Design of Hybrid Beamforming and Phase Shifts in RIS-Aided mmWave Communication Systems

Authors: Bei Guo, Renwang Li, Meixia Tao

Abstract: This paper considers a reconfigurable intelligent surface (RIS)-aided millimeter wave (mmWave) downlink communication system where hybrid analog-digital beamforming is employed at the base station (BS). We formulate a power minimization problem by jointly optimizing hybrid beamforming at the BS and the response matrix at the RIS, under signal-to-interference-plus-noise ratio (SINR) constraints. Th… ▽ More This paper considers a reconfigurable intelligent surface (RIS)-aided millimeter wave (mmWave) downlink communication system where hybrid analog-digital beamforming is employed at the base station (BS). We formulate a power minimization problem by jointly optimizing hybrid beamforming at the BS and the response matrix at the RIS, under signal-to-interference-plus-noise ratio (SINR) constraints. The problem is highly challenging due to the non-convex SINR constraints as well as the non-convex unit-modulus constraints for both the phase shifts at the RIS and the analog beamforming at the BS. A penalty-based algorithm in conjunction with the manifold optimization technique is proposed to handle the problem, followed by an individual optimization method with much lower complexity. Simulation results show that the proposed algorithm outperforms the state-of-art algorithm. Results also show that the joint optimization of RIS response matrix and BS hybrid beamforming is much superior to individual optimization. △ Less

Submitted 25 January, 2021; originally announced January 2021.

Comments: To appear in IEEE WCNC 2021

Sum-Rate-Distortion Function for Indirect Multiterminal Source Coding in Federated Learning

Authors: Naifu Zhang, Meixia Tao, Jia Wang

Abstract: One of the main focus in federated learning (FL) is the communication efficiency since a large number of participating edge devices send their updates to the edge server at each round of the model training. Existing works reconstruct each model update from edge devices and implicitly assume that the local model updates are independent over edge devices. In FL, however, the model update is an indir… ▽ More One of the main focus in federated learning (FL) is the communication efficiency since a large number of participating edge devices send their updates to the edge server at each round of the model training. Existing works reconstruct each model update from edge devices and implicitly assume that the local model updates are independent over edge devices. In FL, however, the model update is an indirect multi-terminal source coding problem, also called as the CEO problem where each edge device cannot observe directly the gradient that is to be reconstructed at the decoder, but is rather provided only with a noisy version. The existing works do not leverage the redundancy in the information transmitted by different edges. This paper studies the rate region for the indirect multiterminal source coding problem in FL. The goal is to obtain the minimum achievable rate at a particular upper bound of gradient variance. We obtain the rate region for the quadratic vector Gaussian CEO problem under unbiased estimator and derive an explicit formula of the sum-rate-distortion function in the special case where gradient are identical over edge device and dimension. Finally, we analyse communication efficiency of convex Minibatched SGD and non-convex Minibatched SGD based on the sum-rate-distortion function, respectively. △ Less

Submitted 13 May, 2021; v1 submitted 21 January, 2021; originally announced January 2021.

Comments: 6 pages, 2 figures

Minimizing L1 over L2 norms on the gradient

Authors: Chao Wang, Min Tao, Chen-Nee Chuah, James Nagy, Yifei Lou

Abstract: In this paper, we study the L1/L2 minimization on the gradient for imaging applications. Several recent works have demonstrated that L1/L2 is better than the L1 norm when approximating the L0 norm to promote sparsity. Consequently, we postulate that applying L1/L2 on the gradient is better than the classic total variation (the L1 norm on the gradient) to enforce the sparsity of the image gradient.… ▽ More In this paper, we study the L1/L2 minimization on the gradient for imaging applications. Several recent works have demonstrated that L1/L2 is better than the L1 norm when approximating the L0 norm to promote sparsity. Consequently, we postulate that applying L1/L2 on the gradient is better than the classic total variation (the L1 norm on the gradient) to enforce the sparsity of the image gradient. To verify our hypothesis, we consider a constrained formulation to reveal empirical evidence on the superiority of L1/L2 over L1 when recovering piecewise constant signals from low-frequency measurements. Numerically, we design a specific splitting scheme, under which we can prove subsequential and global convergence for the alternating direction method of multipliers (ADMM) under certain conditions. Experimentally, we demonstrate visible improvements of L1/L2 over L1 and other nonconvex regularizations for image recovery from low-frequency measurements and two medical applications of MRI and CT reconstruction. All the numerical results show the efficiency of our proposed approach. △ Less

Submitted 6 July, 2021; v1 submitted 4 January, 2021; originally announced January 2021.

Comments: 26 pages

Deep-Learned Approximate Message Passing for Asynchronous Massive Connectivity

Authors: Weifeng Zhu, Meixia Tao, Xiaojun Yuan, Yunfeng Guan

Abstract: This paper considers the massive connectivity problem in an asynchronous grant-free random access system, where a huge number of devices sporadically transmit data to a base station (BS) with imperfect synchronization. The goal is to design algorithms for joint user activity detection, delay detection, and channel estimation. By exploiting the sparsity on both user activity and delays, we formulat… ▽ More This paper considers the massive connectivity problem in an asynchronous grant-free random access system, where a huge number of devices sporadically transmit data to a base station (BS) with imperfect synchronization. The goal is to design algorithms for joint user activity detection, delay detection, and channel estimation. By exploiting the sparsity on both user activity and delays, we formulate a hierarchical sparse signal recovery problem in both the single-antenna and the multiple-antenna scenarios. While traditional compressed sensing algorithms can be applied to these problems, they suffer high computational complexity and often require the perfect statistical information of channel and devices. This paper solves these problems by designing the Learned Approximate Message Passing (LAMP) network, which belongs to model-driven deep learning approaches and ensures efficient performance without tremendous training data. Particularly, in the multiple-antenna scenario, we design three different LAMP structures, namely, distributed, centralized and hybrid ones, to balance the performance and complexity. Simulation results demonstrate that the proposed LAMP networks can significantly outperform the conventional AMP method thanks to their ability of parameter learning. It is also shown that LAMP has robust performance to the maximal delay spread of the asynchronous users. △ Less

Submitted 3 January, 2021; originally announced January 2021.

Comments: 31 pages, 11 figures

Design and verification of the HXI collimator onboard the ASO-S mission

Authors: Chen Dengyi, Hu Yiming, Ma Tao, Su Yang, Yang Jianfeng, Wang Jianping, Xu Guangzhou, Jiang Xiankai, Guo Jianhua, Zhang Yongqiang, Zhang Yan, Chen Wei, Chang Jin, Zhang Zhe

Abstract: A space-borne hard X-ray collimator, comprising 91 pairs of grids, has been developed for the Hard X-ray Imager (HXI). The HXI is one of the three scientific instruments onboard the first Chinese solar mission: the Advanced Space-based Solar Observatory (ASO-S). The HXI collimator (HXI-C) is a spatial modulation X-ray telescope designed to observe hard X-rays emitted by energetic electrons in sola… ▽ More A space-borne hard X-ray collimator, comprising 91 pairs of grids, has been developed for the Hard X-ray Imager (HXI). The HXI is one of the three scientific instruments onboard the first Chinese solar mission: the Advanced Space-based Solar Observatory (ASO-S). The HXI collimator (HXI-C) is a spatial modulation X-ray telescope designed to observe hard X-rays emitted by energetic electrons in solar flares. This paper presents the detailed design of the HXI-C for the qualification model that will be inherited by the flight model. Series tests on the HXI-C qualification model are reported to verify the ability of the HXI-C to survive the launch and to operate normally in on-orbit environments. Furthermore, results of the X-ray beam test for the HXI-C are presented to indirectly identify the working performance of the HXI-C. △ Less

Submitted 18 January, 2021; v1 submitted 2 December, 2020; originally announced December 2020.

Comments: 16 pages, 21 figures

On the Mutual Interference between Spaceborne SARs: Modeling, Characterization, and Mitigation

Authors: Huizhang Yang, Mingliang Tao, Shengyao Chen, Feng Xi, Zhong Liu

Abstract: Since by the International Telecommunications Union (ITU) regulatory the radio spectrum available to spaceborne synthetic aperture radar (SAR) is restricted to certain limited frequency intervals, there are many different spaceborne SAR systems sharing common frequency bands. Due to this reason, it is reported that two spaceborne SARs at orbit cross positions can potentially cause severe mutual in… ▽ More Since by the International Telecommunications Union (ITU) regulatory the radio spectrum available to spaceborne synthetic aperture radar (SAR) is restricted to certain limited frequency intervals, there are many different spaceborne SAR systems sharing common frequency bands. Due to this reason, it is reported that two spaceborne SARs at orbit cross positions can potentially cause severe mutual interference. Specifically, the transmitting signal of a SAR, typically linear frequency modulated (LFM), can be directly received by the side or back lobes of another SAR's antenna, causing radiometric artefacts in the focused image. This paper tries to model and characterize the artefacts, and study efficient methods for mitigating them. To this end, we formulate an analytical model for describing the artefact, which reveals that the mutual interference can introduce a two-dimensional LFM radiometric artefact in image domain with a limited spatial extent. We show that the artefact is low-rank based on range-azimuth decoupling analysis and two-dimensional high-order Taylor expansion. Based on the low rank model, we show that two methods, i.e., principle component analysis and its robust variant, can be adopted to efficiently mitigate the artefact via processing in image domain. The former method has the advantage of fast processing speed, for example, a sub-swath of Sentinel-1 interferometric wide swath image can be processed within 70 seconds via block-wise operation, whereas the latter provides improved accuracy for sparse point-like scatterers. Experiment results demonstrate that the radiometric artefacts caused by mutual interference in Sentinel-1 level-1 images can be efficiently mitigated via the proposed methods. △ Less

Submitted 6 November, 2020; v1 submitted 14 October, 2020; originally announced October 2020.

Comments: Accepted by IEEE TGRS

Deep Learning for Wireless Coded Caching with Unknown and Time-Variant Content Popularity

Authors: Zhe Zhang, Meixia Tao

Abstract: Coded caching is effective in leveraging the accumulated storage size in wireless networks by distributing different coded segments of each file in multiple cache nodes. This paper aims to find a wireless coded caching policy to minimize the total discounted network cost, which involves both transmission delay and cache replacement cost, using tools from deep learning. The problem is known to be c… ▽ More Coded caching is effective in leveraging the accumulated storage size in wireless networks by distributing different coded segments of each file in multiple cache nodes. This paper aims to find a wireless coded caching policy to minimize the total discounted network cost, which involves both transmission delay and cache replacement cost, using tools from deep learning. The problem is known to be challenging due to the unknown, time-variant content popularity as well as the continuous, high-dimensional action space. We first propose a clustering based long short-term memory (C-LTSM) approach to predict the number of content requests using historical request information. This approach exploits the correlation of the historical request information between different files through clustering. Based on the predicted results, we then propose a supervised deep deterministic policy gradient (SDDPG) approach. This approach, on one hand, can learn the caching policy in continuous action space by using the actor-critic architecture. On the other hand, it accelerates the learning process by pre-training the actor network based on the solution of an approximate problem that minimizes the per-slot cost. Real-world trace-based numerical results show that the proposed prediction and caching policy using deep learning outperform the considered existing methods. △ Less

Submitted 17 October, 2020; v1 submitted 21 August, 2020; originally announced August 2020.

DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesis

Authors: Ming Tao, Hao Tang, Fei Wu, Xiao-Yuan Jing, Bing-Kun Bao, Changsheng Xu

Abstract: Synthesizing high-quality realistic images from text descriptions is a challenging task. Existing text-to-image Generative Adversarial Networks generally employ a stacked architecture as the backbone yet still remain three flaws. First, the stacked architecture introduces the entanglements between generators of different image scales. Second, existing studies prefer to apply and fix extra networks… ▽ More Synthesizing high-quality realistic images from text descriptions is a challenging task. Existing text-to-image Generative Adversarial Networks generally employ a stacked architecture as the backbone yet still remain three flaws. First, the stacked architecture introduces the entanglements between generators of different image scales. Second, existing studies prefer to apply and fix extra networks in adversarial learning for text-image semantic consistency, which limits the supervision capability of these networks. Third, the cross-modal attention-based text-image fusion that widely adopted by previous works is limited on several special image scales because of the computational cost. To these ends, we propose a simpler but more effective Deep Fusion Generative Adversarial Networks (DF-GAN). To be specific, we propose: (i) a novel one-stage text-to-image backbone that directly synthesizes high-resolution images without entanglements between different generators, (ii) a novel Target-Aware Discriminator composed of Matching-Aware Gradient Penalty and One-Way Output, which enhances the text-image semantic consistency without introducing extra networks, (iii) a novel deep text-image fusion block, which deepens the fusion process to make a full fusion between text and visual features. Compared with current state-of-the-art methods, our proposed DF-GAN is simpler but more efficient to synthesize realistic and text-matching images and achieves better performance on widely used datasets. △ Less

Submitted 14 October, 2022; v1 submitted 13 August, 2020; originally announced August 2020.

SiENet: Siamese Expansion Network for Image Extrapolation

Authors: Xiaofeng Zhang, Feng Chen, Cailing Wang, Songsong Wu, Ming Tao, Guoping Jiang

Abstract: Different from image inpainting, image outpainting has relative less context in the image center to capture and more content at the image border to predict. Therefore, classical encoder-decoder pipeline of existing methods may not predict the outstretched unknown content perfectly. In this paper, a novel two-stage siamese adversarial model for image extrapolation, named Siamese Expansion Network (… ▽ More Different from image inpainting, image outpainting has relative less context in the image center to capture and more content at the image border to predict. Therefore, classical encoder-decoder pipeline of existing methods may not predict the outstretched unknown content perfectly. In this paper, a novel two-stage siamese adversarial model for image extrapolation, named Siamese Expansion Network (SiENet) is proposed. In two stages, a novel border sensitive convolution named adaptive filling convolution is designed for allowing encoder to predict the unknown content, alleviating the burden of decoder. Besides, to introduce prior knowledge to network and reinforce the inferring ability of encoder, siamese adversarial mechanism is designed to enable our network to model the distribution of covered long range feature for that of uncovered image feature. The results on four datasets has demonstrated that our method outperforms existing state-of-the-arts and could produce realistic results. △ Less

Submitted 7 July, 2020; originally announced July 2020.

Exact and efficient quantum simulation of open quantum dynamics for various of Hamiltonians and spectral densities

Authors: Na-Na Zhang, Ming-Jie Tao, Wan-Ting He, Xin-Yu Chen, Xiang-Yu Kong, Fu-Guo Deng, Neill Lambert, Qing Ai, Yuan-Chung Cheng

Abstract: Recently, we have theoretically proposed and experimentally demonstrated an exact and efficient quantum simulation of photosynthetic light harvesting in nuclear magnetic resonance (NMR), cf. B. X. Wang, \textit{et al.} npj Quantum Inf.~\textbf{4}, 52 (2018). In this paper, we apply this approach to simulate the open quantum dynamics in various photosynthetic systems with different Hamiltonians. By… ▽ More Recently, we have theoretically proposed and experimentally demonstrated an exact and efficient quantum simulation of photosynthetic light harvesting in nuclear magnetic resonance (NMR), cf. B. X. Wang, \textit{et al.} npj Quantum Inf.~\textbf{4}, 52 (2018). In this paper, we apply this approach to simulate the open quantum dynamics in various photosynthetic systems with different Hamiltonians. By numerical simulations, we show that for Drude-Lorentz spectral density the dimerized geometries with strong couplings within the donor and acceptor clusters respectively exhibit significantly-improved efficiency. Based on the optimal geometry, we also demonstrate that the overall energy transfer can be further optimized when the energy gap between the donor and acceptor clusters matches the peak of the spectral density. Moreover, by exploring the quantum dynamics for different types of spectral densities, e.g. Ohmic, sub-Ohmic, and super-Ohmic spectral densities, we show that our approach can be generalized to effectively simulate open quantum dynamics for various Hamiltonians and spectral densities. Because $\log_{2}N$ qubits are required for quantum simulation of an $N$-dimensional quantum system, this quantum simulation approach can greatly reduce the computational complexity compared with popular numerically-exact methods. △ Less

Submitted 5 July, 2020; originally announced July 2020.

Comments: 12 pages, 4 figures

Journal ref: Front. Phys. 16(5), 51501 (2021)

Hessian-Free High-Resolution Nesterov Acceleration for Sampling

Authors: Ruilin Li, Hongyuan Zha, Molei Tao

Abstract: Nesterov's Accelerated Gradient (NAG) for optimization has better performance than its continuous time limit (noiseless kinetic Langevin) when a finite step-size is employed \citep{shi2021understanding}. This work explores the sampling counterpart of this phenonemon and proposes a diffusion process, whose discretizations can yield accelerated gradient-based MCMC methods. More precisely, we reformu… ▽ More Nesterov's Accelerated Gradient (NAG) for optimization has better performance than its continuous time limit (noiseless kinetic Langevin) when a finite step-size is employed \citep{shi2021understanding}. This work explores the sampling counterpart of this phenonemon and proposes a diffusion process, whose discretizations can yield accelerated gradient-based MCMC methods. More precisely, we reformulate the optimizer of NAG for strongly convex functions (NAG-SC) as a Hessian-Free High-Resolution ODE, change its high-resolution coefficient to a hyperparameter, inject appropriate noise, and discretize the resulting diffusion process. The acceleration effect of the new hyperparameter is quantified and it is not an artificial one created by time-rescaling. Instead, acceleration beyond underdamped Langevin in $W_2$ distance is quantitatively established for log-strongly-concave-and-smooth targets, at both the continuous dynamics level and the discrete algorithm level. Empirical experiments in both log-strongly-concave and multi-modal cases also numerically demonstrate this acceleration. △ Less

Submitted 17 June, 2022; v1 submitted 16 June, 2020; originally announced June 2020.

Limited-angle CT reconstruction via the L1/L2 minimization

Authors: Chao Wang, Min Tao, James Nagy, Yifei Lou

Abstract: In this paper, we consider minimizing the L1/L2 term on the gradient for a limited-angle scanning problem in computed tomography (CT) reconstruction. We design a specific splitting framework for an unconstrained optimization model so that the alternating direction method of multipliers (ADMM) has guaranteed convergence under certain conditions. In addition, we incorporate a box constraint that is… ▽ More In this paper, we consider minimizing the L1/L2 term on the gradient for a limited-angle scanning problem in computed tomography (CT) reconstruction. We design a specific splitting framework for an unconstrained optimization model so that the alternating direction method of multipliers (ADMM) has guaranteed convergence under certain conditions. In addition, we incorporate a box constraint that is reasonable for imaging applications, and the convergence for the additional box constraint can also be established. Numerical results on both synthetic and experimental datasets demonstrate the effectiveness and efficiency of our proposed approaches, showing significant improvements over the state-of-the-art methods in the limited-angle CT reconstruction. △ Less

Submitted 17 March, 2021; v1 submitted 31 May, 2020; originally announced June 2020.

Comments: 29 pages

Coded Computing and Cooperative Transmission for Wireless Distributed Matrix Multiplication

Authors: Kuikui Li, Meixia Tao, Jingjing Zhang, Osvaldo Simeone

Abstract: Consider a multi-cell mobile edge computing network, in which each user wishes to compute the product of a user-generated data matrix with a network-stored matrix. This is done through task offloading by means of input uploading, distributed computing at edge nodes (ENs), and output downloading. Task offloading may suffer long delay since servers at some ENs may be straggling due to random computa… ▽ More Consider a multi-cell mobile edge computing network, in which each user wishes to compute the product of a user-generated data matrix with a network-stored matrix. This is done through task offloading by means of input uploading, distributed computing at edge nodes (ENs), and output downloading. Task offloading may suffer long delay since servers at some ENs may be straggling due to random computation time, and wireless channels may experience severe fading and interference. This paper aims to investigate the interplay among upload, computation, and download latencies during the offloading process in the high signal-to-noise ratio regime from an information-theoretic perspective. A policy based on cascaded coded computing and on coordinated and cooperative interference management in uplink and downlink is proposed and proved to be approximately optimal for a sufficiently large upload time. By investing more time in uplink transmission, the policy creates data redundancy at the ENs, which can reduce the computation time, by enabling the use of coded computing, as well as the download time via transmitter cooperation. Moreover, the policy allows computation time to be traded for download time. Numerical examples demonstrate that the proposed policy can improve over existing schemes by significantly reducing the end-to-end execution time. △ Less

Submitted 18 January, 2021; v1 submitted 29 April, 2020; originally announced April 2020.

Comments: To appear in IEEE Transactions on Communications

Linear Precoding for Fading Cognitive Multiple Access Wiretap Channel with Finite-Alphabet Inputs

Authors: Juening Jin, Chengshan Xiao, Meixia Tao, Wen Chen

Abstract: We investigate the fading cognitive multiple access wiretap channel (CMAC-WT), in which two secondary-user transmitters (STs) send secure messages to a secondary-user receiver (SR) in the presence of an eavesdropper (ED) and subject to interference threshold constraints at multiple primary-user receivers (PRs). We design linear precoders to maximize the average secrecy sum rate for multiple-input… ▽ More We investigate the fading cognitive multiple access wiretap channel (CMAC-WT), in which two secondary-user transmitters (STs) send secure messages to a secondary-user receiver (SR) in the presence of an eavesdropper (ED) and subject to interference threshold constraints at multiple primary-user receivers (PRs). We design linear precoders to maximize the average secrecy sum rate for multiple-input multiple-output (MIMO) fading CMAC-WT under finite-alphabet inputs and statistical channel state information (CSI) at STs. For this non-deterministic polynomial time (NP)-hard problem, we utilize an accurate approximation of the average secrecy sum rate to reduce the computational complexity, and then present a two-layer algorithm by embedding the convex-concave procedure into an outer approximation framework. The idea behind this algorithm is to reformulate the approximated average secrecy sum rate as a difference of convex functions, and then generate a sequence of simpler relaxed sets to approach the non-convex feasible set. Subsequently, we maximize the approximated average secrecy sum rate over the sequence of relaxed sets by using the convex-concave procedure. Numerical results indicate that our proposed precoding algorithm is superior to the conventional Gaussian precoding method in the medium and high signal-to-noise ratio (SNR) regimes. △ Less

Submitted 29 March, 2020; originally announced March 2020.

Comments: TVT

Probabilistic Caching for Small-Cell Networks with Terrestrial and Aerial Users

Authors: Fei Song, Jun Li, Ming Ding, Long Shi, Feng Shu, Meixia Tao, Wen Chen, H. Vincent Poor

Abstract: The support for aerial users has become the focus of recent 3GPP standardizations of 5G, due to their high maneuverability and flexibility for on-demand deployment. In this paper, probabilistic caching is studied for ultra-dense small-cell networks with terrestrial and aerial users, where a dynamic on-off architecture is adopted under a sophisticated path loss model incorporating both line-of-sigh… ▽ More The support for aerial users has become the focus of recent 3GPP standardizations of 5G, due to their high maneuverability and flexibility for on-demand deployment. In this paper, probabilistic caching is studied for ultra-dense small-cell networks with terrestrial and aerial users, where a dynamic on-off architecture is adopted under a sophisticated path loss model incorporating both line-of-sight and non-line-of-sight transmissions. Generally, this paper focuses on the successful download probability (SDP) of user equipments (UEs) from small-cell base stations (SBSs) that cache the requested files under various caching strategies. To be more specific, the SDP is first analyzed using stochastic geometry theory, by considering the distribution of such two-tier UEs and SBSs as Homogeneous Poisson Point Processes. Second, an optimized caching strategy (OCS) is proposed to maximize the average SDP. Third, the performance limits of the average SDP are developed for the popular caching strategy (PCS) and the uniform caching strategy (UCS). Finally, the impacts of the key parameters, such as the SBS density, the cache size, the exponent of Zipf distribution and the height of aerial user, are investigated on the average SDP. The analytical results indicate that the UCS outperforms the PCS if the SBSs are sufficiently dense, while the PCS is better than the UCS if the exponent of Zipf distribution is large enough. Furthermore, the proposed OCS is superior to both the UCS and PCS. △ Less

Submitted 29 March, 2020; originally announced March 2020.

Comments: TVT

Position Based Compressed Channel Estimation and Pilot Design for High Mobility OFDM Systems

Authors: Xiang Ren, Wen Chen, Meixia Tao

Abstract: With the development of high speed trains (HST) in many countries, providing broadband wireless services in HSTs is becoming crucial. Orthogonal frequency-division multiplexing (OFDM) has been widely adopted for broadband wireless communications due to its high spectral efficiency. However, OFDM is sensitive to the time selectivity caused by high-mobility channels, which costs large spectrum or ti… ▽ More With the development of high speed trains (HST) in many countries, providing broadband wireless services in HSTs is becoming crucial. Orthogonal frequency-division multiplexing (OFDM) has been widely adopted for broadband wireless communications due to its high spectral efficiency. However, OFDM is sensitive to the time selectivity caused by high-mobility channels, which costs large spectrum or time resources to obtain the accurate channel state information (CSI). Therefore, the channel estimation in high-mobility OFDM systems has been a long-standing challenge. In this paper, we first propose a new position-based high-mobility channel model,in which the HST's position information and Doppler shift are utilized to determine the positions of the dominant channel coefficients. %In this way, we can reduce the estimation complexity and to design the transmitted pilot.Then, we propose a joint pilot placement and pilot symbol design algorithm for compressed channel estimation. It aims to reduce the coherence between the pilot signal and the proposed channel model, and hence can improve the channel estimation accuracy. Simulation results demonstrate that the proposed method achieves better performances than existing channel estimation methods over high-mobility channels. Furthermore, we give an example of the designed pilot codebook to show the practical applicability of the proposed scheme. △ Less

Submitted 1 March, 2020; originally announced March 2020.

Gradient Statistics Aware Power Control for Over-the-Air Federated Learning

Authors: Naifu Zhang, Meixia Tao

Abstract: Federated learning (FL) is a promising technique that enables many edge devices to train a machine learning model collaboratively in wireless networks. By exploiting the superposition nature of wireless waveforms, over-the-air computation (AirComp) can accelerate model aggregation and hence facilitate communication-efficient FL. Due to channel fading, power control is crucial in AirComp. Prior wor… ▽ More Federated learning (FL) is a promising technique that enables many edge devices to train a machine learning model collaboratively in wireless networks. By exploiting the superposition nature of wireless waveforms, over-the-air computation (AirComp) can accelerate model aggregation and hence facilitate communication-efficient FL. Due to channel fading, power control is crucial in AirComp. Prior works assume that the signals to be aggregated from each device, i.e., local gradients have identical statistics. In FL, however, gradient statistics vary over both training iterations and feature dimensions, and are unknown in advance. This paper studies the power control problem for over-the-air FL by taking gradient statistics into account. The goal is to minimize the aggregation error by optimizing the transmit power at each device subject to peak power constraints. We obtain the optimal policy in closed form when gradient statistics are given. Notably, we show that the optimal transmit power is continuous and monotonically decreases with the squared multivariate coefficient of variation (SMCV) of gradient vectors. We then propose a method to estimate gradient statistics with negligible communication cost. Experimental results demonstrate that the proposed gradient-statistics-aware power control achieves higher test accuracy than the existing schemes for a wide range of scenarios. △ Less

Submitted 25 November, 2020; v1 submitted 4 March, 2020; originally announced March 2020.

Comments: 30 pages, 8 figures

Compressed Channel Estimation with Position-Based ICI Elimination for High-Mobility SIMO-OFDM Systems

Authors: Xiang Ren, Meixia Tao, Wen Chen

Abstract: Orthogonal frequency-division multiplexing (OFDM) is widely adopted for providing reliable and high data rate communication in high-speed train systems. However, with the increasing train mobility, the resulting large Doppler shift introduces intercarrier interference (ICI) in OFDM systems and greatly degrades the channel estimation accuracy. Therefore, it is necessary and important to investigate… ▽ More Orthogonal frequency-division multiplexing (OFDM) is widely adopted for providing reliable and high data rate communication in high-speed train systems. However, with the increasing train mobility, the resulting large Doppler shift introduces intercarrier interference (ICI) in OFDM systems and greatly degrades the channel estimation accuracy. Therefore, it is necessary and important to investigate reliable channel estimation and ICI mitigation methods in high-mobility environments. In this paper, we consider a typical HST communication system and show that the ICI caused by the large Doppler shift can be mitigated by exploiting the train position information as well as the sparsity of the conventional basis expansion model (BEM) based channel model. Then, we show that for the complex-exponential BEM (CE-BEM) based channel model, the ICI can be completely eliminated to get the ICI-free pilots at each receive antenna. After that, we propose a new pilot pattern design algorithm to reduce the system coherence and hence can improve the compressed sensing (CS) based channel estimation accuracy. The proposed optimal pilot pattern is independent of the number of receive antennas, the Doppler shifts, the train position, or the train speed. Simulation results confirms the performance merits of the proposed scheme in high-mobility environments. In addition, it is also shown that the proposed scheme is robust to the respect of high mobility. △ Less

Submitted 1 March, 2020; originally announced March 2020.

Improving Sampling Accuracy of Stochastic Gradient MCMC Methods via Non-uniform Subsampling of Gradients

Authors: Ruilin Li, Xin Wang, Hongyuan Zha, Molei Tao

Abstract: Many Markov Chain Monte Carlo (MCMC) methods leverage gradient information of the potential function of target distribution to explore sample space efficiently. However, computing gradients can often be computationally expensive for large scale applications, such as those in contemporary machine learning. Stochastic Gradient (SG-)MCMC methods approximate gradients by stochastic ones, commonly via… ▽ More Many Markov Chain Monte Carlo (MCMC) methods leverage gradient information of the potential function of target distribution to explore sample space efficiently. However, computing gradients can often be computationally expensive for large scale applications, such as those in contemporary machine learning. Stochastic Gradient (SG-)MCMC methods approximate gradients by stochastic ones, commonly via uniformly subsampled data points, and achieve improved computational efficiency, however at the price of introducing sampling error. We propose a non-uniform subsampling scheme to improve the sampling accuracy. The proposed exponentially weighted stochastic gradient (EWSG) is designed so that a non-uniform-SG-MCMC method mimics the statistical behavior of a batch-gradient-MCMC method, and hence the inaccuracy due to SG approximation is reduced. EWSG differs from classical variance reduction (VR) techniques as it focuses on the entire distribution instead of just the variance; nevertheless, its reduced local variance is also proved. EWSG can also be viewed as an extension of the importance sampling idea, successful for stochastic-gradient-based optimizations, to sampling tasks. In our practical implementation of EWSG, the non-uniform subsampling is performed efficiently via a Metropolis-Hastings chain on the data index, which is coupled to the MCMC algorithm. Numerical experiments are provided, not only to demonstrate EWSG's effectiveness, but also to guide hyperparameter choices, and validate our \emph{non-asymptotic global error bound} despite of approximations in the implementation. Notably, while statistical accuracy is improved, convergence speed can be comparable to the uniform version, which renders EWSG a practical alternative to VR (but EWSG and VR can be combined too). △ Less

Submitted 22 September, 2021; v1 submitted 20 February, 2020; originally announced February 2020.

Forecasting Realized Volatility Matrix With Copula-Based Models

Authors: Wenjing Wang, Minjing Tao

Abstract: Multivariate volatility modeling and forecasting are crucial in financial economics. This paper develops a copula-based approach to model and forecast realized volatility matrices. The proposed copula-based time series models can capture the hidden dependence structure of realized volatility matrices. Also, this approach can automatically guarantee the positive definiteness of the forecasts throug… ▽ More Multivariate volatility modeling and forecasting are crucial in financial economics. This paper develops a copula-based approach to model and forecast realized volatility matrices. The proposed copula-based time series models can capture the hidden dependence structure of realized volatility matrices. Also, this approach can automatically guarantee the positive definiteness of the forecasts through either Cholesky decomposition or matrix logarithm transformation. In this paper we consider both multivariate and bivariate copulas; the types of copulas include Student's t, Clayton and Gumbel copulas. In an empirical application, we find that for one-day ahead volatility matrix forecasting, these copula-based models can achieve significant performance both in terms of statistical precision as well as creating economically mean-variance efficient portfolio. Among the copulas we considered, the multivariate-t copula performs better in statistical precision, while bivariate-t copula has better economical performance. △ Less

Submitted 20 February, 2020; originally announced February 2020.

Comments: 26 pages, 3 figures

Why Do Deep Residual Networks Generalize Better than Deep Feedforward Networks? -- A Neural Tangent Kernel Perspective

Authors: Kaixuan Huang, Yuqing Wang, Molei Tao, Tuo Zhao

Abstract: Deep residual networks (ResNets) have demonstrated better generalization performance than deep feedforward networks (FFNets). However, the theory behind such a phenomenon is still largely unknown. This paper studies this fundamental problem in deep learning from a so-called "neural tangent kernel" perspective. Specifically, we first show that under proper conditions, as the width goes to infinity,… ▽ More Deep residual networks (ResNets) have demonstrated better generalization performance than deep feedforward networks (FFNets). However, the theory behind such a phenomenon is still largely unknown. This paper studies this fundamental problem in deep learning from a so-called "neural tangent kernel" perspective. Specifically, we first show that under proper conditions, as the width goes to infinity, training deep ResNets can be viewed as learning reproducing kernel functions with some kernel function. We then compare the kernel of deep ResNets with that of deep FFNets and discover that the class of functions induced by the kernel of FFNets is asymptotically not learnable, as the depth goes to infinity. In contrast, the class of functions induced by the kernel of ResNets does not exhibit such degeneracy. Our discovery partially justifies the advantages of deep ResNets over deep FFNets in generalization abilities. Numerical results are provided to support our claim. △ Less

Submitted 22 December, 2020; v1 submitted 14 February, 2020; originally announced February 2020.

Comments: Accepted in NeurIPS 2020

Stochasticity of Deterministic Gradient Descent: Large Learning Rate for Multiscale Objective Function

Authors: Lingkai Kong, Molei Tao

Abstract: This article suggests that deterministic Gradient Descent, which does not use any stochastic gradient approximation, can still exhibit stochastic behaviors. In particular, it shows that if the objective function exhibit multiscale behaviors, then in a large learning rate regime which only resolves the macroscopic but not the microscopic details of the objective, the deterministic GD dynamics can b… ▽ More This article suggests that deterministic Gradient Descent, which does not use any stochastic gradient approximation, can still exhibit stochastic behaviors. In particular, it shows that if the objective function exhibit multiscale behaviors, then in a large learning rate regime which only resolves the macroscopic but not the microscopic details of the objective, the deterministic GD dynamics can become chaotic and convergent not to a local minimizer but to a statistical distribution. A sufficient condition is also established for approximating this long-time statistical limit by a rescaled Gibbs distribution. Both theoretical and numerical demonstrations are provided, and the theoretical part relies on the construction of a stochastic map that uses bounded noise (as opposed to discretized diffusions). △ Less

Submitted 2 November, 2020; v1 submitted 14 February, 2020; originally announced February 2020.

Comments: NeurIPS 2020. v1->v2: Weakened conditions needed for the theory. Added connections to neural network. Corrected typo

Time-Varying Gaussian-Cauchy Mixture Models for Financial Risk Management

Authors: Shuguang Zhang, Minjing Tao, Xu-Feng Niu, Fred Huffer

Abstract: There are various metrics for financial risk, such as value at risk (VaR), expected shortfall, expected/unexpected loss, etc. When estimating these metrics, it was very common to assume Gaussian distribution for the asset returns, which may underestimate the real risk of the market, especially during the financial crisis. In this paper, we propose a series of time-varying mixture models for risk a… ▽ More There are various metrics for financial risk, such as value at risk (VaR), expected shortfall, expected/unexpected loss, etc. When estimating these metrics, it was very common to assume Gaussian distribution for the asset returns, which may underestimate the real risk of the market, especially during the financial crisis. In this paper, we propose a series of time-varying mixture models for risk analysis and management. These mixture models contain two components: one component with Gaussian distribution, and the other one with a fat-tailed Cauchy distribution. We allow the distribution parameters and component weights to change over time to increase the flexibility of the models. Monte Carlo Expectation-Maximization algorithm is utilized to estimate the parameters. To verify the good performance of our models, we conduct some simulation studies, and implement our models to the real stock market. Based on these studies, our models are appropriate under different economic conditions, and the component weights can capture the correct pattern of the market volatility. △ Less

Submitted 14 February, 2020; originally announced February 2020.

Mobile Communications, Computing and Caching Resources Optimization for Coded Caching with Device Computing

Authors: Yingjiao Li, Zhiyong Chen, Meixia Tao

Abstract: Edge caching and computing have been regarded as an efficient approach to tackle the wireless spectrum crunch problem. In this paper, we design a general coded caching with device computing strategy for content computation, e.g., virtual reality (VR) rendering, to minimize the average transmission bandwidth with the caching capacity and the energy constraints of each mobile device, and the maximum… ▽ More Edge caching and computing have been regarded as an efficient approach to tackle the wireless spectrum crunch problem. In this paper, we design a general coded caching with device computing strategy for content computation, e.g., virtual reality (VR) rendering, to minimize the average transmission bandwidth with the caching capacity and the energy constraints of each mobile device, and the maximum tolerable delay constraint of each task. The key enabler is that because both coded data and stored data can be the data before or after computing, the proposed scheme has numerous edge computing and caching paths corresponding to different bandwidth requirement. We thus formulate a joint coded caching and computing optimization problem to decide whether the mobile devices cache the input data or the output data, which tasks to be coded cached and which tasks to compute locally. The optimization problem is shown to be 0-1 nonconvex nonsmooth programming and can be decomposed into the computation programming and the coded caching programming. We prove the convergence of the computation programming problem by utilizing the alternating direction method of multipliers (ADMM), and a stationary point can be obtained. For the coded cache programming, we design a low complexity algorithm to obtain an acceptable solution. Numerical results demonstrate that the proposed scheme provides a significant bandwidth saving by taking full advantage of the caching and computing capability of mobile devices. △ Less

Submitted 14 February, 2020; originally announced February 2020.

Comments: the short version of this paper is to appear in IEEE ICC 2020