Showing 1–5 of 5 results for author: Rezaabad, A L

Few-Max: Few-Shot Domain Adaptation for Unsupervised Contrastive Representation Learning

Authors: Ali Lotfi Rezaabad, Sidharth Kumar, Sriram Vishwanath, Jonathan I. Tamir

Abstract: Contrastive self-supervised learning methods learn to map data points such as images into non-parametric representation space without requiring labels. While highly successful, current methods require a large amount of data in the training phase. In situations where the target training set is limited in size, generalization is known to be poor. Pretraining on a large source data set and fine-tunin… ▽ More Contrastive self-supervised learning methods learn to map data points such as images into non-parametric representation space without requiring labels. While highly successful, current methods require a large amount of data in the training phase. In situations where the target training set is limited in size, generalization is known to be poor. Pretraining on a large source data set and fine-tuning on the target samples is prone to overfitting in the few-shot regime, where only a small number of target samples are available. Motivated by this, we propose a domain adaption method for self-supervised contrastive learning, termed Few-Max, to address the issue of adaptation to a target distribution under few-shot learning. To quantify the representation quality, we evaluate Few-Max on a range of source and target datasets, including ImageNet, VisDA, and fastMRI, on which Few-Max consistently outperforms other approaches. △ Less

Submitted 22 June, 2022; v1 submitted 21 June, 2022; originally announced June 2022.

Hyperbolic Graph Embedding with Enhanced Semi-Implicit Variational Inference

Authors: Ali Lotfi Rezaabad, Rahi Kalantari, Sriram Vishwanath, Mingyuan Zhou, Jonathan Tamir

Abstract: Efficient modeling of relational data arising in physical, social, and information sciences is challenging due to complicated dependencies within the data. In this work, we build off of semi-implicit graph variational auto-encoders to capture higher-order statistics in a low-dimensional graph latent representation. We incorporate hyperbolic geometry in the latent space through a Poincare embedding… ▽ More Efficient modeling of relational data arising in physical, social, and information sciences is challenging due to complicated dependencies within the data. In this work, we build off of semi-implicit graph variational auto-encoders to capture higher-order statistics in a low-dimensional graph latent representation. We incorporate hyperbolic geometry in the latent space through a Poincare embedding to efficiently represent graphs exhibiting hierarchical structure. To address the naive posterior latent distribution assumptions in classical variational inference, we use semi-implicit hierarchical variational Bayes to implicitly capture posteriors of given graph data, which may exhibit heavy tails, multiple modes, skewness, and highly correlated latent structures. We show that the existing semi-implicit variational inference objective provably reduces information in the observed graph. Based on this observation, we estimate and add an additional mutual information term to the semi-implicit variational inference learning objective to capture rich correlations arising between the input and latent spaces. We show that the inclusion of this regularization term in conjunction with the Poincare embedding boosts the quality of learned high-level representations and enables more flexible and faithful graphical modeling. We experimentally demonstrate that our approach outperforms existing graph variational auto-encoders both in Euclidean and in hyperbolic spaces for edge link prediction and node classification. △ Less

Submitted 10 March, 2021; v1 submitted 31 October, 2020; originally announced November 2020.

Long Short-Term Memory Spiking Networks and Their Applications

Authors: Ali Lotfi Rezaabad, Sriram Vishwanath

Abstract: Recent advances in event-based neuromorphic systems have resulted in significant interest in the use and development of spiking neural networks (SNNs). However, the non-differentiable nature of spiking neurons makes SNNs incompatible with conventional backpropagation techniques. In spite of the significant progress made in training conventional deep neural networks (DNNs), training methods for SNN… ▽ More Recent advances in event-based neuromorphic systems have resulted in significant interest in the use and development of spiking neural networks (SNNs). However, the non-differentiable nature of spiking neurons makes SNNs incompatible with conventional backpropagation techniques. In spite of the significant progress made in training conventional deep neural networks (DNNs), training methods for SNNs still remain relatively poorly understood. In this paper, we present a novel framework for training recurrent SNNs. Analogous to the benefits presented by recurrent neural networks (RNNs) in learning time series models within DNNs, we develop SNNs based on long short-term memory (LSTM) networks. We show that LSTM spiking networks learn the timing of the spikes and temporal dependencies. We also develop a methodology for error backpropagation within LSTM-based SNNs. The developed architecture and method for backpropagation within LSTM-based SNNs enable them to learn long-term dependencies with comparable results to conventional LSTMs. △ Less

Submitted 9 July, 2020; originally announced July 2020.

Learning Representations by Maximizing Mutual Information in Variational Autoencoders

Authors: Ali Lotfi Rezaabad, Sriram Vishwanath

Abstract: Variational autoencoders (VAEs) have ushered in a new era of unsupervised learning methods for complex distributions. Although these techniques are elegant in their approach, they are typically not useful for representation learning. In this work, we propose a simple yet powerful class of VAEs that simultaneously result in meaningful learned representations. Our solution is to combine traditional… ▽ More Variational autoencoders (VAEs) have ushered in a new era of unsupervised learning methods for complex distributions. Although these techniques are elegant in their approach, they are typically not useful for representation learning. In this work, we propose a simple yet powerful class of VAEs that simultaneously result in meaningful learned representations. Our solution is to combine traditional VAEs with mutual information maximization, with the goal to enhance amortized inference in VAEs using Information Theoretic techniques. We call this approach InfoMax-VAE, and such an approach can significantly boost the quality of learned high-level representations. We realize this through the explicit maximization of information measures associated with the representation. Using extensive experiments on varied datasets and setups, we show that InfoMax-VAE outperforms contemporary popular approaches, including Info-VAE and $β$-VAE. △ Less

Submitted 7 January, 2020; v1 submitted 21 December, 2019; originally announced December 2019.

Ultra-Dense 5G Small Cell Deployment for Fiber and Wireless Backhaul-Aware Infrastructures

Authors: Ali Lotfi Rezaabad, Hamzeh Beyranvand, Jawad A. Salehi, Martin Maier

Abstract: In this paper, we study the cell planning problem for a two-tier cellular network containing two types of base stations (BSs)-- i.e. with fiber backhaul, referred to as wired BSs (W-BSs), and BSs with wireless backhaul, referred to as unwired-BSs (U-BSs). In-band full-duplex wireless communications is used to connect U-BSs and W-BSs. We propose an algorithm to determine the minimum number of W-BSs… ▽ More In this paper, we study the cell planning problem for a two-tier cellular network containing two types of base stations (BSs)-- i.e. with fiber backhaul, referred to as wired BSs (W-BSs), and BSs with wireless backhaul, referred to as unwired-BSs (U-BSs). In-band full-duplex wireless communications is used to connect U-BSs and W-BSs. We propose an algorithm to determine the minimum number of W-BSs and U-BSs to satisfy given cell and capacity coverage constraints. Furthermore, we apply our proposed non-dominated sorting genetic algorithm II (NSGA-II) to solve both cell planning and joint cell and backhaul planning problem to minimize the cost of planning, while maximizing the coverage simultaneously. Additionally, the considered cell planning program is developed into an optimization by including the problem of minimizing the cost of fiber backhaul deployment. In order to analyze the performance of the proposed algorithm, we study three different deployment scenarios based on different spatial distributions of users and coverage areas. The results show the superiority of our proposed NSGA-II algorithm for both cell planning and joint cell and backhaul planning to other well-known optimization algorithms. The results also reveal that there is a trade-off between cell deployment costs and SINR/rate coverage, and W-BSs are placed in congested areas to consume less resources for wireless backhauls. Similarly, a trade-off between cell and fiber deployment costs and SINR/rate coverage is observed in planning. We show that for realistic scenarios desirable solutions can be selected from the Pareto front of the introduced multi-objective problem based on given cellular operator policies. △ Less

Submitted 22 August, 2018; v1 submitted 3 August, 2018; originally announced August 2018.