-
Distributed Power Control for Large Energy Harvesting Networks: A Multi-Agent Deep Reinforcement Learning Approach
Authors:
Mohit K. Sharma,
Alessio Zappone,
Mohamad Assaad,
Merouane Debbah,
Spyridon Vassilaras
Abstract:
In this paper, we develop a multi-agent reinforcement learning (MARL) framework to obtain online power control policies for a large energy harvesting (EH) multiple access channel, when only causal information about the EH process and wireless channel is available. In the proposed framework, we model the online power control problem as a discrete-time mean-field game (MFG), and analytically show th…
▽ More
In this paper, we develop a multi-agent reinforcement learning (MARL) framework to obtain online power control policies for a large energy harvesting (EH) multiple access channel, when only causal information about the EH process and wireless channel is available. In the proposed framework, we model the online power control problem as a discrete-time mean-field game (MFG), and analytically show that the MFG has a unique stationary solution. Next, we leverage the fictitious play property of the mean-field games, and the deep reinforcement learning technique to learn the stationary solution of the game, in a completely distributed fashion. We analytically show that the proposed procedure converges to the unique stationary solution of the MFG. This, in turn, ensures that the optimal policies can be learned in a completely distributed fashion. In order to benchmark the performance of the distributed policies, we also develop a deep neural network (DNN) based centralized as well as distributed online power control schemes. Our simulation results show the efficacy of the proposed power control policies. In particular, the DNN based centralized power control policies provide a very good performance for large EH networks for which the design of optimal policies is intractable using the conventional methods such as Markov decision processes. Further, performance of both the distributed policies is close to the throughput achieved by the centralized policies.
△ Less
Submitted 22 October, 2019; v1 submitted 1 April, 2019;
originally announced April 2019.
-
Problem-Adapted Artificial Intelligence for Online Network Optimization
Authors:
Spyridon Vassilaras,
Luigi Vigneri,
Nikolaos Liakopoulos,
Georgios S. Paschos,
Apostolos Destounis,
Thrasyvoulos Spyropoulos,
Merouane Debbah
Abstract:
Future 5G wireless networks will rely on agile and automated network management, where the usage of diverse resources must be jointly optimized with surgical accuracy. A number of key wireless network functionalities (e.g., traffic steering, power control) give rise to hard optimization problems. What is more, high spatio-temporal traffic variability coupled with the need to satisfy strict per sli…
▽ More
Future 5G wireless networks will rely on agile and automated network management, where the usage of diverse resources must be jointly optimized with surgical accuracy. A number of key wireless network functionalities (e.g., traffic steering, power control) give rise to hard optimization problems. What is more, high spatio-temporal traffic variability coupled with the need to satisfy strict per slice/service SLAs in modern networks, suggest that these problems must be constantly (re-)solved, to maintain close-to-optimal performance. To this end, we propose the framework of Online Network Optimization (ONO), which seeks to maintain both agile and efficient control over time, using an arsenal of data-driven, online learning, and AI-based techniques. Since the mathematical tools and the studied regimes vary widely among these methodologies, a theoretical comparison is often out of reach. Therefore, the important question `what is the right ONO technique?' remains open to date. In this paper, we discuss the pros and cons of each technique and present a direct quantitative comparison for a specific use case, using real data. Our results suggest that carefully combining the insights of problem modeling with state-of-the-art AI techniques provides significant advantages at reasonable complexity.
△ Less
Submitted 26 March, 2019; v1 submitted 30 May, 2018;
originally announced May 2018.
-
Optimizing Access Mechanisms for QoS Provisioning in Hardware Constrained Dynamic Spectrum Access
Authors:
Spyridon Vassilaras,
George C. Alexandropoulos
Abstract:
One of the major challenges in Dynamic Spectrum Access (DSA) systems is to guarantee a required level of Quality of Service (QoS) to secondary users of the spectrum. In this paper, we propose efficient algorithms for deriving optimal policies for the sensing / transmitting trade-off in hardware-constrained DSA systems. Unlike previous approaches which seek to maximize mean data rate for the second…
▽ More
One of the major challenges in Dynamic Spectrum Access (DSA) systems is to guarantee a required level of Quality of Service (QoS) to secondary users of the spectrum. In this paper, we propose efficient algorithms for deriving optimal policies for the sensing / transmitting trade-off in hardware-constrained DSA systems. Unlike previous approaches which seek to maximize mean data rate for the secondary users, the proposed algorithms derive policies which minimize the probability of excessive queuing delays. Large Deviations (LD) asymptotics are used to approximate the probability of interest and policies maximizing the associated LD exponent are proposed. Although dynamic programming is not able to identify the optimal policy in this case, much more efficient algorithms than exhaustive search are proposed. These algorithms are based on specific properties of the optimal policy which are described and proven in this paper.
△ Less
Submitted 7 July, 2016;
originally announced July 2016.
-
Robust measurement-based buffer overflow probability estimators for QoS provisioning and traffic anomaly prediction applicationm
Authors:
Spyridon Vassilaras,
Ioannis Ch. Paschalidis
Abstract:
Suitable estimators for a class of Large Deviation approximations of rare event probabilities based on sample realizations of random processes have been proposed in our earlier work. These estimators are expressed as non-linear multi-dimensional optimization problems of a special structure. In this paper, we develop an algorithm to solve these optimization problems very efficiently based on their…
▽ More
Suitable estimators for a class of Large Deviation approximations of rare event probabilities based on sample realizations of random processes have been proposed in our earlier work. These estimators are expressed as non-linear multi-dimensional optimization problems of a special structure. In this paper, we develop an algorithm to solve these optimization problems very efficiently based on their characteristic structure. After discussing the nature of the objective function and constraint set and their peculiarities, we provide a formal proof that the developed algorithm is guaranteed to always converge. The existence of efficient and provably convergent algorithms for solving these problems is a prerequisite for using the proposed estimators in real time problems such as call admission control, adaptive modulation and coding with QoS constraints, and traffic anomaly detection in high data rate communication networks.
△ Less
Submitted 2 May, 2016;
originally announced May 2016.
-
Placing Dynamic Content in Caches with Small Population
Authors:
Mathieu Leconte,
Georgios Paschos,
Lazaros Gkatzikis,
Moez Draief,
Spyridon Vassilaras,
Symeon Chouvardas
Abstract:
This paper addresses a fundamental limitation for the adoption of caching for wireless access networks due to small population sizes. This shortcoming is due to two main challenges: (i) making timely estimates of varying content popularity and (ii) inferring popular content from small samples. We propose a framework which alleviates such limitations.
To timely estimate varying popularity in a co…
▽ More
This paper addresses a fundamental limitation for the adoption of caching for wireless access networks due to small population sizes. This shortcoming is due to two main challenges: (i) making timely estimates of varying content popularity and (ii) inferring popular content from small samples. We propose a framework which alleviates such limitations.
To timely estimate varying popularity in a context of a single cache we propose an Age-Based Threshold (ABT) policy which caches all contents requested more times than a threshold $\widetilde N(τ)$, where $τ$ is the content age. We show that ABT is asymptotically hit rate optimal in the many contents regime, which allows us to obtain the first characterization of the optimal performance of a caching system in a dynamic context. We then address small sample sizes focusing on $L$ local caches and one global cache. On the one hand we show that the global cache learns L times faster by aggregating all requests from local caches, which improves hit rates. On the other hand, aggregation washes out local characteristics of correlated traffic which penalizes hit rate. This motivates coordination mechanisms which combine global learning of popularity scores in clusters and LRU with prefetching.
△ Less
Submitted 15 January, 2016;
originally announced January 2016.
-
Bit Error Rate Analysis of Cooperative Beamforming for Transmitting Individual Data Streams
Authors:
Spyridon Vassilaras,
George C. Alexandropoulos,
Antonis A. Kalis
Abstract:
Cooperative beamforming (CB) has been proposed as a special case of coordinated multi-point techniques in wireless communications. In wireless sensor networks, CB can enable low power communication by allowing a collection of sensor nodes to transmit data simultaneously to a distant fusion center in one hop. Besides the traditional CB approach where all nodes need to share and transmit the same da…
▽ More
Cooperative beamforming (CB) has been proposed as a special case of coordinated multi-point techniques in wireless communications. In wireless sensor networks, CB can enable low power communication by allowing a collection of sensor nodes to transmit data simultaneously to a distant fusion center in one hop. Besides the traditional CB approach where all nodes need to share and transmit the same data, a more recent technique allows each node to transmit its own data while still achieving the benefits of cooperation. However, the intricacies of varying beamforming gains in the direct sequence spread spectrum with binary frequency shift keying multiple access scheme used in this context need to be taken into account when evaluating the performance of this beamforming technique. In this paper, we take the first step towards a more comprehensive understanding of this individual-data CB technique by proposing a best suited decoding scheme and analyzing its bit error rate (BER) performance over an additive white Gaussian noise channel. Through analytical expressions and simulation results BER curves are drawn and the achieved performance improvement offered by the CB gain is quantified.
△ Less
Submitted 28 October, 2015;
originally announced October 2015.
