Showing 1–8 of 8 results for author: Rivera, P E I

LLM-Based Intent Processing and Network Optimization Using Attention-Based Hierarchical Reinforcement Learning

Authors: Md Arafat Habib, Pedro Enrique Iturria Rivera, Yigit Ozcan, Medhat Elsayed, Majid Bavand, Raimundus Gaigalas, Melike Erol-Kantarci

Abstract: Intent-based network automation is a promising tool to enable easier network management however certain challenges need to be effectively addressed. These are: 1) processing intents, i.e., identification of logic and necessary parameters to fulfill an intent, 2) validating an intent to align it with current network status, and 3) satisfying intents via network optimizing functions like xApps and r… ▽ More Intent-based network automation is a promising tool to enable easier network management however certain challenges need to be effectively addressed. These are: 1) processing intents, i.e., identification of logic and necessary parameters to fulfill an intent, 2) validating an intent to align it with current network status, and 3) satisfying intents via network optimizing functions like xApps and rApps in O-RAN. This paper addresses these points via a three-fold strategy to introduce intent-based automation for O-RAN. First, intents are processed via a lightweight Large Language Model (LLM). Secondly, once an intent is processed, it is validated against future incoming traffic volume profiles (high or low). Finally, a series of network optimization applications (rApps and xApps) have been developed. With their machine learning-based functionalities, they can improve certain key performance indicators such as throughput, delay, and energy efficiency. In this final stage, using an attention-based hierarchical reinforcement learning algorithm, these applications are optimally initiated to satisfy the intent of an operator. Our simulations show that the proposed method can achieve at least 12% increase in throughput, 17.1% increase in energy efficiency, and 26.5% decrease in network delay compared to the baseline algorithms. △ Less

Submitted 10 June, 2024; originally announced June 2024.

Comments: Submitted paper to GLOBECOM 2024

RL meets Multi-Link Operation in IEEE 802.11be: Multi-Headed Recurrent Soft-Actor Critic-based Traffic Allocation

Authors: Pedro Enrique Iturria Rivera, Marcel Chenier, Bernard Herscovici, Burak Kantarci, Melike Erol-Kantarci

Abstract: IEEE 802.11be -Extremely High Throughput-, commercially known as Wireless-Fidelity (Wi-Fi) 7 is the newest IEEE 802.11 amendment that comes to address the increasingly throughput hungry services such as Ultra High Definition (4K/8K) Video and Virtual/Augmented Reality (VR/AR). To do so, IEEE 802.11be presents a set of novel features that will boost the Wi-Fi technology to its edge. Among them, Mul… ▽ More IEEE 802.11be -Extremely High Throughput-, commercially known as Wireless-Fidelity (Wi-Fi) 7 is the newest IEEE 802.11 amendment that comes to address the increasingly throughput hungry services such as Ultra High Definition (4K/8K) Video and Virtual/Augmented Reality (VR/AR). To do so, IEEE 802.11be presents a set of novel features that will boost the Wi-Fi technology to its edge. Among them, Multi-Link Operation (MLO) devices are anticipated to become a reality, leaving Single-Link Operation (SLO) Wi-Fi in the past. To achieve superior throughput and very low latency, a careful design approach must be taken, on how the incoming traffic is distributed in MLO capable devices. In this paper, we present a Reinforcement Learning (RL) algorithm named Multi-Headed Recurrent Soft-Actor Critic (MH-RSAC) to distribute incoming traffic in 802.11be MLO capable networks. Moreover, we compare our results with two non-RL baselines previously proposed in the literature named: Single Link Less Congested Interface (SLCI) and Multi-Link Congestion-aware Load balancing at flow arrivals (MCAA). Simulation results reveal that the MH-RSAC algorithm is able to obtain gains in terms of Throughput Drop Ratio (TDR) up to 35.2% and 6% when compared with the SLCI and MCAA algorithms, respectively. Finally, we observed that our scheme is able to respond more efficiently to high throughput and dynamic traffic such as VR and Web Browsing (WB) when compared with the baselines. Results showed an improvement of the MH-RSAC scheme in terms of Flow Satisfaction (FS) of up to 25.6% and 6% over the the SCLI and MCAA algorithms. △ Less

Submitted 15 March, 2023; originally announced March 2023.

Comments: Accepted in ICC'23

Uplink Scheduling in Federated Learning: an Importance-Aware Approach via Graph Representation Learning

Authors: Marco Skocaj, Pedro Enrique Iturria Rivera, Roberto Verdone, Melike Erol-Kantarci

Abstract: Federated Learning (FL) has emerged as a promising framework for distributed training of AI-based services, applications, and network procedures in 6G. One of the major challenges affecting the performance and efficiency of 6G wireless FL systems is the massive scheduling of user devices over resource-constrained channels. In this work, we argue that the uplink scheduling of FL client devices is a… ▽ More Federated Learning (FL) has emerged as a promising framework for distributed training of AI-based services, applications, and network procedures in 6G. One of the major challenges affecting the performance and efficiency of 6G wireless FL systems is the massive scheduling of user devices over resource-constrained channels. In this work, we argue that the uplink scheduling of FL client devices is a problem with a rich relational structure. To address this challenge, we propose a novel, energy-efficient, and importance-aware metric for client scheduling in FL applications by leveraging Unsupervised Graph Representation Learning (UGRL). Our proposed approach introduces a relational inductive bias in the scheduling process and does not require the collection of training feedback information from client devices, unlike state-of-the-art importance-aware mechanisms. We evaluate our proposed solution against baseline scheduling algorithms based on recently proposed metrics in the literature. Results show that, when considering scenarios of nodes exhibiting spatial relations, our approach can achieve an average gain of up to 10% in model accuracy and up to 17 times in energy efficiency compared to state-of-the-art importance-aware policies. △ Less

Submitted 27 January, 2023; originally announced January 2023.

Comments: 6 pages, 6 figures, conference paper

Hierarchical Deep Q-Learning Based Handover in Wireless Networks with Dual Connectivity

Authors: Pedro Enrique Iturria Rivera, Medhat Elsayed, Majid Bavand, Raimundas Gaigalas, Steve Furr, Melike Erol-Kantarci

Abstract: 5G New Radio proposes the usage of frequencies above 10 GHz to speed up LTE's existent maximum data rates. However, the effective size of 5G antennas and consequently its repercussions in the signal degradation in urban scenarios makes it a challenge to maintain stable coverage and connectivity. In order to obtain the best from both technologies, recent dual connectivity solutions have proved thei… ▽ More 5G New Radio proposes the usage of frequencies above 10 GHz to speed up LTE's existent maximum data rates. However, the effective size of 5G antennas and consequently its repercussions in the signal degradation in urban scenarios makes it a challenge to maintain stable coverage and connectivity. In order to obtain the best from both technologies, recent dual connectivity solutions have proved their capabilities to improve performance when compared with coexistent standalone 5G and 4G technologies. Reinforcement learning (RL) has shown its huge potential in wireless scenarios where parameter learning is required given the dynamic nature of such context. In this paper, we propose two reinforcement learning algorithms: a single agent RL algorithm named Clipped Double Q-Learning (CDQL) and a hierarchical Deep Q-Learning (HiDQL) to improve Multiple Radio Access Technology (multi-RAT) dual-connectivity handover. We compare our proposal with two baselines: a fixed parameter and a dynamic parameter solution. Simulation results reveal significant improvements in terms of latency with a gain of 47.6% and 26.1% for Digital-Analog beamforming (BF), 17.1% and 21.6% for Hybrid-Analog BF, and 24.7% and 39% for Analog-Analog BF when comparing the RL-schemes HiDQL and CDQL with the with the existent solutions, HiDQL presented a slower convergence time, however obtained a more optimal solution than CDQL. Additionally, we foresee the advantages of utilizing context-information as geo-location of the UEs to reduce the beam exploration sector, and thus improving further multi-RAT handover latency results. △ Less

Submitted 13 January, 2023; originally announced January 2023.

Comments: 5 Figures, 4 tables, 2 algorithms. Accepted in Globecom'22

Traffic Steering for 5G Multi-RAT Deployments using Deep Reinforcement Learning

Authors: Md Arafat Habib, Hao Zhou, Pedro Enrique Iturria Rivera, Medhat Elsayed, Majid Bavand, Raimundas Gaigalas, Steve Furr, Melike Erol-Kantarci

Abstract: In 5G non-standalone mode, traffic steering is a critical technique to take full advantage of 5G new radio while optimizing dual connectivity of 5G and LTE networks in multiple radio access technology (RAT). An intelligent traffic steering mechanism can play an important role to maintain seamless user experience by choosing appropriate RAT (5G or LTE) dynamically for a specific user traffic flow w… ▽ More In 5G non-standalone mode, traffic steering is a critical technique to take full advantage of 5G new radio while optimizing dual connectivity of 5G and LTE networks in multiple radio access technology (RAT). An intelligent traffic steering mechanism can play an important role to maintain seamless user experience by choosing appropriate RAT (5G or LTE) dynamically for a specific user traffic flow with certain QoS requirements. In this paper, we propose a novel traffic steering mechanism based on Deep Q-learning that can automate traffic steering decisions in a dynamic environment having multiple RATs, and maintain diverse QoS requirements for different traffic classes. The proposed method is compared with two baseline algorithms: a heuristic-based algorithm and Q-learningbased traffic steering. Compared to the Q-learning and heuristic baselines, our results show that the proposed algorithm achieves better performance in terms of 6% and 10% higher average system throughput, and 23% and 33% lower network delay, respectively. △ Less

Submitted 12 January, 2023; originally announced January 2023.

Comments: 6 pages, 6 figures and 1 table. Accepted in CCNC'23

Competitive Multi-Agent Load Balancing with Adaptive Policies in Wireless Networks

Authors: Pedro Enrique Iturria Rivera, Melike Erol-Kantarci

Abstract: Using Machine Learning (ML) techniques for the next generation wireless networks have shown promising results in the recent years, due to high learning and adaptation capability of ML algorithms. More specifically, ML techniques have been used for load balancing in Self-Organizing Networks (SON). In the context of load balancing and ML, several studies propose network management automation (NMA) f… ▽ More Using Machine Learning (ML) techniques for the next generation wireless networks have shown promising results in the recent years, due to high learning and adaptation capability of ML algorithms. More specifically, ML techniques have been used for load balancing in Self-Organizing Networks (SON). In the context of load balancing and ML, several studies propose network management automation (NMA) from the perspective of a single and centralized agent. However, a single agent domain does not consider the interaction among the agents. In this paper, we propose a more realistic load balancing approach using novel Multi-Agent Deep Deterministic Policy Gradient with Adaptive Policies (MADDPG-AP) scheme that considers throughput, resource block utilization and latency in the network. We compare our proposal with a single-agent RL algorithm named Clipped Double Q-Learning (CDQL) . Simulation results reveal a significant improvement in latency, packet loss ratio and convergence time △ Less

Submitted 13 October, 2021; originally announced October 2021.

QoS-Aware Load Balancing in Wireless Networks using Clipped Double Q-Learning

Authors: Pedro Enrique Iturria Rivera, Melike Erol-Kantarci

Abstract: In recent years, long-term evolution (LTE) and 5G NR (5th Generation New Radio) technologies have showed great potential to utilize Machine Learning (ML) algorithms in optimizing their operations, both thanks to the availability of fine-grained data from the field, as well as the need arising from growing complexity of networks. The aforementioned complexity sparked mobile operators' attention as… ▽ More In recent years, long-term evolution (LTE) and 5G NR (5th Generation New Radio) technologies have showed great potential to utilize Machine Learning (ML) algorithms in optimizing their operations, both thanks to the availability of fine-grained data from the field, as well as the need arising from growing complexity of networks. The aforementioned complexity sparked mobile operators' attention as a way to reduce the capital expenditures (CAPEX) and the operational (OPEX) expenditures of their networks through network management automation (NMA). NMA falls under the umbrella of Self-Organizing Networks (SON) in which 3GPP has identified some challenges and opportunities in load balancing mechanisms for the Radio Access Networks (RANs). In the context of machine learning and load balancing, several studies have focused on maximizing the overall network throughput or the resource block utilization (RBU). In this paper, we propose a novel Clipped Double Q-Learning (CDQL)-based load balancing approach considering resource block utilization, latency and the Channel Quality Indicator (CQI). We compare our proposal with a traditional handover algorithm and a resource block utilization based handover mechanism. Simulation results reveal that our scheme is able to improve throughput, latency, jitter and packet loss ratio in comparison to the baseline algorithms. △ Less

Submitted 9 July, 2021; originally announced July 2021.

Multi Agent Team Learning in Disaggregated Virtualized Open Radio Access Networks (O-RAN)

Authors: Pedro Enrique Iturria Rivera, Shahram Mollahasani, Melike Erol-Kantarci

Abstract: Starting from the Cloud Radio Access Network (C-RAN), continuing with the virtual Radio Access Network (vRAN) and most recently with Open RAN (O-RAN) initiative, Radio Access Network (RAN) architectures have significantly evolved in the past decade. In the last few years, the wireless industry has witnessed a strong trend towards disaggregated, virtualized and open RANs, with numerous tests and de… ▽ More Starting from the Cloud Radio Access Network (C-RAN), continuing with the virtual Radio Access Network (vRAN) and most recently with Open RAN (O-RAN) initiative, Radio Access Network (RAN) architectures have significantly evolved in the past decade. In the last few years, the wireless industry has witnessed a strong trend towards disaggregated, virtualized and open RANs, with numerous tests and deployments world wide. One unique aspect that motivates this paper is the availability of new opportunities that arise from using machine learning to optimize the RAN in closed-loop, i.e. without human intervention, where the complexity of disaggregation and virtualization makes well-known Self-Organized Networking (SON) solutions inadequate. In our view, Multi-Agent Systems (MASs) with team learning, can play an essential role in the control and coordination of controllers of O-RAN, i.e. near-real-time and non-real-time RAN Intelligent Controller (RIC). In this article, we first present the state-of-the-art research in multi-agent systems and team learning, then we provide an overview of the landscape in RAN disaggregation and virtualization, as well as O-RAN which emphasizes the open interfaces introduced by the O-RAN Alliance. We present a case study for agent placement and the AI feedback required in O-RAN, and finally, we identify challenges and open issues to provide a roadmap for researchers. △ Less

Submitted 22 February, 2021; v1 submitted 8 December, 2020; originally announced December 2020.

Comments: 7 pages, 3 figures, 1 table, submitted to IEEE Wireless Communications Magazine on Feb, 2021