-
Learning Decentralized Traffic Signal Controllers with Multi-Agent Graph Reinforcement Learning
Authors:
Yao Zhang,
Zhiwen Yu,
Jun Zhang,
Liang Wang,
Tom H. Luan,
Bin Guo,
Chau Yuen
Abstract:
This paper considers optimal traffic signal control in smart cities, which has been taken as a complex networked system control problem. Given the interacting dynamics among traffic lights and road networks, attaining controller adaptivity and scalability stands out as a primary challenge. Capturing the spatial-temporal correlation among traffic lights under the framework of Multi-Agent Reinforcem…
▽ More
This paper considers optimal traffic signal control in smart cities, which has been taken as a complex networked system control problem. Given the interacting dynamics among traffic lights and road networks, attaining controller adaptivity and scalability stands out as a primary challenge. Capturing the spatial-temporal correlation among traffic lights under the framework of Multi-Agent Reinforcement Learning (MARL) is a promising solution. Nevertheless, existing MARL algorithms ignore effective information aggregation which is fundamental for improving the learning capacity of decentralized agents. In this paper, we design a new decentralized control architecture with improved environmental observability to capture the spatial-temporal correlation. Specifically, we first develop a topology-aware information aggregation strategy to extract correlation-related information from unstructured data gathered in the road network. Particularly, we transfer the road network topology into a graph shift operator by forming a diffusion process on the topology, which subsequently facilitates the construction of graph signals. A diffusion convolution module is developed, forming a new MARL algorithm, which endows agents with the capabilities of graph learning. Extensive experiments based on both synthetic and real-world datasets verify that our proposal outperforms existing decentralized algorithms.
△ Less
Submitted 7 November, 2023;
originally announced November 2023.
-
Label-free Deep Learning Driven Secure Access Selection in Space-Air-Ground Integrated Networks
Authors:
Zhaowei Wang,
Zhisheng Yin,
Xiucheng Wang,
Nan Cheng,
Yuan Zhang,
Tom H. Luan
Abstract:
In Space-air-ground integrated networks (SAGIN), the inherent openness and extensive broadcast coverage expose these networks to significant eavesdropping threats. Considering the inherent co-channel interference due to spectrum sharing among multi-tier access networks in SAGIN, it can be leveraged to assist the physical layer security among heterogeneous transmissions. However, it is challenging…
▽ More
In Space-air-ground integrated networks (SAGIN), the inherent openness and extensive broadcast coverage expose these networks to significant eavesdropping threats. Considering the inherent co-channel interference due to spectrum sharing among multi-tier access networks in SAGIN, it can be leveraged to assist the physical layer security among heterogeneous transmissions. However, it is challenging to conduct a secrecy-oriented access strategy due to both heterogeneous resources and different eavesdropping models. In this paper, we explore secure access selection for a scenario involving multi-mode users capable of accessing satellites, unmanned aerial vehicles, or base stations in the presence of eavesdroppers. Particularly, we propose a Q-network approximation based deep learning approach for selecting the optimal access strategy for maximizing the sum secrecy rate. Meanwhile, the power optimization is also carried out by an unsupervised learning approach to improve the secrecy performance. Remarkably, two neural networks are trained by unsupervised learning and Q-network approximation which are both label-free methods without knowing the optimal solution as labels. Numerical results verify the efficiency of our proposed power optimization approach and access strategy, leading to enhanced secure transmission performance.
△ Less
Submitted 28 August, 2023;
originally announced August 2023.
-
Green Interference Based Symbiotic Security in Integrated Satellite-terrestrial Communications
Authors:
Zhisheng Yin,
Nan Cheng,
Tom H. Luan,
Yilong Hui,
Wei Wang
Abstract:
In this paper, we investigate secure transmissions in integrated satellite-terrestrial communications and the green interference based symbiotic security scheme is proposed. Particularly, the co-channel interference induced by the spectrum sharing between satellite and terrestrial networks and the inter-beam interference due to frequency reuse among satellite multi-beam serve as the green interfer…
▽ More
In this paper, we investigate secure transmissions in integrated satellite-terrestrial communications and the green interference based symbiotic security scheme is proposed. Particularly, the co-channel interference induced by the spectrum sharing between satellite and terrestrial networks and the inter-beam interference due to frequency reuse among satellite multi-beam serve as the green interference to assist the symbiotic secure transmission, where the secure transmissions of both satellite and terrestrial links are guaranteed simultaneously. Specifically, to realize the symbiotic security, we formulate a problem to maximize the sum secrecy rate of satellite users by cooperatively beamforming optimizing and a constraint of secrecy rate of each terrestrial user is guaranteed. Since the formulated problem is non-convex and intractable, the Taylor expansion and semi-definite relaxation (SDR) are adopted to further reformulate this problem, and the successive convex approximation (SCA) algorithm is designed to solve it. Finally, the tightness of the relaxation is proved. In addition, numerical results verify the efficiency of our proposed approach.
△ Less
Submitted 6 June, 2022;
originally announced June 2022.
-
Collaborative Driving: Learning- Aided Joint Topology Formulation and Beamforming
Authors:
Yao Zhang,
Changle Li,
Tom H. Luan,
Chau Yuen Yuchuan Fu
Abstract:
Currently, autonomous vehicles are able to drive more naturally based on the driving policies learned from millions of driving miles in real environments. However, to further improve the automation level of vehicles is a challenging task, especially in the case of multi-vehicle cooperation. In recent heated discussions of 6G, millimeter-wave (mmWave) and terahertz (THz) bands are deemed to play im…
▽ More
Currently, autonomous vehicles are able to drive more naturally based on the driving policies learned from millions of driving miles in real environments. However, to further improve the automation level of vehicles is a challenging task, especially in the case of multi-vehicle cooperation. In recent heated discussions of 6G, millimeter-wave (mmWave) and terahertz (THz) bands are deemed to play important roles in new radio communication architectures and algorithms. To enable reliable autonomous driving in 6G, in this paper, we envision collaborative autonomous driving, a new framework that jointly controls driving topology and formulate vehicular networks in the mmWave/THz bands. As a swarm intelligence system, the collaborative driving scheme goes beyond existing autonomous driving patterns based on single-vehicle intelligence in terms of safety and efficiency. With efficient data sharing, the proposed framework is able to achieve cooperative sensing and load balancing so that improve sensing efficiency with saved computational resources. To deal with the new challenges in the collaborative driving framework, we further illustrate two promising approaches for mmWave/THz-based vehicle-to-vehicle (V2V) communications. Finally, we discuss several potential open research problems for the proposed collaborative driving scheme.
△ Less
Submitted 18 March, 2022;
originally announced March 2022.
