Deep Reinforcemnet Learning for Robust Beamforming in Integrated Sensing, Communication and Power Transmission Systems

A communication network integrating multiple modes can effectively support the sustainable development of next-generation wireless communications. Integrated sensing, communication, and power transfer (ISCPT) represents an emerging technological paradigm that not only facilitates information transmission but also enables environmental sensing and wireless power transfer. To achieve optimal beamforming in transmission, it is crucial to satisfy multiple constraints, including quality of service (QoS), radar sensing accuracy, and power transfer efficiency, while ensuring fundamental system performance. The presence of multiple parametric constraints makes the problem a non-convex optimization challenge, underscoring the need for a solution that balances low computational complexity with high precision. Additionally, the accuracy of channel state information (CSI) is pivotal in determining the achievable rate, as imperfect or incomplete CSI can significantly degrade system performance and beamforming efficiency. Deep reinforcement learning (DRL), a machine learning technique where an agent learns by interacting with its environment, offers a promising approach that can dynamically optimize system performance through adaptive decision-making strategies. In this paper, we propose a DRL-based ISCPT framework, which effectively manages complex environmental states and continuously adjusts variables related to sensing, communication, and energy harvesting to enhance overall system efficiency and reliability. The achievable rate upper bound can be inferred through robust, learnable beamforming in the ISCPT system. Our results demonstrate that DRL-based algorithms significantly improve resource allocation, power management, and information transmission, particularly in dynamic and uncertain environments with imperfect CSI.