Enhancing the landing guidance of a reusable launch vehicle by improving genetic algorithm-based deep reinforcement learning using Hybrid Deterministic-Stochastic algorithm

PLoS One. 2024 Feb 29;19(2):e0292539. doi: 10.1371/journal.pone.0292539. eCollection 2024.

Abstract

The PbGA-DDPG algorithm, which uses a potential-based GA-optimized reward shaping function, is a versatiledeep reinforcement learning/DRLagent that can control a vehicle in a complex environment without prior knowledge. However, when compared to an established deterministic controller, it consistently falls short in terms of landing distance accuracy. To address this issue, the HYDESTOC Hybrid Deterministic-Stochastic (a combination of DDPG/deep deterministic policy gradient and PID/proportional-integral-derivative) algorithm was introduced to improve terminal distance accuracy while keeping propellant consumption low. Results from extensive cross-validated Monte Carlo simulations show that a miss distance of less than 0.02 meters, landing speed of less than 0.4 m/s, settling time of 20 seconds or fewer, and a constant crash-free performance is achievable using this method.

MeSH terms

  • Algorithms
  • Learning
  • Reinforcement, Psychology*
  • Reward
  • Spacecraft*

Grants and funding

The research and scholarship is funded by Direktorat Riset and Pengembangan, Universitas Indonesia/NKB-483/UN2.RST/HKP.05.00/2022/ - Dr. Sastra Kusuma Wijaya Lembaga Ilmu Pengetahuan Indonesia/SK Kepala LIPI No. 59/H/2020 - Mr. Larasmoyo Nugroho Kementerian Riset dan Teknologi /Badan Riset dan Inovasi Nasional/SK KaORPA No.15/III/HK/2022 - Mr. Larasmoyo Nugroho. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.