Intelligent Action Selection for NGSO Networks with Interference Constraints: A Modified Q-Learning Approach

With the rapid development of non-geostationary orbit (NGSO) constellation networks, the co-frequency interference between geostationary orbit (GSO) and NGSO systems is becoming increasingly inevitable. Due to the characteristics of complex multi-mission planning and the large spatial and temporal scale of NGSO constellations, it is difficult to maximize the system performance under the constraints of GSO networks' interference protections. In this paper, based on an iterative greedy algorithm, a modified Q-learning approach in reinforcement learning (RL) is proposed to upgrade the system performance of NGSO network. Compared with the traditional RL algorithms, this modified approach has the advantages of improving the system performance and meeting the aggregate interference protections of GSO networks, by redefining the actions, rewards, and Q-tables. Simulation results show that by adopting the proposed algorithm, all NGSO links timely select and flexibly adapt their actions, including access objects, transmission powers, communication rates, as well as modulation and coding schemes, to achieve more global rewards with co-frequency interference constraints.