Air combat autonomous maneuver decision for one-on-one within visual range engagement base on robust multi-agent reinforcement learning

Based on a robust multi-agent reinforcement learning (MARL) algorithm framework, an autonomous maneuver decision-making algorithm for UCAV air combat in one-on-one combat in the visible range is designed and implemented. This algorithm can solve the problem that the single agent reinforcement learning algorithm cannot converge during the training process due to the unstable environment. At the same time, considering the shortcomings of the MADDPG algorithm in a strong competitive environment, it is easy to obtain a very fragile strategy, which is only targeted at a specific equilibrium strategy. In this paper, a minimax module is introduced to obtain the expected perturbation, which can locally approach the worst-case perturbation through the gradient. Through simulation tests of algorithm convergence and policy quality, the algorithm is found to be effective.