Coordinating Multi-Agent Deep Reinforcement Learning in Wargame

The successful application of deep reinforcement learning in RTS games such as StarCraft II has inspired people to apply multi-agent deep reinforcement learning(MADRL) to more fields. In the field of wargame, hexagonal maps are often used for simulation, which can't adapt to the rapid development of wargame. In continuous space of wargame, we construct a ship-defense scenario that includes multiple aircraft and ships. We apply deep Q network(DQN) method to MADRL, CNN to extract the features of multiple entities, and a centralized and distributed decision-making training architecture to control the aircraft's fixed-wing module components. Experiment results demonstrate the effectiveness of the proposed formulation, which show that the CNN-based feature extraction model can effectively defeat the built-in rule bot with multiple levels, and the training effect of CNN-based is better than the feature extraction method by full connection.