Mastering the Game of Amazons Fast by Decoupling Network Learning

In this work, we propose a deep reinforcement learning (DRL) algorithm DoubleJump which can master the game of Amazons efficiently. To address the bottleneck problem of sparse supervision signal in DRL, we split the neural network into rule network and skill network, using huge amounts of inexpensive data with game rule information and scarce data containing game skill information to train two networks respectively. Besides, we split the three sub-actions of each action into independent states during Monte-Carlo tree search (MCTS), to improve the probability of finding the global optimal state and reduce the average branching factor. The experimental results show our algorithm reaches about 130:70 in the zero-knowledge learning compared with the AlphaGo Zero algorithm, significantly improves the learning speed, and then alleviates the severe dependence on computing resources.