Handling large-scale action space in deep Q network

Deep reinforcement learning (DRL) is a new topic in recent years. Deep Q Network is a popular DRL implement. It is a well-studied technique and has achieved significant improvement on several challenging tasks such as Atari 2600 games. However, in some kinds of games, there are a large number of possible actions. Thus the output layer in DQN could be complicated due to the large-scale action space, which could harm the performance of DQN. In this paper, we proposed a variant structure of DQN to handle this problem. We could reduce the size of output layer in DQN. The experimental results show that our method improves significantly in some tasks with large-scale action space.