Centralized Traffic Signal Control for Multiple Intersections based on Sequence-to-Sequence model and Attention Mechanism

Recently, the use of deep reinforcement learning techniques (DRL) has attracted increasing interest due to its ability of dynamical traffic signal control for multiple intersections. Only a few researches use the centralized control with single-agent to intelligently control all the signals, because the major problem, i.e., the curse of dimensionality, has not been successfully solved. We propose a novel centralized control method based on sequence-to-sequence model and attention mechanism to deal with this problem. The idea is similar to the Divide and Conquer paradigm. We mitigate the difficulty of searching in the huge space by dividing the state and action space into sub-spaces through sequence-to-sequence model. In addition, we greatly facilitate the communication and cooperation of traffic signals among intersections by introducing the attention mechanism. The DRL agent is trained by an efficient off-policy learning method - Proximal Policy Optimization. To the best of our knowledge, we are the first to use sequence-to-sequence method to deal with the huge search space problem in the traffic control. The comprehensive experiments demonstrate that our method can efficiently solve the curse of dimensionalitly problem and outperforms the traditional methods and other centralized control methods based on DRL.