Q-Learning Approach to Coordinated Optimization of Passenger Inflow Control with Train Skip-stopping on a Urban Rail Transit Line

Abstract In the case of an over-crowded urban rail transit (URT) line, a large number of passengers may be left stranded and daily timetable may become infeasible. This paper proposes a coordinated optimization scheme for a URT line, which combines both the coordinated passenger inflow control with train rescheduling strategies. With the aim of minimizing the penalty value of passengers being stranded along the whole line, the coordinated passenger inflow control helps relieve demand pressure and ensure safety at over-crowded URT stations while the train rescheduling of skip-stopping helps to balance the utilization of train capacity. A novel Q-learning based approach to this combination optimization problem is developed. Simulation experiments are carried out on a real-world URT line in Shanghai. Basic principles of Q-learning are presented, which consist of the environment and its states, learning agents and their respective actions, and rewards. The results show that the coordinated optimization scheme solved by the Q-learning approach is effective in relieving the passenger congestion on the URT line. The Q-learning approach can offer accurate scheme to deal with the problem of passenger congestion and train operation on a URT line.