Adaptive signal control for bus service reliability with connected vehicle technology via reinforcement learning

Abstract This paper presents an adaptive signal controller for managing traffic delays and urban bus service reliability with fully adaptable acyclic timing plans. The signal controller is built upon a reinforcement learning framework that consists of a model-based and a data-driven component. The model-based component is represented by a hybrid kinematic wave traffic model that integrates macroscopic flow-based and microscopic vehicle-based state variables subject to stochastic demands and bus service status. To cope with the high dimensional solution space, the data-driven component is incorporated as a multi-layer artificial neural network and is used to approximate future traffic states and system performances with respect to prevailing control settings. Before the controller can be used, the neural network is to be trained through a series of realised dynamic state transitions via an on-policy temporal difference learning algorithm. The proposed control framework is tested over a real world corridor scenario in London, UK. The proposed controller is able to reduce both traffic delays and bus service variabilities subject to stochastic demands with acyclic timing plans that can be derived in short computational time. This study contributes to the design of adaptive network traffic control for multi-modal networks with connected vehicle technology and advanced learning-based optimisation techniques.