Framework for control and deep reinforcement learning in traffic

Recent advances in deep reinforcement learning (RL) offer an opportunity to revisit complex traffic control problems at the level of vehicle dynamics, with the aim of learning locally optimal policies (with respect to the policy parameterization) for a variety of objectives such as matching a target velocity or minimizing fuel consumption. In this article, we present a framework called CISTAR (Customized Interface for SUMO, TraCI, and RLLab) that integrates the widely used traffic simulator SUMO with a standard deep reinforcement learning library RLLab. We create an interface allowing for easy customization of SUMO, allowing users to easily implement new controllers, heterogeneous experiments, and user-defined cost functions that depend on arbitrary state variables. We demonstrate the usage of CISTAR with several benchmark control and RL examples.