How to Train your ITS? Integrating Machine Learning with Vehicular Network Simulation

Machine Learning (ML) is becoming ever more popular in many application domains, including vehicular networking. It has been shown already that Intelligent Transportation Systems (ITS) can greatly benefit from this approach, particularly from Reinforcement Learning (RL). To implement Vehicular Ad-hoc Network (VANET) environments for RL training, researchers often start from scratch. Because up until now, there is neither an established interface to ML toolkits nor a common scenario for VANET applications. Though such established standards would be a great benefit to research: Previous results would be easier to reproduce and different solutions could be compared in equal situations and using the same metrics. We developed Veins-Gym to bridge this gap. Veins-Gym combines the popular Veins vehicular networking simulator with OpenAI Gym. Using an exemplary VANET application, we show that RL techniques can be easily applied to ITSs with this framework. This enabled us to train an agent that outperformed hand-written algorithms.