Building an Autonomous Lane Keeping Simulator Using Real-World Data and End-to-End Learning

Autonomous lane keeping is an important safety feature for intelligent vehicles. This paper presents a lane keeping simulator that is built with image projections of recorded data in conjunction with vehicle dynamics estimation. An end-to-end learning method using convolutional neural network (CNN) takes front-view camera data as input and produces the proper steering wheel angle to keep the vehicle in lane. A novel method of data augmentation is proposed using vehicle dynamic model and vehicle trajectory tracking, which can create additional training data as if a vehicle drives off-lane in any displacement and orientation. Experimental results demonstrate that the CNN model trained with the simulator can achieve higher accuracy for autonomous lane keeping and much lower failure rate. The simulator can serve as a platform for both training and evaluation of vision based autonomous driving algorithms. The experimental dataset is made available at http://computing.wpi.edu/dataset.html .