Developing a Merge Lane Change Decision Policy for Autonomous Vehicles by Deep Reinforcement Learning

With autonomous vehicles (AVs) being actively developed, it becomes possible to optimize vehicle control policies and traffic management tools in a mixed vehicular environment. For individual AV control, acceleration and lane change are the two elementary driving behaviors that need to be coordinated to minimize disturbance to the entire traffic dynamics. In this paper, a joint decision policy of acceleration and lane change actions for AVs on a merging ramp is proposed and trained in a mixed autonomy traffic, using the technique of deep reinforcement learning. Our method is able to train policies that have limited impact on highway traffic while maintaining a relatively high merge throughput. We experimented with two reward functions, designed for the AV's selfish benefits and for the network traffic's social benefits. This paper then examines the emergent behaviors exhibited by the trained policies and their impacts on the main highway traffic at different density levels.