A Deep Reinforcement Learning Based Energy Management Strategy for Hybrid Electric Vehicles in Connected Traffic Environment

Abstract This paper proposed a deep reinforcement learning based energy management strategy in the connected traffic environment. At the upper layer, a deep deterministic policy gradient algorithm is used to systematically integrate ego vehicle reference speed planning into the energy management strategy in the car following scenarios. At the bottom layer, a driver model and an adaptive equivalent consumption minimization strategy are implemented to conduct the optimal power split control based on the planned reference speed. As distance headway, fuel consumption, and terrain information are taken into consideration by the deep reinforcement learning controller, the proposed strategy can not only maintain a safe distance headway but also improve the energy efficiency of the entire driving cycle. Finally, the fuel consumption score of the proposed strategy is reduced by 3.5% compared with a car following strategy based on a proportional-integral controller. The simulation results show that the proposed strategy can learn a reasonable car following policy through the training process. In addition, the simulation duration for each reference speed planning step is only about 4.1 ms, which proves the proposed strategy can be applied online.