Reinforcement Learning Based on Energy Management Strategy for HEVs

This paper presents a new architecture of real-time HEV’s energy management problem under a V2V and V2I environment using policy-based deep reinforcement learning. The ideal energy management controller that minimizes HEV energy costs needs to run engines most efficiently in the whole running considering battery SoC. The controller needs to predict the future vehicle speed and plan the power distribution to achieve it because the thermal efficiency of engines is more efficient when its rotational speed is higher. The future vehicle speed has relationship with connectivity information such as the behavior of the car in front, the traffic light signals, crowd of cars, and so on. This paper assumes the connectivity environment in the future and applies proximal policy optimization (PPO) [5] that is known as policy-based deep reinforcement learning algorithm to achieve the optimal power distribution predicting the future behavior by using connectivity information. In addition, this paper shows that locating the local controller in the reinforcement learning loop enables the AI controller to learn robustly. The local controller corrects against an exploration that is obviously not optimal or doesn’t satisfy the constraints.