Deep reinforcement learning based energy management strategy of fuel cell hybrid railway vehicles considering fuel cell aging

Abstract In the rail transportation industry, growing energy and environmental awareness requires the use of alternatives to combustion engines. These include hybrid electrically driven railway vehicles powered by fuel cells and batteries. The cost of hydrogen consumption and the lifetime of fuel cells are currently the main challenges that need to be addressed before widespread deployment of fuel cell railway vehicles can be realized. With this in mind, this work focuses on the energy management system with emphasis on optimizing the energy distribution to reduce the overall operational cost. The presented energy management strategy (EMS) aims at minimizing hydrogen consumption and fuel cell aging costs while achieving a favorable balance between battery charging and discharging. In order to take fuel cell aging into account in energy management and mitigate fuel cell aging trough power distribution, an online fuel cell aging estimation model based on four operation modes is introduced and applied. Moreover, the advanced deep reinforcement learning method Twin Delayed Deep Deterministic Policy Gradient (TD3) is used to obtain a promising EMS. To improve the adaptability of the strategy, a stochastic training environment, which is based on real measured speed profiles considering passenger numbers is used for training. Assuming different environmental and passenger transport volumes, the results confirm that the proposed TD3-EMS achieves battery charge-sustaining at low hydrogen consumption while slowing down fuel cell degradation.