Deep reinforcement learning–based approach for optimizing energy conversion in integrated electrical and heating system with renewable energy

Abstract With advanced information technologies applied in integrated energy systems (IESs), controlling the energy conversion has become an effective method for improving grid flexibility and reducing the operating cost of IESs. This study proposes a dynamic energy conversion strategy for the energy management of an IES with renewable energy, which considers the system operator’s (SO) operating cost. Deep reinforcement learning (DRL) is used to illustrate the hierarchical decision-making process, in which the dynamic energy conversion problem is formulated as a discrete finite Markov decision process, and proximal policy optimization (PPO) is adopted to solve the decision-making problem. Using DRL, the SO can adaptively decide the wind power conversion ratio during the online learning process, where the uncertainties of customers’ load demand profiles, flexibility of spot electricity prices, and wind power generation are addressed. Simulations show that the proposed PPO-based renewable energy conversion algorithm can effectively reduce the SO’s operating cost.