Distributed Online Dispatch For Microgrids Using Hierarchical Reinforcement Learning Embedded With Operation Knowledge

This paper considers the problem of distributed online economic dispatch (DOED) from sequential data using reinforcement learning. Learning operation behavior in high-dimension environments with constraints is a major challenge for the DOED of networked microgrids (MGs), where insufficient exploration disables agents to build complex policies. Therefore, this paper develops a hierarchical reinforcement learning (HRL) to handle the DOED problem, where radial basis function (RBF) approximation is incorporated to make policies in continuous space. Based on the hierarchical framework, the HRL algorithm increases the training efficiency and reduces computational cost. The online HRL achieves distributed selfadaption and better performance of real-time dispatch by a modest number of interacting variables. In addition, guided by domain knowledge, the HRL algorithm avoids onerous additional learning beyond feasible action space. In the case of an actual networked MGcluster in Qingdao with real operation data, simulation is conducted to verify that the proposed learning framework can reduce longterm operation costs and enhance operation stability. To explore the learning process, we also provide its convergence condition andanalyze the sensitivity of the learning parameters.