Multi-agent Deep Reinforcement Learning for Distributed Energy Management and Strategy Optimization of Microgrid Market

Abstract With the increasing demands for autonomous decision, market initiative, and privacy protection, a distributed energy management and strategy-making framework is more appropriate for the future microgrid. Besides, the integration of various distributed generations (DGs) and load facilities with high randomness poses challenges to traditional model-free microgrid market management approaches. Therefore, this paper concentrates on the distributed energy management and strategy optimization for a regional microgrid from the following aspects. First, a multi-agent reinforcement learning (MARL) framework is applied to generate independent real-time market decisions and to keep the benefits balance during agents’ interactive learning. Moreover, to solve the defects of traditional RL in the microgrid case of continuous state inputs, a multi-agent deep Q-network (MADQN) is adopted as the approximation of value function. Finally, an optimal equilibrium selection (OES) mechanism is proposed to calculate the collective update objective for promoting learning efficiency and to ensure benefits equilibrium. Combining theoretical analysis and simulation results, the proposed MARL framework can improve the microgrid operation performance from economy, independence, and benefit balance among microgrid participants.