Electric Water Heaters Management via Reinforcement Learning with Time-Delay in Isolated Microgrids

Isolated microgrids powered by renewable energy sources, battery storage, and backup diesel generators need appropriate demand response to utilize available energy and reduce diesel consumption efficiently. However, real-time demand-side management has become a significant challenge due to the communication time-delay issue. In this paper, a distributed model-free strategy is proposed to manage the demand of Electric Water Heater (EWH) units. The distributed artificial intelligence technology based on Reinforcement Learning (RL) is adopted to independently control the 150 EWHs using a virtual tariff. Two different strategies are proposed to generate the virtual tariff and they are compared to each other to investigate the impact of communication time-delay to the proposed RL algorithm in real-time control scenario. The first strategy is based on measuring the battery State of Charge (SOC) in real time while the second method is based on predicting the SOC 24-hours in advance using an Artificial Neural Network (ANN). The results show that the communication time-delay greatly influences the convergence result of the first method while the second method showed high immunity. The results also show that the proposed algorithm reduces the use of energy consumption by an average of 8.91%(6.675kW) for each EWH, which symbolizes the viability of the proposed approach.