Reinforcement Learning Based Penetration Testing of a Microgrid Control Algorithm

Microgrids (MGs) are small-scale power systems which interconnect distributed energy resources and loads within clearly defined regions. However, the digital infrastructure used in an MG to relay sensory information and perform control commands can potentially be compromised due to a cyberattack from a capable adversary. An MG operator is interested in knowing the inherent vulnerabilities in their system and should regularly perform Penetration Testing (PT) activities to prepare for such an event. PT generally involves looking for defensive coverage blind spots in software and hardware infrastructure, however the logic in control algorithms which act upon sensory information should also be considered in PT activities. This paper demonstrates a case study of PT for an MG control algorithm by using Reinforcement Learning (RL) to uncover malicious input which compromises the effectiveness of the controller. Through trial-and-error episodic interactions with a simulated MG, we train an RL agent to find malicious input which reduces the effectiveness of the MG controller.