Simulation of a Daytime-Based Q-Learning Control Strategy for Environmental Harvesting WSN Nodes

Environmental wireless sensor networks (EWSN) are designed to collect environmental data in remote locations where maintenance options are limited. It is therefore essential for the system to make a good use of the available energy so as to operate efficiently. This paper describes a simulation framework of an EWSN node, which allows to simulate various configurations and parameters before implementing the control system in a physical hardware model, which was developed in our previous study. System operation, namely environmental data acquisition and subsequent data transmission to a network, is governed by a model-free Q-learning algorithm, which do not have any prior knowledge of its environment. Real-life historical meteorological data acquired in the years 2008–2012 in Canada was used to test the capabilities of the control algorithm. The results show that setting of the learning rate is crucial to EWSN node’s performance. When set improperly, the system tends to fail to operate by depleting its energy storage. One of the aspects to consider when improving the algorithm is to reduce the amount of wasted harvested energy. This could be done through tuning of the Q-learning reward signal.