Transfer learning for operational planning of batteries in commercial buildings

Recently, building owners are investing in rooftop photovoltaic (PV) installations and batteries in order to meet the (facility) load in their buildings. As a consequence, several commercial and research solutions have emerged for battery energy management in such buildings. Most of these solutions rely on sufficiently accurate system models and are tailor-made for those systems. This work proposes the use of transfer learning in model-free reinforcement learning (RL) to control the operation of batteries in buildings. This enables knowledge from the control of a battery in one building to be used by a RL algorithm to control a battery in another building with similar characteristics. In this paper, the K-shape clustering algorithm is used to group buildings with similar characteristics - based on their energy consumption patterns. To plan the operation of the batteries, we use fitted Q-iteration, a RL algorithm. Simulation results using real-world data show that by including forecast information on energy consumption and PV generation in the feature space of the control algorithm, RL competes with mixed integer linear programming - which assumes perfect knowledge of the system. We also investigate through simulation, the effect of transferring a policy learned with data from one building to another building - all buildings belonging to the same cluster. Simulation results show a faster convergence - convergence achieved with fewer training samples required - to a near optimal policy.