Computationally-Efficient Energy Management in Buildings with Phase Change Materials using Approximate Dynamic Programming

This paper considers energy management in buildings with phase change material (PCM) that serves as a thermal energy storage system. In this setting, the optimal scheduling of an HVAC system is challenging because of the nonlinear characteristics of the PCM, which makes solving the corresponding optimization problem using conventional optimization techniques impractical. Instead, we propose a novel approximate dynamic programming (ADP) methodology to reduce the computational burden, while maintaining the quality of the solution. Specifically, the method incorporates multi-timescale Markov decision processes and a neural network function approximator of the state transition model, coupled with an underlying state-space approximation. The method is demonstrated on an energy management problem for a typical building in Sydney, Australia, over a year. The results demonstrate that the proposed method performs well with a computational speed-up of up to 157,600 times compared to the direct application of DP.