Whole Building Energy Model for HVAC Optimal Control: A Practical Framework based on Deep Reinforcement Learning

Abstract Whole building energy model (BEM) is a physics-based modeling method for building energy simulation. It has been widely used in the building industry for code compliance, building design optimization, retrofit analysis, and other uses. Recent research also indicates its strong potential for the control of heating, ventilation and air-conditioning (HVAC) systems. However, its high-order nature and slow computational speed limit its practical application in real-time HVAC optimal control. Therefore, this study proposes a practical control framework (named BEM-DRL) that is based on deep reinforcement learning. The framework is implemented and assessed in a novel radiant heating system in an existing office building as a case study. The complete implementation process is presented in this study, including: building energy modeling for the novel heating system, multi-objective BEM calibration using the Bayesian method and the Genetic Algorithm, deep reinforcement learning training and simulation results evaluation, and control deployment. By analyzing the real-life control deployment data, it is found that BEM-DRL achieves 16.7% heating demand reduction with more than 95% probability compared to the old rule-based control. However, the framework still faces the practical challenges including building energy modeling of novel HVAC systems and multi-objective model calibration. Systematic study is also needed for the design of deep reinforcement learning training to provide a guideline for practitioners.