The Optimized Reinforcement Learning Approach to Run-Time Scheduling in Data Center

Warehouse data center is very large scale and complex, which constains tens of thousands servers and accomodates various applications. What’s more important, energy consumption has risen to a critical point. Scheduling needs to maintain performance and reduce energy consumption as much as possible. Previous researches have proposed RL (reinforcement learning) as a solution. These approaches have reduced energy consumption in IT equipment to the extent of “improved operation” while maintain SLA (Service Level Agreement) and throughput. However, with consideration of the special condition of data center, more optimization of RL could be discovered. We find through careful optimization of RL, we could save much more energy in IT equipment (about 12%) while maintain performance. When we consider 110 billion kWh/year energy consumption in 2011, 12% saving of energy in IT equipment would make a dramatic difference in operation cost or even feasibility. Our approach sets up ideal state which would maintain performance and reduce energy consumption. When some machines deviate from that ideal state, the schedulers in the data center compute the gains and costs of VM migrations to determine whether operations such as consolidation or ban lancing would be carried. In this way, we apply long range value estimation, exploration elimination, combining of planning and learning to optimize RL. The simulation of real internet workload trace shows our design saves much more energy in IT equipment and maintains performance with much less VM migration.