DWPE, a new data center energy-efficiency metric bridging the gap between infrastructure and workload

To determine whether a High-Performance Computing (HPC) data center is energy efficient, various aspects have to be taken into account: the data center's power distribution and cooling infrastructure, the HPC system itself, the influence of the system management software, and the HPC workloads; all can contribute to the overall energy efficiency of the data center. Currently, two well-established metrics are used to determine energy efficiency for HPC data centers and systems: Power Usage Effectiveness (PUE) and FLOPS per Watt (as defined by the Green500 in their ranking list). PUE evaluates the overhead for running a data center and FLOPS per Watt characterizes the energy efficiency of a system running the High-Performance Linpack (HPL) benchmark, i.e. floating point operations per second achieved with 1 watt of electrical power. Unfortunately, under closer examination even the combination of both metrics does not characterize the overall energy efficiency of a HPC data center. First, HPL does not constitute a representative workload for most of today's HPC applications and the rev 0.9 Green500 run rules for power measurements allows for excluding subsystems (e.g. networking, storage, cooling). Second, even a combination of PUE with FLOPS per Watt metric neglects that the total energy efficiency of a system can vary with the characteristics of the data center in which it is operated. This is due to different cooling technologies implemented in HPC systems and the difference in costs incurred by the data center removing the heat using these technologies. To address these issues, this paper introduces the metrics system PUE (sPUE) and Data center Workload Power Efficiency (DWPE). sPUE calculates the overhead for operating a given system in a certain data center. DWPE is then calculated by determining the energy efficiency of a specific workload and dividing it by the sPUE. DWPE can then be used to define the energy efficiency of running a given workload on a specific HPC system in a specific data center and is currently the only fully-integrated metric suitable for rating an HPC data center's energy efficiency. In addition, DWPE allows for predicting the energy efficiency of different HPC systems in existing HPC data centers, thus making it an ideal approach for guiding HPC system procurement. This paper concludes with a demonstration of the application of DWPE using a set of representative HPC workloads.