BOPS, Not FLOPS! A New Metric and Roofline Performance Model For Datacenter Computing

The past decades witness FLOPS (Floating-point Operations per Second) as an important computation-centric performance metric. However, for datacenter (in short, DC) computing workloads, such as Internet services or big data analytics, previous work reports that they have extremely low floating point operation intensity, and the average FLOPS efficiency is only 0.1%, while the average IPC is 1.3 (the theoretic IPC is 4 on the Intel Xeon E5600 platform). Furthermore, we reveal that the traditional FLOPS based Roofline performance model is not suitable for modern DC workloads, and gives misleading information for system optimization. These observations imply that FLOPS is inappropriate for evaluating DC computer systems. To address the above issue, we propose a new computation-centric metric BOPs (Basic OPerations) that measures the efficient work defined by the source code, includes floating-point operations and the arithmetic, logical, comparing, and array addressing parts of integer operations. We define BOPS as the average number of BOPs per second, and propose replacing FLOPS with BOPS to measure DC computer systems. On the basis of BOPS, we propose a new Roofline performance model for DC computing, which we call DC-Roofline model, with which we optimize DC workloads with the improvement varying from 119% to 325%.