Development of benchmark automation suite and evaluation of various high-performance computing systems

This study aimed to develop a dynamic benchmark automation suite to measure a range of benchmark performances and evaluate the various high-performance computing (HPC) systems. Our suite supports an automated scaling test and profiling data based on hardware performance counters to analyze the system characteristics. We selected four HPC benchmarks—STREAM, High-Performance Linpack, High-Performance Conjugate Gradient, and NAS Parallel Benchmark-for experiments and configured testbeds based on five different systems and an Intel Knights Landing (KNL) cluster with 16 nodes. The Intel KNL system showed both unstable memory and high benchmark performances for a specific input range. Modern Intel systems also exhibited proper characteristics on compute-intensive workloads, whereas the up-to-date AMD system showed high efficiency and proper characteristics on memory-intensive and real application workloads. We also verified that each system has an optimal environment and characteristic for various combinations of experimental variables and profiling data.