CSSMT: Compiler Based Software Simultaneous Multithreading (SMT)

Simultaneous multithreading (SMT) is a unique computer architecture feature to increase the pipeline utilization and therefore, increase the instruction throughput. It improves instruction level throughput by simultaneously filling both vertical and horizontal super-scalar pipeline slots that are left unfilled by native threads. So far SMT has been implemented in hardware. However, hardware SMT implementations have its limitations. First, it is complex and expensive to implement—only higher-end processors are equipped with it even though lower-end processors have same pipeline design as the higher-end variants and might also benefit from it. SMT also introduces great power/energy and area overheads. Moreover, being a hardware feature, SMT is limited by the range and depth of instruction analysis that it can afford at execution time, therefore it is unlikely to benefit from high-level software knowledge about instruction mix and might lose many improvement opportunities. In this paper, we address the limitation of the hardware-based SMT and introduce CSSMT: Compiler based Software Simultaneous Multithreading (SMT). The main contribution of CSSMT is that it exploits high- level program profiles to purposefully "re-mix" instructions from multiple programs to better fill vertical and horizontal super- scalar pipeline slots so that the overall throughput is improved. Furthermore, CSSMT is a software-transformation technique that enables SMT at software level during compilation time. Therefore, it can help overcome the limitation of the hardware-based SMT implementation and is more portable. We test CSSMT with programs from SPEC2006 and NAS benchmarks and achieve up to 12% speedup of execution time (30.7% improvement in terms of multi-program throughput).