Gpgpu multitasking and scheduling

The set-top and portable device market continues to grow, as does the demand for improved performance, cost, and power. The integration of Graphics Processing Units (GPUs) into these devices and the emergence of general-purpose computations on graphics hardware enable a new set of highly parallel applications. To facilitate these applications, I propose and make the case for a novel GPU multitasking technique called spatial multitasking. Traditional GPU multitasking techniques, such as cooperative and preemptive multitasking, partition GPU time among applications, while spatial multitasking allows GPU resources to be partitioned among multiple applications simultaneously. I demonstrate the potential benefits of spatial multitasking with an analysis and characterization of General-Purpose GPU (GPGPU) applications. My analysis indicates that many GPGPU applications fail to utilize available GPU resources fully, which suggests the potential for significant performance benefits using spatial multitasking instead of, or in combination with, preemptive or cooperative multitasking. I use simulation to evaluate a simple implementation of spatial multitasking against cooperative multitasking. This implementation of spatial multitasking shows an average speedup of 1.14, 1.22, and 1.30 compared to cooperative multitasking when two, three, and four applications share the GPU, respectively. I follow this with an evaluation of more complex resource partitioning heuristics for spatial multitasking and show even greater speedup of spatial multitasking over cooperative multitasking. Finally, I explore providing Quality-of-Service to GPGPU applications using spatial multitasking and show significant benefits compared to Quality-of-Service using temporal multitasking.