Efficient scheduling of streams on GPGPUs

Graphics processing units (GPUs) are widely used for scientific and engineering applications with high level of parallelism. The computing power of GPUs is improving through enhancing their architectural facilities. NVIDIA’s compute unified device architecture (CUDA) stream using hyper-Q on NVIDIA graphic cards is a reputable capability for performance improvement. Without any synchronization and based on architectural capabilities, NVIDIA’s CUDA stream allows some processes to run simultaneously. Experimental results show that how to stream a number of programs affect the execution time. Therefore, the stream set with the highest amount of performance improvement is the efficient stream set. This article proposes a framework to predict the efficient stream set on two streams without trying all combinations, which would be a very time-consuming process. The proposed framework employs a performance model and a scheduler. The performance model estimates the duration of simultaneous portions of streamed programs and the scheduler uses the estimation of the model to predict the efficient stream set. The proposed prediction method relies on non-stream features of programs. The results show that even with 33% error of performance model in average, the scheduler predicts the optimized sets with 100% precision.