An Experimental Study on Deep Learning Based on Different Hardware Configurations

Deep learning has exhibited high accuracy and applicability in machine learning field recently, by consuming tremendous computational resources processing massive data. To improve the performance of deep learning, GPUs have been introduced to accelerate the training phase. The complex data processing infrastructure demands high-efficient collaboration among underlying hardware components, such as CPU, GPU, memory, and storage devices. Unfortunately, few work has presented a systematic analysis about the impact of hardware configurations on the overall performance of deep learning. In this paper, we aim to make an experimental study on a standalone system to evaluate how various hardware configurations affect the overall performance of deep learning. We conducted a series of experiments using varied configurations on storage devices, main memory, CPU, and GPU to observe the overall performance quantitatively. Based on analyzing these results, we found that the performance greatly relies on the hardware configurations. Specifically, the computation is still the primary bottleneck as double GPUs and triple GPUs shorten the execution time by 44\% and 59\% respectively. Besides, both CPU frequency and storage subsystem can significantly affect running time while the memory size has no obvious effect on the running time for training neural network models. We believe our experimental results can help shed light on further optimizing the performance of deep learning in computer systems.