An introduction to multi-GPU programming for physicists

We present and compare different approaches for using multiple Graphics Processing Units in the simulation of physical systems. As benchmarks we consider the time required to update a single spin of the 3D Heisenberg spin glass model, by using both the Over-relaxation and the Heat Bath algorithms, and the solution of a Poisson equation by using a finite-difference method. The results show that a suitable combination of techniques allows to hide almost completely the communication overhead by using the CPU as a communication coprocessor of the GPU. Large scale simulations on clusters of GPUs can be efficiently carried out by following the same approach for other applications where a clear cut exists between bulk and boundaries data.