Evaluating Stencil Codes at Scale

Stencil-based codes are widely used in Scientific Computing and are considered to be good candidates for running at scale as their communication depends on their local neighbours and is therefore independent of the number of parallel tasks. A 2-dimensional stencil kernel has been written to determine whether stencil-based codes do indeed scale on todays supercomputers. This kernel is able to perform halo communication that represents the communication a stencil code would require, for a range of data sizes and parallel tasks. It is shown that this kernel does not scale well on an IBM BGQ (Blue Joule, 64K cores), a Cray XK7 (Titan, 16K cores) and a Cray XC30 (ARCHER, 16K cores) even if the suggested topology mapping tools are used. In weak scaling tests, performance degradations of up to x26, x34 and x28 respectively, are observed. A new task-to-topology mapping scheme is presented for a torus network which maximises intra-node communication and minimises network contention. Weak scaling performance results on the IBM BGQ show that when using this scheme there is no performance degradation for all data sizes and number of parallel tasks. The resultant time savings give a reduction in energy consumption of up to 64%.