A PGAS Implementation by Co-design of the ECMWF Integrated Forecasting System (IFS)

Today the European Centre for Medium-Range Weather Forecasts (ECMWF) runs a 16 km global T1279 operational weather forecast model using 1,536 cores of an IBM Power7. Following the historical evolution in resolution upgrades, ECMWF could expect to be running a 2.5 km global forecast model by 2030 on an Exascale system that should be available and hopefully affordable by then. To achieve this would require IFS to run efficiently on about 1000 times the number of cores it uses today. This is a significant challenge, one that we are addressing within the CRESTA project. After implementing an initial set of improvements ECMWF has now demonstrated IFS running a 10 km global model efficiently on over 50,000 cores of HECToR, a Cray XE6 at EPCC, Edinburgh. Of course, getting to over a million cores remains a formidable challenge, and many scalability improvements have yet to be implemented. Within CRESTA, ECMWF is exploring the use of Fortran2008 coarrays; in particular it is possibly the first time that coarrays have been used in a world leading production application within the context of OpenMP parallel regions. The purpose of these optimizations is primarily to allow the overlap of computation and communication, and further, in the semi-Lagrangian advection scheme, to reduce the volume of data communicated. The importance of this research is such that if these developments are successful then the IFS model may continue to use the spectral method to 2030 and beyond on an Exascale sized system.