ExAscale Pre- and Post-ProcessingStrategy and Software Technology
2014
Today’s large-scale simulations deal with complex geometries and numerical data on an extreme
scale. As computation approaches the exascale, it will no longer be possible to write and store
full-sized result data sets. In-situ data analysis and scientific visualisation provide feasible
solutions to the analysis of complex large-scale simulations. To bring pre- and post-processing to
the exascale we must consider modifications to data structure and memory layout, and address
latency and error resiliency.
Load balancing is a crucial pre-processing task on extremely parallel systems. Here, our focus is on a
load balancing strategy that supports multiple simulation phases and includes their costs to calculate
a data distribution that leads to an optimal performance for the full simulation.
The software library PPStee developed in CRESTA already incorporates this idea. It was explicitly
designed to support multiple simulation phases. The user provides communication costs of a
simulation phase represented as edge weights of a graph corresponding to the simulation data. The
according computation costs of the phase are matched to vertex weights. As multiple weight sets can
be included, the partitioning of the simulation data is calculated to achieve an optimal load balance
covering the full simulation cycle. PPStee supports various partitioning tools, repartitioning and
improved initial data distributions by exploiting knowledge from previous simulation runs. PPStee
was successfully integrated into several large-scale fluid simulation codes. Here, we demonstrate the
flexibility of PPStee with the hemodynamic simulation code HemeLB.
In-situ processing has become a key concept in exascale data post-processing and visualisation.
Waiting for a simulation to finish and writing out huge amounts of simulation output is no longer a
viable solution for data analysis. Instead, visualisation and data analysis must happen when and where
a certain simulation step has been carried out, as the so-called in-situ processing.
Our in-situ processing system provides scalable distributed post-processing. This system supports
on-the-fly data analysis and user interaction to on-going simulations. Here, we demonstrate the
feasibility of our system by an online-monitoring scenario with the hemodynamic simulation code
HemeLB.
Remote hybrid rendering (RHR) is used to access remote exascale simulations from immersive
projection environments over the Internet. The display system may range from a desktop computer
to an immersive virtual environment such as a CAVE. The display system forwards user input to the
visualisation cluster, which uses highly scalable methods to render images of the post-processed
simulation data and returns them to the display system. The display system enriches these with
context information rendered locally, before they are shown. RHR decouples local interaction from
remote rendering and thus guarantees smooth interactivity during exploration of large remote data
sets.
Here, we discuss strategies, algorithms and techniques for RHR in exascale scenarios and present
performance measurements for a prototype developed in CRESTA. For performance analysis, the
prototype has been instrumented to collect timing information, compression ratios and image quality
metrics.
Keywords:
- Correction
- Source
- Cite
- Save
- Machine Reading By IdeaReader
0
References
0
Citations
NaN
KQI