ParGAL: A Scalable Grid-Aware Analysis Library for Ultra Large Datasets.

Many fields that employ computation require extensive analysis of the output from a petascale simulation of a grid(or mesh)-based application in order to complete their scientific goals or produce a visual image or animation. Often this analysis cannot be done in-situ because it requires calculating time-series statistics from state sampled over the entire length of the run or analyzing the relationship between similar time series from previous simulations or observations. The programs that perform this analysis are not nearly as flexible in their choice of grids or as high-performing as the primary applications. We will describe a new Parallel Gridded Analysis Library (ParGAL) that performs data-parallel versions of several common analysis algorithms on data from a structured or unstructured grid simulation. The library builds on several scalable systems starting with the Mesh Oriented DataBase (MOAB). MOAB is a library for representing mesh data that supports structured, unstructured finite element and polyhedral grids and also supports parallel operations on those grids including loading to and from disk using parallel I/O. We are using the Parallel-NetCDF (PNetCDF) library to perform parallel I/O operations between the popular NetCDF format and ParGAL. Finally, we also make use of Intrepid, an extensible library for computing operators (such as gradient, curl, divergence, etc.) acting on discretized fields. The design and performance of ParGAL will be described and an example of its application to climate compared to a widely used tool is given.