STARE Companion Files for NASA Earth Science Data

SpatioTemporal Adaptive Resolution Encoding (STARE) is an integer encoding scheme for spatial and temporal coordinates and volumes. The spatial element of STARE encodes the traversal through a recursive partitioning (quad-furcation) of spherical triangles (or trixels) on the unit sphere to index discrete solid angles. In addition, the index contains neighborhood information. The (approximate) resolution of a model grid cell or a satellite image pixel can thus be indicated together with its geolocation in one 64-bit integer number. The temporal element of STARE has a similar design with partitioning based on calendrical units rather than uniform quadfurcation. As such, pairs of STARE spatiotemporal indices index spatiotemporal volume intervals of various sizes. Therefore, not only can STARE optimize search and subsetting performance for geo-spatiotemporal data, but also facilitate spatiotemporal data placement alignment on distributed parallel computing resources to maximize scalability by minimizing costly unnecessary communication among these resources. In this paper, we focus our discussion on the application of STARE spatial encoding to NASA satellite swath data. This is accomplished by generating STARE companion or “sidecar” files, one file per data granule, which contain the STARE indices for the geolocations, together with approximate resolutions, of the data in the granule. The STARE sidecar files are stored as netCDF-4/HDF5 files. In the design of file metadata, we strive to ensure compatibility with the Climate and Forecast (CF) conventions, widely used in the geo-science community. Specifically, in accordance with the practices of CF conventions, variables are not identified by their names, but by including special attributes. This allows variable names to vary, which they invariably do, while ensuring that reading programs can properly parse the sidecar file.