Lossy compression of Earth system model data based on a hierarchical tensor with Adaptive-HGFDR (v1.0)
2021
Abstract. Lossy compression has been applied to the data compression of
large-scale Earth system model data (ESMD) due to its advantages of a high
compression ratio. However, few lossy compression methods consider both
global and local multidimensional coupling correlations, which could lead to
information loss in data approximation of lossy compression. Here, an
adaptive lossy compression method, adaptive hierarchical geospatial field data representation (Adaptive-HGFDR), is developed based on the
foundation of a stream compression method for geospatial data called blocked
hierarchical geospatial field data representation (Blocked-HGFDR). In addition, the
original Blocked-HGFDR method is also improved from the following perspectives.
Firstly, the original data are divided into a series of data blocks of a
more balanced size to reduce the effect of the dimensional unbalance of
ESMD. Following this, based on the mathematical relationship between the compression
parameter and compression error in Blocked-HGFDR, the control mechanism is
developed to determine the optimal compression parameter for the given
compression error. By assigning each data block an independent compression
parameter, Adaptive-HGFDR can capture the local variation of
multidimensional coupling correlations to improve the approximation
accuracy. Experiments are carried out based on the Community Earth System
Model (CESM) data. The results show that our method has higher compression
ratio and more uniform error distributions compared with ZFP and
Blocked-HGFDR. For the compression results among 22 climate variables,
Adaptive-HGFDR can achieve good compression performances for most flux
variables with significant spatiotemporal heterogeneity and fast changing rate.
This study provides a new potential method for the lossy compression of the
large-scale Earth system model data.
Keywords:
- Correction
- Source
- Cite
- Save
- Machine Reading By IdeaReader
34
References
1
Citations
NaN
KQI