Switching and information exchange in compressed estimation of coupled high dimensional processes

Abstract Compressed Estimation approaches, such as the Generalised Compressed Kalman Filter (GCKF), reduce the computational cost and complexity of high-dimensional and high-frequency data assimilation problems, usually without sacrificing optimality. Configured using adequate cores, such as the Unscented Kalman Filter (UKF), the GCKF could also treat certain high-dimensional non-linear cases. However, the application of a compressed estimation process is limited to a class of problems which inherently allow the estimation process to be divided, at certain intervals of time, into a set of lower-dimensional problems. This limitation prohibits applying the compressing techniques for estimating coupled high-dimensional processes. However, those limitations can be overcome by applying proper techniques. In this paper, the concepts of subsystem switching and information exchange architecture, namely ‘Exploiting Local Statistical Dependency’ (ELSD), have been derived and explored, allowing compressed estimators to mimic optimal full-Gaussian estimators. The performances of the methods have been verified through applications in solving usual types of Stochastic Partial Differential Equations (SPDEs). The computational advantages of using the proposed techniques have also been highlighted with a recommendation for its usage over the full filter when dealing with high-dimensional and high-frequency data assimilation.