State Space Models with Dynamical and Sparse Variances, and Inference by EM Message Passing

Sparse Bayesian learning (SBL) is a probabilistic approach to estimation problems based on representing sparsitypromoting priors by Normals with Unknown Variances. This representation blends well with linear Gaussian state space models (SSMs). However, in classical SBL the unknown variances are a priori independent, which is not suited for modeling group sparse signals, or signals whose variances have structure. To model signals with, e.g., exponentially decaying or piecewiseconstant (in particular block-sparse) variances, we propose SSMs with dynamical and sparse variances (SSM-DSV). These are two-layer SSMs, where the bottom layer models physical signals, and the top layer models dynamical variances that are subject to abrupt changes. Inference and learning in these hierarchical models is performed with a message passing version of the expectation maximization (EM) algorithm, which is a special instance of the more general class of variational message passing algorithms. We validated the proposed model and estimation algorithm with two applications, using both simulated and real data. First, we implemented a block-outlier insensitive Kalman smoother by modeling the disturbance process with a SSM-DSV. Second, we used SSM-DSV to model the oculomotor system and employed EM-message passing for estimating neural controller signals from eye position data.