Locally Low-Rank Regularized Video Stabilization with Motion Diversity Constraints

This paper presents a novel motion aware regularization model with diversity constraints for motion smoothing in video stabilization. Differing from the global path optimization methods, the proposed model puts an emphasis on the relations of inter-frame motions and incorporates sliding windowed low-rank and smoothness constraints. The rationale behind it is cinematography rules which assume that camera motions can be divided into diverse patterns: zero velocity, constant velocity, and acceleration motion. Firstly, a locally motion aware fidelity term is adopted in light of the local motion stationarity. Secondly, to improve the robustness of the model for different motion patterns, a locally low-rank constrained regularization term is further introduced by considering the motion correlation in a local temporal window. Moreover, to cope with the over-smoothing problem in rapid motion situations with extreme acceleration, a motion steering kernel and varying window length are employed to enhance the flexibility of the proposed model. The experimental results demonstrate the superiority of the proposed optimization model and the efficiency to suppress over-smoothing when rapid motions occur. Meanwhile, we compare the results with some state-of-the-art methods quantitatively and qualitatively, and our method can achieve a comparable or even better stabilization effect.