Robust panorama from MPEG video

A novel approach to image mosaicking from MPEG video is presented in this paper. The motion vectors in both P- and B-frames are used for global motion estimation. The bi-directional information in B-frames provides multiple routes to warp a frame to its previous anchor frame. A least median of squares based algorithm is adopted for robust motion estimation. In the case of a large proportion of outliers, we detect possible algorithm failure and perform re-estimation along a different route. Based on the motion parameters between consecutive frames, the static background panorama and dynamic foreground panorama are constructed from warped images over a whole video sequence.