A best-features based digital rotoscope

The paper presents a best-features based digital rotoscope to animate a video sequence. Our thesis is that corners in an image are viable feature points for animation because they move appreciably across frames. Our proposed rotoscope processes a video sequence one frame at a time and comprises four image processing stages namely, the background subtraction stage, two-phase Shi-Tomasi feature extraction stage, watershed segmentation stage, and the color palette stage. The background subtraction stage subtracts the background from the input image, thereby isolating the foreground colors and producing an inverted image. The two-phase Shi-Tomasi feature extraction stage performs two passes of corner detection on the inverted image to identify a user-defined number of best-features. The watershed segmentation stage uses the best-features as markers to segment the input frame. Finally, the color palette stage colors each one of the segments with the average of the input frame's color values in that segment, thereby creating a rotoscoped frame. After processing all of the frames in a video sequence, the digital rotoscope produces an animated movie. We study the impact of choosing different numbers of best-features on the quality of animation. Our empirical study reveals that for a frame-size of 480 × 640 × 3, 1000 features or more produce effective animation. Our four-stage digital rotoscope provides opportunities for parallel implementations on high-performance architectures, thereby creating avenues for fast analysis.