Selection of keyframes for video colourization using steerable filtering

The appropriate selection of distinctive keyframes to represent the salient contents of a video is a critical task in video processing applications that rely on content analysis or information retrieval. Although many of the existing keyframe selection techniques perform satisfactorily in capturing salient visual contents, they often fail to adequately highlight the changes in visual information brought about by motion of objects between frames. In this paper, we propose a technique for keyframe selection by formulating the dissimilarity between the frames of a video shot in terms of the change in orientations that the corresponding objects of the two frames have undergone due to motion. This is accomplished by steerable filtering of the frames in order to extract the information about the local orientation of pixels within each frame. The frame to frame dissimilarity is adaptively thresholded over a group of frames in order to select the keyframes. In essence, keyframes are selected at the temporal instances where the change in orientation attains local maxima. Our keyframe selection methodology is specifically relevant to video colourization due to the fact that the keyframes that are to be employed for colourization must be chosen such that they capture all orientational changes effectively, while ensuring adequate content coverage.