Weakly-supervised action localization based on seed superpixels

In this paper, we present action localization based on weak supervision with seed superpixels. In order to benefit from the superpixel segmentation and to learn a priori knowledge we select the seed superpixels from the action and non-action areas of few video frames of an action sequence equally. We compute correlation, joint entropy and joint histogram as the features of the video frame superpixels based on the optical flow magnitudes and intensity information. An SVM is trained with the action and non-action seed superpixels features and is used to classify the video frame superpixels as action and non-action. The superpixels classified as action provide the action localization. The localized action superpixels are used to recognize the action class by the Dendrogram-SVM based on the already extracted features. We evaluate the performance of the proposed approach for action localization and recognition using UCF sports and UCF-101 actions datasets, which demonstrates that the seed superpixels provide effective action localization and in turn facilitates to recognize the action class.