Nonnegative Matrix Cofactorization for Weakly Supervised Image Parsing

Image parsing, which is the task of assigning each pixel with a semantic label, is a challenging problem when only supervised under image-level tags. In this letter, we propose a Nonnegative Matrix Cofactorization method to perform image parsing with noisy tags, i.e., some tags may be incorrect or missing. Given a collection of noisily tagged images, we first oversegment them into superpixels. Then, the superpixels' label matrix, which is aimed to estimate, and the feature matrix are simultaneously decomposed into nonnegative factor matrices with a graph Laplacian constraint and an orthogonal constraint. This cofactorization is able to jointly learn a discriminative dictionary and a linear classifier. The proposed approach therefore is robust to noise. Experimental results on two real-world image datasets MSRC-21 and LabelMe demonstrate the encouraging performance of our approach in comparison with the state of the arts.