Weakly-Supervised Attribute Segmentation

Semantic segmentation is a fundamental vision problem which aims to divide an image into non-overlapped regions and then assign them with predefined object labels. Despite its latest development (especially deep neural network-based), semantic segmentation is still far from comprehensive understanding of the visual world around us. For example, the obtained object labels are not fine-grained enough for solving more complicated vision tasks such as fine-grained retrieval with multiple attribute-based queries (e.g. the query ‘a bird’ along with the attribute ‘has wing colour: grey’). In this paper, we thus choose to study a more challenging vision problem called attribute segmentation, which aims to localize attributes within the targeted object (e.g. bird) in a given image. Due to the high cost of collecting pixel-level attribute annotations, we focus on weakly-supervised attribute segmentation (WSAS) that utilizes only image-level attribute labels for model training. Note that this new WSAS problem degrades to the conventional weakly-supervised semantic segmentation (WSSS) problem when attribute labels are considered as object labels. To overcome the attribute label noise caused by image-level weak supervision, we propose a novel sparse learning method for solving the WSAS problem. Extensive experiments demonstrate that the proposed WSAS method significantly outperforms the state-of-the-art WSSS and attribute localization methods.