Exploiting object semantic cues for Multi-label Material Recognition

Recognizing materials on an object's surface is important because it significantly benefits understanding quality and functionality of the concerned object. This paper focuses on a Multi-Label Material Recognition (M-LMR) problem that is to identify multiple material categories on an object from a single photograph. As a distinct task, material categorization is different from traditional vision recognition tasks such as recognition of shapes, objects, or scenes, and cannot be explained in terms of simple feature judgments. To address this problem, besides employing state-of-the-art image descriptors (e.g., image features learned by deep convolutional network) for distinguishing materials, we focus on exploiting object semantic cues to facilitate the M-LMR. Specifically, we derive a binary-SVM based framework that integrates image features with the object identity as input to judge surface material categories. We argue that the use of object information is essentially for exploiting correlations of material labels, where label correlations are very useful for facilitating a multi-label recognition problem. Experimental results shows consistent improvements of the presented method over state-of-the-arts, even though the object identity is automatically inferred.