Optimal Discriminative Feature and Dictionary Learning for Image Set Classification

Abstract Image set classification has drawn increasing attention and it has been widely applied to many real-life domains. Due to the existence of multiple images in a set, which contain various view appearance changes, image set classification is a rather challenging task. One potential solution is to learn powerful representations from multiple images to decrease the intra-class diversity and enlarge the inter-class separation. In this paper, we propose an optimal discriminative feature and dictionary learning (ODFDL) method, which attempts to learn a feature mapping matrix and a dictionary such that in the mapped feature space the inter-class sparse reconstruction error of data is maximized and the intra-class sparse reconstruction error is minimized. This learning strategy enforces the learned sparse representations from image sets have large inter-class separation and small intra-class scatter. Furthermore, to better exploit the non-linear information of data from different image sets, we also present two non-linear ODFDL methods, termed Kernel-ODFDL and Hierarchy-ODFDL to further improve the classification performance. Experiments on five commonly used image sets exhibit that our approaches are comparable with many state-of-the-arts.