Boosting Occluded Image Classification via Subspace Decomposition-Based Estimation of Deep Features

Classification of partially occluded images is a highly challenging computer vision problem even for the cutting-edge deep learning technologies. To achieve a robust image classification for occluded images, this article proposes a novel scheme using the subspace decomposition-based estimation (SDBE). The proposed SDBE-based classification scheme first employs a base convolutional neural network to extract the deep feature vector (DFV) and then utilizes the SDBE to compute the DFV of the original occlusion-free image for classification. The SDBE is performed by projecting the DFV of the occluded image onto the linear span of a class dictionary (CD) along the linear span of an occlusion error dictionary (OED). The CD and OED are constructed, respectively, by concatenating the DFVs of a training set and the occlusion error vectors of an extra set of image pairs. Two implementations of the SDBE are studied in this article: 1) the l₁-norm and 2) the squared l₂-norm regularized least-squares estimates. By employing the ResNet-152, pretrained on the ImageNet Large-Scale Visual Recognition Challenge 2012 (ILSVRC2012) training set, as the base network, the proposed SBDE-based classification scheme is extensively evaluated on the Caltech-101 and ILSVRC2012 datasets. Extensive experimental results demonstrate that the proposed SDBE-based scheme dramatically boosts the classification accuracy for occluded images, and achieves around 22.25% increase in classification accuracy under 20% occlusion on the ILSVRC2012 dataset.