Hyperspectral Image Classification With Stacking Spectral Patches and Convolutional Neural Networks

Jointly combining the spatial and spectral features has proved to dramatically improve the performance of classifying hyperspectral image. Recently, utilizing neural networks to automatically model the spatial-spectral feature representations for hyperspectral images has become of great interest. This paper proposes a simple but innovative framework to classify hyperspectral image with two shallow convolutional neural networks (CNNs). First, principal component analysis whitening is applied to decorrelate hundreds of spectral bands. Instead of selecting the principal components to reduce the spectral dimensionality, we retain all the spectral bands but compress the image cuboid into a one-channel spectral quilt by stacking spectral patches . In this way, not only all the spectral information is retained but also the computational complexity of training a neural network is reduced compared with conventional networks that directly input the spectral volumes. Moreover, the spectral quilt will contain some novel textural patterns that are effective at distinguishing classes. Two shallow CNNs are then applied to classify the spectral quilts. As shown in the experiments, both networks can outperform the standard analysis methods.