Sparse spectral signal reconstruction for one proposed nine-band multispectral imaging system

Abstract Multispectral filter array (MSFA) imaging with one single sensor is a portable and inexpensive means of acquiring spectral image which is widely used for object detection, material analysis and mechanical system diagnosis. The most challenging task for MSFA imaging is the multispectral demosaicking with the aim of reconstructing the captured raw/mosaic image, especially for the systems with many bands which result in higher sparseness of the raw data. In this paper, we present a 9-band MSFA imaging system in a repetitive 4 × 4 filter array on a single sensor, and propose a demosaicking algorithm for reconstructing the raw spectral image. Within the 4 × 4 MSFA pattern, the fifth spectral band takes up half of the total spatial position while the remaining eight bands occupy 1/16 respectively. To reconstruct the sparse raw data, we first recover the fifth band by propagating the neighboring sampled pixels to the unsampled position using the image gradients, and then employ the reconstructed fifth band as a guided image to demosaick the other bands with the guided filter and residual interpolation. Finally, we estimate the spectral reflectance values from the multispectral image and the characterization matrix. In the experiment, we evaluate the performance of the 9-band imaging system with the binary tree-based edge-sensing (BTES) algorithm, compressed sensing (CS) algorithm, and our proposed demosaicking algorithm. The experiment results demonstrate that our demosaicking algorithm not only outperforms BTES and CS algorithms in terms of objective image quality, e.g., PSNR values and spectral errors, but also reduces the demosaicking artifacts in terms of subjective evaluations.