Fast evaluation of unhealthy and healthy neonates using hyperspectral features on 700-850 nm wavelengths, ROI extraction, and 3D-CNN

Abstract Objectives Hyperspectral imaging (HSI) has great potential in detecting the health conditions of neonates as it provides diagnostic information about the tissue by avoiding tissue biopsy. HSI gives more features than thermal imaging, which can obtain images in a single wavelength, as it can obtain images in a large number of wavelengths. The data obtained with hyperspectral sensors are 3-dimensional data called hypercube including first two-dimensional spatial information and third-dimensional spectral information. Material and methods In this study, hyperspectral data were obtained from 19 different neonates in the Neonatal Intensive Care Unit (NICU) of Selcuk University, Medical Faculty. There are 16 hypercubes from 16 unhealthy neonates, 16 hypercubes from 3 healthy neonates in a period of three months, and 32 hypercubes in total are available. For the training of 3D-CNN model, data augmentation methods, such as rotation, height shifting, width shifting, and shearing were applied to hyperspectral data. A number of 32 hypercubes taken from neonates in NICU were augmented to 160 hypercubes. Spectral signatures were examined and 51 bands in the range of 700-850 nm with distinctive features were used for the classification. The spectral dimension was reduced by applying Principal Component Analysis (PCA) to all hypercubes. In addition, it is aimed to obtain both spectral and spatial features with the 3D-CNN. For increasing the classification efficiency, ROI extraction was made and four datasets were created in different spatial dimensions. These datasets contain 160, 640, 1440, and 5760 hypercubes, respectively. Results The best result was achieved by using 5760 hypercubes of 25x25x51. As a result of the classification of the hypercubes, accuracy 98.00%, sensitivity 97.22%, and specificity 98.78% were obtained. It was determined how many PCs used to achieve the best result. Further, the proposed 3D-CNN model is compared to 2D-CNN model to evaluate the performance of the study. Conclusion It was aimed to evaluate the health status of neonates fastly by using HSI and 3D-CNN for the first time. The obtained results are an indication that HSI and 3D-CNN are very effective for the evaluation of unhealthy and healthy neonates.