Melanoma Detection Based on Color and Hyperspectral Imaging

Melanoma is the most fatal form of all skin cancer types, being diagnosed mostly among lightly pigmented skin. An early screening of melanoma can greatly contribute to successful treatment, hence reliable early detection systems are highly demanded and key technologies need to be developed. The existing automated melanoma detection algorithms are dominantly based on color images. Early methods adopted machine vision algorithms which require hand-crafted features to be designed. With the development of machine learning models and the access to the large skin image datasets, deep learning has been introduced for melanoma detection so effective feature can be automatically learned. However, the research on this topic is at its early stage. While machine learning models can be created for skin cancer detection, this task can also be boosted by introducing new imaging technology beyond the traditional color imaging process. To this end, hyperspectral images show its advantage because of their multiple spectral bands, thereby providing extra reflectance information that is related to the intrinsic properties of skins and its composition. The challenges on this technology are that there is no open dataset to support the research and how to effectively use the spectral and spatial information in the hyperspectral images for melanoma detection remains unsolved. To address the above issues, in this thesis, we introduce three methods for melanoma detection. The first method is based on machine vision. This method follows the common image classifi cation pipeline, i.e. pre-processing, segmentation, feature extraction, and classifi cation. The novelty of this method is that before classifi cation, we introduce a dimensionality reduction method to the extracted features as a post-processing step. This post-processing procedure is based on Mahalanobis distance learning and constrained graph regularized nonnegative matrix factorization. The proposed method allows supervised learning for feature dimensionality reduction by incorporating both global geometry and local manifold, so as to enhance the discriminability of the classifi cation performance. The proposed method is evaluated on PH2 Dermoscopy Image Dataset and Edinburgh Dermofi t Image Library, with comparison against four alternative classifi cation methods. The experimental results demonstrate that the best performance is achieved with the proposed method compared with another NMF baseline method and direct classifi cation without post-processing. The second melanoma detection method is deep learning-based. Deep learning is a datadriven technique that does not require hand-crafted feature design, thereby improving the generalization capacity of the model. However, a well-trained deep learning model from one dataset often cannot be generalized to other datasets, even when all datasets have the same categories. This is mainly because of the domain shift between datasets of different cohorts in the data capture process. On this regards, we exploit two methods to relieve this issue by evaluating on two different skin disease datasets, MoleMap and HAM10000. The fi rst option is parameter-based transfer learning. We use a progressive transfer learning scheme to share transferable knowledge between multiple datasets, i.e. transferring knowledge from a task-different source dataset (ImageNet) to a category-same but dataset-different intermediate dataset (MoleMap) and at last to the target dataset (HAM10000). For the second option, we use cycle-consistent generative adversarial networks to translate the images from the source domain into the target domain for pixel-wise image adaptation. The synthesized image data are integrated with the training samples in the original target domain during the training stage, therefore forming the methods of dataset adaptation and modality domain adaptation. The results of progressive transfer learning show that it achieves better performance and generalization capacity than 1-step transfer learning model and model training from scratch. Furthermore, both dataset adaptation and modality domain adaptation show improvements of the model generalization capacity, melanoma detection, skin cancer detection, and skin disease classi fication. The third method is by means of hyperspectral imaging. Besides the spatial information, hyperspectral imaging provides fine resolution in spectral wavelength. With the abundant spectral-spatial information, hyperspectral imaging can facilitate melanoma detection. In this research, we introduce a hyperspectral dermoscopic dataset and describe a detailed description of the hardware and software of the data collection system developed in the Spectral Imaging Lab of Griffth University. As far as we know, this is the fi rst open hyperspectral dermoscopic benchmark dataset. Based on this dataset, we provide the baselines using machine learning methods, which include sparse coding, support vector machine, and deep learning. We show the performance of spatial features, spectral features and joint spectral-spatial features on this dataset. The experiments show that the classifi cation performance is improved with extra spectral features.