A spectra partition algorithm based on spectral clustering for interval variable selection

Abstract Variable selection is recognized as a way to build a robust model and improve its prediction performance in NIR spectral analysis. Interval variable selection as one of two main strategies of variable selection, is believed to have a good spectra interpretation. Many interval variable selection algorithms have been proposed and developed. Among them dividing the spectra into intervals manually is normally needed, and generally the equal-width partition is adopted, which is arbitrary and subjective. Clustering is known for self-discovering the inner structure of data. In this work, a new method called spectral clustering-based interval partition (SCIP), is purposed as an alternative to the equal-width partition. It partitions the spectra by using spectral clustering based on the correlation between variables. When SCIP is adopted, compared with the equal-width partition under the same number of partitions, the commonly used interval variable selection methods, such as forward interval partial least regression (fiPLS), backward interval partial least regression (biPLS), synergy interval partial least regression (siPLS), genetic algorithm-interval partial least regression (GA-iPLS), and interval combination optimization (ICO) can supply more reasonable wavelength intervals and contribute better prediction performances in four near infrared datasets.