Extraction of Category Orthonormal Subspace for Multi-Class Classification

Abstract Extraction of pattern class associated discriminative subspace is critical to many pattern classification problems. Traditionally, pattern class labels are regarded as indicators to discriminate between pattern classes. In this work, a novel indicator model is proposed to extract discriminant subspace by projecting samples onto a space where the projected categories are mutually orthogonal and in-category normalized. Category orthonormal property and its connections to discriminative subspace extraction are derived. It is shown that the proposed method has a strong connection with the existing Fukunaga-Koontz Transformation but extends the category number from two to multiple. For applications with a large dimension size but limited number of samples, an analytic least-norm solver is developed for calculating the projection function. A discriminative subspace extraction method for multiple classes is proposed and is evaluated by a combination with classifiers. Experiments demonstrate a promising result of using the extracted category orthonormal subspace for multi-class subspace extraction when sample number is small.