Identity Verification Through the Fusion of Face and Speaker Recognition

Abstract : In this research, face recognition and speaker identification systems are each converted into verification systems. The two verification systems are then fused to form a single identity verification system. Finally, the use of the Karhunen-Loeve Transform (KLT) for dimensional reduction is examined for suitability in the verification task. The base face recognition system used the KLT for feature reduction and a back-propagation neural net for classification. Verification involved training a net for each individual in the database for two classes of outputs, 'Joe' or 'not Joe.' The base speaker identification system used Cepstral analysis for feature extraction and a distortion measure for classification. Verification in this case involved performing the KLT on the Cepstral coefficients and then classifying using a two-class neural net for each individual, similarly to the face verifier implementation. KLT feature reduction is compared to alternative linear and non-linear methods, and the KLT is found to provide superior performance. The fusion of the two base verification systems is shown to provide superior performance over either system alone.