Fusion of audio and video information for multi modal person authentication

Abstract We present an algorithm functioning as a supervisor module in a multi-expert decision making machine. It uses the Bayes theory in order to estimate the biases of individual expert opinions. The biases are used to calibrate and conciliate expert opinions to a single decision. This supervision technique is applied to the real case of a person authentication technique using two modalities, face and speech. The visual part involves the matching of a coarse grid containing Gabor phase information from face images. The acoustic part is performed by a text-dependent speaker verification system based on Hidden Markov Models. Experimental results show that the proposed fusion method improves the quality of individual expert decisions by reaching success rates of 99.5%.