Multi-Channel Biometrics for eHealth Combining Acoustic and Machine Vision Analysis of Speech, Lip Movement and Face: a Case Study

The purpose of this work is to present a solution combining user-friendliness and cost-effectiveness use of audio (speech) & visual (video/image) biometrics, for eHealth, able to achieve better accuracy and increase the ability to avoid counterfeiting. This work shows the evaluation results for an eHealth pilot study that tested the security, privacy, usability and cost-effective features of a user authentication platform for the management of sensitive heterogeneous multi-scale medical data (i.e. medical imaging such as MRI/CT scans, physical reports, and laboratory results), through easy acquisition of biometric data via laptops, and tablets equipped with cameras and microphones. Regarding the user enrollment and verification, audio-visual biometric information from an individual is captured, processed and stored as a biometric template. In subsequent uses, biometric information is captured and compared with the biometric templates. If the comparison is successful the verified user could be allowed to sign in to a medical collaboration platform of the hospitals infrastructure. In this work we present the biometric platform developed, the testing methodology and the administrative framework and legal processes, related to GDPR, for the eHealth pilot study and the results from our quantitative and qualitative analysis that was performed.