Deception in the Eyes of Deceiver: A Computer Vision and Machine Learning Based Automated Deception Detection

Abstract There is growing interest in the use of automated psychological profiling systems, specifically applying machine learning to the field of deception detection. Several psychological studies and machine-based models have been reporting the use of eye interaction, gaze and facial movements as important clues to deception detection. However, the identification of very specific and distinctive features is still required. For the first time, we investigate the fine-grained level eyes and facial micro-movements to identify the distinctive features that provide significant clues for the automated deception detection. A real-time deception detection approach was developed utilizing advanced computer vision and machine learning approaches to model the non-verbal deceptive behavior. Artificial neural networks, random forests and support vector machines were selected as base models for the data on the total of 262,000 discrete measurements with 1,26,291 and 128,735 of deceptive and truthful instances, respectively. The data set used in this study is part of an ongoing programme to collect a larger dataset on the effects of gender and ethnicity on deception detection. Some observations are made based on this data which should not be interpreted as scientific conclusions, but pointers for future work. Analysis of the above models revealed that eye movements carry relatively important clues to distinguish truthful and deceptive behaviours. The research outcomes align with the findings from forensic psychologists who also reported the eye movements as distinctive for the truthful and deceptive behavior. The research outcomes and proposed approach are beneficial for human experts and has many applications within interdisciplinary domains.