Face emotional responses correlate with chaotic dynamics of eye movements

Background and Objective. Paul Ekman has demonstrated that we can estimate emotions on the basis of individual facial muscles movements. However, there is a lack of information in the literature about correlation between eye movements and facial emotions. Our objective was to find out whether emotion could be also visible in the dynamical properties of the eye movements. Methods. We have performed our experiment in two sessions related to different video presentations: 1) a reference session with all emotions presentation, and 2) the main session showing presentation of some specific emotions. During both sessions, we have video recordings of the subjects' face expressions (FE) and eye movements (EM). We have calculated parameters of FE and EM dynamical changes and content of the noise. On this basis, by using time changes of EM, we have predicted 6 different face emotions. Results. We have recorded face expressions of 49 subjects who had the strongest responses to Happiness and Contempt facial emotions. We found statistically significant differences in parameters' values describing FE and EM between the reference and the main sessions trials. Parameters connected to the Chaos showed highly positive correlations with Happiness, while both the Linear and the Noise components were mostly negatively correlated with this emotion. We achieved highest results with help of the K-Nearest Neighbors algorithm obtaining: Accuracy of 0.89 (+/- 0.01) with the ROC-AUC score of 0.88 (F1 = 0.89), Precision = 0.85, Sensitivity/Recall = 0.93, Specificity = 0.82. Conclusions. We have observed that when the intensity of the Happiness increases, the eye movements become more chaotic and behave less noisy. We see possibilities for use of presented methods as a support for predictions in face expressions, when the lower part of the face is partially hidden, e.g. by a protective mask worn during the COVID epidemic. This could also be a method of confirming the authenticity of the Happiness mimicry, because it would be difficult to voluntarily correlate eye movements with certain levels of the Chaos. © 2021 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0) Peer-review under responsibility of the scientific committee of KES International.