Classification of Emotions Based on Audio-Visual Stimulus by EEG Signals

Emotions play an important role in communication between humans. Emotions can be expressed by words, voice intonation, facial expression and body language. In contrast, Brain Computer Interface (BCI) systems have not reached the desired level to interpret the people’s emotions. BCI systems need new resources that can be taken from humans and processed by these systems to understand emotions. Electroencephalogram (EEG) signals is one of the most important resources to achieve this target. EEG signal is the method that measures brain waves with the electrical signals of the monitoring activities. Frequency component of the EEG signals contain important information about brain activity. The aim of this study was to classify EEG signals related to negative and positive emotions based on audio-visual stimulus. SAM (Self Assessment Manikins) was used to determine participants’ emotional states. Participants rated each audiovisual stimulus in terms of the level of valence, arousal, like/dislike and dominance. Participants reported the dimension of their emotions in numerical values from 1 to 9 in decimal form. In this study, only valence assessments of participants were taken into account. Participants made their valence ratings in 1-9 range. 1 corresponds to completely unhappy; 9 correspond to completely happy emotion. In this study, assessments below 5 are accepted as negative emotion and assessments above 5 are accepted as positive emotion based on valence rating. Discrete wavelet transform (DWT) was used for feature extraction from EEG signals related to negative and positive emotions. DWT decompose a signal into detail and approximation sub-bands. The docomposition of the signal into sub-bands is obtained by consecutive high-pass and low pass filtering of the time domain signal. In this study, since theta band dinamics of EEG signals were considered to classify different emotions based on audio-visual stimulus, the number of decomposition levels was chosen as 4. Dabuechies wavelets have provided useful results in analyzing EEG signals. Hence, daubechies wavelet of order 2 (db2) was chosen in this study. Wavelet coefficients contain important information about the characteristics of the relevant signals, the wavelet coefficients of EEG signals were assumed as feature vectors and statistical features were used to reduce dimension of feature vector. In this study, different clusters consisting of EEG signals related to positive and negative emotions groups have been classified by artificial neural network (ANN). Firstly, ANN was used to obtain final feature vectors. For each participant, EEG channels offering the best classification performance were determined. it was observed that 5 EEG channels that offer the best classification performance for each participant are respectively P3, FC2, AF3, O1 and Fp1. The features vectors of these EEG channels that offer the best classification performance were composed to obtained the final feature vectors. The classification procedures have been carried out for 20 participants. The maximum classification accuracy was found as 90% and average classification accuracy was found as 76.5% by using ANN classification algorithm for 20 participants.