EEG Emotion Recognition Combined with Attention Mechanism and Feature Fusion 1DCNN

doi:10.3778/j.issn.1002-8331.2204-0158

Abstract

Abstract: In the EEG emotion recognition field, when processing one-dimensional data into two-dimensional or three-dimensional data, and then using the 2DCNN or 3DCNN model for processing and recognition, there is a large number of parameters and the parameter optimization method is time-consuming and laborious. A 1D convolutional neural network（1DCNN） model based on the attention mechanism of frequency bands and brain regions is proposed. Firstly, it extracts features from EEG signals and uses t test to select optimal features; then a new 1DCNN emotion recognition model is designed according to the structure of the extracted features, which provides an interpretable model for parameter selection and convolution operations. Finally, according to the difference in the ability of the left and right brain regions to reflect emotions, a brain region attention mechanism is proposed, and combined with the frequency band attention mechanism, it can better pay attention to the brain regions and frequency bands related to emotion. The average recognition rate of the FBA-1DCNN model proposed in the valence dimension and arousal dimension two-classification experiments of the DEAP EEG emotional database reaches 94.01% and 93.55%, and the average recognition rate in valence-arousal dimension four-classification experiment reaches 89.38%, which is 2.96, 3.31 and 7.69 percentage points higher than the existing 1DCNN model.

Key words: EEG emotion recognition, t test, deep learning, 1D convolutional neural network（1DCNN）, attention mechanism

摘要： 针对脑电情感识别领域中处理一维数据时将其映射为二维或三维数据，然后利用2DCNN或3DCNN模型进行处理和识别时，存在参数量大且参数寻优方法费时费力的问题，提出一种基于频段和脑区注意力机制的1DCNN模型。对脑电信号提取特征并采用t检验进行最优特征选择；根据提取特征的结构设计了一种新型的1DCNN情感识别模型，为模型的参数选择和卷积操作提供可解释性;最后根据左、右脑区对情感反应能力的不同，提出一种脑区注意力机制，并与频段注意力机制相结合更好地关注与情感相关的脑区与频段。提出的FBA-1DCNN模型在DEAP脑电情感数据库的效价维和唤醒维二分类实验上的平均识别率分别达到了94.01%和93.55%，在效价-唤醒维四分类实验上的平均识别率达到了89.38%，比现有的1DCNN模型分别提升了2.96、3.31和7.69个百分点。

关键词: 脑电情感识别, t检验, 深度学习, 一维卷积神经网络（1DCNN）, 注意力机制

YAN Chao, ZHANG Xueying, ZHANG Jing, CHEN Guijun, HUANG Lixia. EEG Emotion Recognition Combined with Attention Mechanism and Feature Fusion 1DCNN[J]. Computer Engineering and Applications, 2023, 59(13): 171-177.

闫超, 张雪英, 张静, 陈桂军, 黄丽霞. 结合注意力机制和特征融合1DCNN的脑电情感识别[J]. 计算机工程与应用, 2023, 59(13): 171-177.

References

[1] HELLER W.Neuropsychological mechanisms of individual differences in emotion，personality，and arousal[J].Neuropsychology，1993，7（4）：476-489.
[2] 张雪英，王薇蓉，孙颖，等.面向脑电情感识别的改进多分类RVM模型研究[J].计算机工程与应用，2019，55（9）：112-117.
ZHANG X Y，WANG W R，SUN Y，et al.Research on improved multi-classification RVM for emotional recognition of EEG signal[J].Computer Engineering and Applications，2019，55（9）：112-117.
[3] 陈景霞，郑茹，贾小云，等.共空间模式结合小波包分解的脑电情感分类[J].计算机工程与应用，2019，55（1）：149-153.
CHEN J X，ZHENG R，JIA X Y，et al.EEG emotion classification based on common spatial patterns and Wavelet packet decomposition[J].Computer Engineering and Applications，2019，55（1）：149-153.
[4] 周奕隽，李冬冬，王喆，等.融合倒谱特征的脑电（EEG）情感分类[J].计算机工程与应用，2020，56（21）：164-169.
ZHOU Y J，LI D D，WANG Z，et al.Cepstrum feature fusion for EEG emotion classification[J].Computer Engineering and Applications，2020，56（21）：164-169.
[5] 张冠华，余旻婧，陈果，等.面向情绪识别的脑电特征研究综述[J].中国科学：信息科学，2019，49（9）：1097-1118.
ZHANG G H，YU M J，CHEN G，et al.A review of EEG features for emotion recognition[J].SCIENTIA SINICA Informationis，2019，49（9）：1097-1118.
[6] 陈景霞，王丽艳，贾小云，等.基于深度卷积神经网络的脑电信号情感识别[J].计算机工程与应用，2019，55（18）：103-110.
CHEN J X，WANG L Y，JIA X Y，et al.Emotion recognition of EEG signals based on deep convolutional neural network[J].Computer Engineering and Applications，2019，55（18）：103-110.
[7] ELHAM S S，MAHMOUD E.EEG-based emotion recognition using 3D convolutional neural networks[J].International Journal of Advanced Computer Science and Applications，2018，9（8）：329-337.
[8] YANG Y，WU Q，FU Y，et al.Continuous convolutional neural network with 3D input for EEG-based emotion recognition[C]//Proceedings of the International Conference on Neural Information Processing，2018：433-443.
[9] CHEN S，ABHINAV S，SAURABH S，et al.Revisiting unreasonable effectiveness of data in deep learning era[C]//Proceedings of the IEEE International Conference on Computer Vision（ICCV），2017：843-852.
[10] TAO W，LI C，SONG R，et al.EEG-based emotion recognition via channel-wise attention and self attention[J].IEEE Transactions on Affective Computing，2020（1）：1-12.
[11] KOELSTRA S.DEAP：a database for emotion analysis using physiological signals[J].IEEE Transactions on Affective Computing，2012，3（1）：18-31.
[12] TORRES E P，TORRES E A，MYRIAM H A，et al.EEG-based BCI emotion recognition：a survey[J].Sensors，2020，20（18）：1-36.
[13] YIN Z，ZHAO M，WANG Y，et al.Recognition of emotions using multimodal physiological signals and an ensemble deep learning model[J].Computer Methods & Programs in Biomedicine，2017，140：93-110.
[14] GARG A，KAPOOR A，BEDI A K，et al.Merged LSTM model for emotion classification using EEG signals[C]//Proceedings of the 2019 International Conference on Data Science and Engineering（ICDSE），2019：139-143.
[15] PIHO L，TJAHJAD T.A mutual information based adaptive windowing of informative EEG for emotion recognition[J].IEEE Transactions on Affective Computing，2018，11（4）：722-735.
[16] PANE E S，WIBAWA A D，PURNOMO M H.Improving the accuracy of EEG emotion recognition by combining valence lateralization and ensemble learning with tuning parameters[J].Cognitive Processing，2019，20（4）：405-417.
[17] SHARMA R，PACHORI R B，SIRCAR P.Automated emotion recognition based on higher order statistics and deep learning algorithm[J].Biomedical Signal Processing and Control，2020，58：1-10.