采用多尺度多路混合注意力机制的脑电情绪识别方法

doi:10.3778/j.issn.1002-8331.2309-0201

摘要/Abstract

摘要： 为了充分利用电极间的空间分布特性和多域深层次特征，提高脑电信号（electroencephalogram, EEG）情绪识别精度，提出一种采用多尺度多路混合注意力机制的方法进行脑电情绪识别。对EEG进行基线处理与空间重构，减小个体差异的同时增强空间特征信息。将时频二维和空间三维数据共同输入到多尺度多路混合注意网络（multi-scale and multi-path hybrid attention network，MS-MPHAN）进行训练，其中，设计双流时空融合模块加强时域特征和空域特征的联系；引入多尺度卷积核进行多尺度的初步特征提取，进而增加特征的视野维度；利用多路结构二次提取不同层次的深层特征，并在通道层次、二维空间层次和精细化的坐标层次对特征进行增强与融合。在DEAP数据集和DREAMER数据集上进行EEG情绪四分类识别，结果显示DEAP数据集和DREAMER数据集的平均准确率达到93.75%和98.93%，证明了所提方法在脑电情绪识别上的良好性。

关键词: 神经网络, 脑电信号, 情绪识别, 注意力机制

Abstract: In order to fully utilize the spatial distribution characteristics and multi-domain deep level features between electrodes, and improve the accuracy of EEG emotion recognition, a method using multi-scale and multi-channel mixed attention mechanism is proposed for EEG emotion recognition. Firstly, baseline processing and spatial reconstruction are performed on EEG to reduce individual differences while enhancing spatial feature information. Then, the time-frequency two-dimensional and spatial three-dimensional data are jointly input into the multi-scale and multi-path hybrid attention network (MS-MPHAN) for training. A dual stream spatiotemporal fusion module is designed to strengthen the connection between temporal and spatial features; it introduces multi-scale convolution kernels for preliminary feature extraction at multiple scales, thereby increasing the field of view dimension of features; it uses a multi-channel structure to extract deep features at different levels, and enhances and fuses features at the channel level, two-dimensional spatial level, and refined coordinate level. Finally, EEG emotion four classification recognition is performed on the DEAP and DREAMER datasets, and the results show that the average accuracy of the DEAP and DREAMER datasets reaches 93.75% and 98.93%, demonstrating the good performance of the proposed method in EEG emotion recognition.

Key words: neural network, electroencephalogram, emotional recognition, attention mechanism

谷学静, 刘佳, 郭宇承, 杨赵辉. 采用多尺度多路混合注意力机制的脑电情绪识别方法[J]. 计算机工程与应用, 2024, 60(19): 130-138.

GU Xuejing, LIU Jia, GUO Yucheng, YANG Zhaohui. EEG Emotion Recognition Method Using Multi-Scale and Multi-Path Hybrid Attention Mechanism[J]. Computer Engineering and Applications, 2024, 60(19): 130-138.

参考文献

[1] ZHANG G, DAVOODNIA V, ETEMAD A. PARSE: pairwise alignment of representations in semi-supervised EEG learning for emotion recognition[J]. IEEE Transactions on Affective Computing, 2022, 13(4): 2185-2200.
[2] ARYA R, SINGH J, KUMAR A. A survey of multidisciplinary domains contributing to affective computing[J]. Computer Science Review, 2021, 40(3): 100399.
[3] 陈景霞, 唐喆喆, 林文涛, 等. 用于脑电数据增强和情绪识别的自注意力GAN[J]. 计算机工程与应用, 2023, 59(5): 160-168.
CHEN J X, TANG Z Z, LIN W T, et al. Self-attention GAN for EEG data augmentation and emotion recognition[J]. Computer Engineering and Applications, 2023, 59(5): 160-168.
[4] 柳长源, 李文强, 毕晓君.基于脑电信号的情绪特征提取与分类[J]. 传感技术学报, 2019, 32(1): 82-88.
LIU C Y, LI W Q, BI X J. Emotional feature extraction and classification based on EEG signals[J]. Chinese Journal of Sensors and Actuators, 2019, 32(1): 82-88.
[5] 李贤哲, 暴伟, 谢能刚. 基于脑电信号的情绪识别[J]. 北京生物医学工程, 2022, 41(1): 8-16.
LI X Z, BAO W, XIE N G. Emotional recognition based on electroencephalograph signals[J]. Beijing Biomedical Engineering, 2022, 41(1): 8-16.
[6] 廖健熙, 吕勇, 王振宇, 等. 基于随机森林算法的脑电情绪识别研究[J]. 电脑与信息技术, 2022, 30(3): 1-4.
LIAO J X, LV Y, WANG Z Y, et al. Emotion recognition of EEG based on random forest algorithm[J]. Computer and Information Technology, 2022, 30(3): 1-4.
[7] 洪腾蛟, 丁凤娟, 王鹏, 等. 深度学习在轴承故障诊断领域的应用研究[J]. 科学技术与工程, 2021, 21(22): 9203-9211.
HONG T J, DING F J, WANG P, et al. Application of deep learning in the bearing fault diagnosis [J]. Science Technology and Engineering, 2021, 21(22): 9203-9211.
[8] HUANG W, XUE Y, HU L, et al. S-EEGNet: electroencephalogram signal classification based on a separable convolution neural network with bilinear interpolation[J]. IEEE Access, 2020, 8: 131636-131646.
[9] 梁椰舷, 李婷, 姬昊余. 多通道连续卷积神经网络脑电信号情绪识别[J]. 计算机系统应用, 2023, 32(1): 399-405.
LIANG Y X, LI T, JI H Y. Emotion recognition of EEG signals based on multi-channel and continuous CNN[J]. Computer Systems & Applications, 2023, 32(1): 399-405.
[10] 赵丹丹, 赵倩, 董宜先, 等. 基于EEG和DE-CNN-GRU的情绪识别[J]. 计算机系统应用, 2023, 32(4): 206-213.
ZHAO D D, ZHAO Q, DONG Y X, et al. Emotional recognition based on EEG and DE-CNN-GRU[J]. Computer Systems & Applications, 2023, 32(4): 206-213.
[11] YANG Y, WU Q, QIU M, et al. Emotion recognition from multi-channel EEG through parallel convolutional recurrent neural network[C]//Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN), 2018: 1-7.
[12] 蔡冬丽, 钟清华, 朱永升, 等. 三维输入卷积神经网络脑电信号情感识别[J]. 计算机工程与应用, 2021, 57(5): 161-167.
CAI D L, ZHONG Q H, ZHU Y S, et al. EEG emotion recognition using convolutional neural network with 3D input [J]. Computer Engineering and Applications, 2021, 57(5): 161-167.
[13] 张冰雪, 李文楷. 少量通道脑电信号的实时情绪分类模型[J]. 小型微型计算机系统, 2024, 45(2): 271-277.
ZHANG B X, LI W K. Real-time emotion classification model for few-channel EEG signals[J]. Journal of Chinese Computer Systems, 2024, 45(2): 271-277.
[14] KIM Y, CHOI A. EEG-based emotion classification using long short-term memory network with attention mechanism[J]. Sensors (Basel, Switzerland), 2020, 20(23): 6727.
[15] CHANG W, XU L, YANG Q, et al. EEG signal-driven human-computer interaction emotion recognition model using an attentional neural network algorithm[J]. Journal of Mechanics in Medicine and Biology, 2023: 2340080.
[16] KOELSTRA S, MUHL C, SOLEYMANI M, et al. DEAP: a database for emotion analysis; using physiological signals[J]. IEEE Transactions on Affective Computing, 2011, 3(1): 18-31.
[17] KATSIGIANNIS S, RAMZAN N. DREAMER: a database for emotion recognition through EEG and ECG signals from wireless low-cost off-the-shelf devices[J]. IEEE Journal of Biomedical and Health Informatics, 2017, 22(1): 98-107.
[18] LANG P J, BRADLEY M M, CUTHBERT B N.International affective picture system(IAPS): technical manual and affective ratings[J]. Psychology, 1997, 77(4): 352-360.
[19] ZHENG W L, GUO H T, LU B L. Revealing critical channels and frequency bands for emotion recognition from EEG with deep belief network[C]//Proceedings of the 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER), 2015: 154-157.
[20] LU Y, WANG M, WU W, et al. Dynamic entropy-based pattern learning to identify emotions from EEG signals across individuals[J]. Measurement, 2020, 150: 107003.
[21] 汪佳衡, 王跃明, 姚林. 基于滤波器组长短时记忆网络的脑电信号情绪识别[J]. 生物医学工程学杂志, 2021, 38(3): 447-454.
WANG J H, WANG Y M, YAO L. Using electroencephalogram for emotion recognition based on filter-bank long short-term memory networks[J]. Journal of Biomedical Engineering, 2021, 38(3): 447-454.
[22] HU Z, CHEN L, LUO Y, et al. EEG-based emotion recognition using convolutional recurrent neural network with multi-head self-attention[J]. Applied Sciences, 2022, 12(21): 11255.
[23] 闫超, 张雪英, 张静, 等. 结合注意力机制和特征融合1DCNN的脑电情感识别[J]. 计算机工程与应用, 2023, 59(13): 171-177.
YAN C, ZHANG X Y, ZHANG J, et al. EEG emotion recognition combined with attention mechanism and feature fusion 1DCNN[J]. Computer Engineering and Applications, 2023, 59(13): 171-177.
[24] 张英杰, 谢云. 基于CNN-LSTM的脑电情感四分类研究[J]. 科学技术与工程, 2023, 23(24): 10437-10444.
ZHANG Y J, XIE Y. Four classification of EEG emotion based on CNN-LSTM[J]. Science Technology and Engineering, 2023, 23(24): 10437-10444.
[25] ZHANG T, WANG X, XU X, et al. GCB-Net: graph convolutional broad network and its application in emotion recognition[J]. IEEE Transactions on Affective Computing, 2019, 13(1): 379-388.
[26] 张永, 刘纪奎, 柯文龙. 基于并行可分离卷积和标签平滑正则化的脑电情感识别[J]. 电信科学, 2023, 39(5): 116-128.
ZHANG Y, LIU J K, KE W L. EEG emotion recognition based on parallel separable convolution and label smooth regularization[J]. Telecommunications Science, 2023, 39(5): 116-128.
[27] 陶威. 基于注意力机制的脑电情绪识别方法研究[D]. 合肥: 合肥工业大学, 2021.
TAO W. The research of EEG emotion recognition based on attention mechanism[D]. Hefei: Hefei University of Technology, 2021.