Motor Imagery Recognition Based on Graph Convolution Network

doi:10.3778/j.issn.1002-8331.2008-0404

Abstract

Abstract: Motor imagery recognition is an important research direction of brain-computer interface（BCI）, which converts brain neural activity signals into code to realize mind control. In recent years, the application of deep learning algorithm further improves the accuracy of motor imagery recognition. However, the current motor imagery analysis based on deep learning takes multi-channel EEG signals as two-dimensional matrix signals, and ignores the different nodes spatial association information. To solve this problem, the graph convolution network（GCN） is applied to the classification of motor imagery. The EEG structure is established by the correlation coefficients of multiple nodes’ EEG signals, and the time-frequency feature information of EEG signals is extracted as the input of GCN. Then, node time-frequency features are aggregated by graph convolution network to learn spectral domain features. Finally, the classification results are output through full connection layer. This method achieves 80.9% accuracy and 0.74 kappa coefficient on BCI Competition IV Dataset 2a. This method can fully learn the feature information of time, frequency and spectrum domain, and achieve better recognition effect. It provides a new idea and method for brain-computer interface of motor imagery.

Key words: graph convolution network（GCN）, motor imagery, deep learning, time-frequency feature

摘要： 运动想象识别将大脑的神经活动信号转为编码输出以实现意念控制，是脑机接口的一个重要研究方向。近年来深度学习算法的应用进一步提高了运动想象识别的准确率，但是当前基于深度学习的运动想象分析都将多路脑电信号作为二维矩阵信号，忽视了不同节点的空间关联信息。为了解决这个问题，将图卷积网络算法应用到运动想象分类中，通过多个节点脑电信号的相关系数建立脑电图结构，提取脑电信号的时频域特征信息作为输入，再经过图卷积网络进行节点特征聚合以学习谱域特征，最后通过全连接层输出分类结果。该方法在BCI Competition IV Dataset 2a数据集上取得80.9%的准确率和0.74的kappa系数，相比其他方法可以充分学习时、频、谱域的特征信息，取得更好的识别效果，为运动想象脑机接口提供一种新的思路和方法。

关键词: 图卷积网络（GCN）, 运动想象, 深度学习, 时频特征

XU Xuetian, CAI Yuexin. Motor Imagery Recognition Based on Graph Convolution Network[J]. Computer Engineering and Applications, 2022, 58(4): 186-191.

许学添, 蔡跃新. 基于图卷积网络的运动想象识别[J]. 计算机工程与应用, 2022, 58(4): 186-191.

References

[1] MINGUILLON J，LOPEZ-GORDO M A，PELAYO F.Trends in EEG-BCI for daily-life：requirements for artifact removal[J].Biomedical Signal Processing and Control，2017，31：407-418.
[2] ABDULKADER S，ATIA A，MOSTAFA M.Brain computer interfacing：applications and challenges[J].Egyptian Informatics Journal，2015，16（2）：213-230.
[3] ANG K K，CHIN Z Y，WANG C C，et al.Filter bank common spatial pattern algorithm on BCI competition IV data-
sets 2a and 2b[J].Frontiers in Neuroscience，2012，29（6）：39-48.
[4] SHAHID S，PRASAD G.Bispectrum-based feature extraction technique for devising a practical brain-computer interface[J].Journal of Neural Engineering，2011，8（2）：025014.
[5] LU N，YIN T.Motor imagery classification via combinatory decomposition of ERP and ERSP using sparse nonnegative matrix factorization[J].Journal of Neuroscience Methods，2015，249：41-49.
[6] SCHIRRMEISTER R，SPRINGENBERG J，FIEDERER L，et al.Deep learning with convolutional neural networks for EEG decoding and visualization[J].Human Brain Mapping，2017，38（11）：5391-5420.
[7] LU N，LI T，REN X，et al.A deep learning scheme for motor imagery classification based on restricted Boltzmann machines[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering，2017，25（6）：566-576.
[8] WANG P，JIANG A，LIU X，et al.LSTM-based EEG classification in motor imagery tasks[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering，2018，26（11）：2086-2095.
[9] 胡章芳，崔婷婷，罗元，等.基于CBLSTM算法的脑电信号特征分类[J].计算机工程与应用，2019，55（24）：110-116.
HU Z F，CUI T T，LUO Y，et al.EEG signal classification algorithm based on CBLSTM[J].Computer Engineering and Applications，2019，55（24）：110-116.
[10] 唐智川，张克俊，李超，等.基于深度卷积神经网络的运动想象分类及其在脑控外骨骼中的应用[J].计算机学报，2017，40（6）：1367-1378.
TANG Z H，ZHANG K J，LI C，et al.Motor imagery classification based on deep convolutional neural network and its application in exoskeleton controlled by EEG[J].Chinese Journal of Computers，2017，40（6）：1367-1378.
[11] WU Z H，PAN S R，CHEN F U，et al.A comprehensive survey on graph neural networks[J].IEEE Transactions on Neural Networks and Learning Systems，2021，32（1）：4-24.
[12] ZHANG Z W，CUI P，ZHU W W，et al.Deep learning on graphs：a survey[J].IEEE Transactions on Knowledge and Data Engineering，2022，34（1）：249-270.
[13] KIPF T，WELLING M.Semi-supervised classification with graph convolutional networks[C]//2017 International Conference on Learning Representations，2017.
[14] YING R，YOU J，MORRIS C，et al.Hierarchical graph representation learning with differentiable pooling[C]//2018 Conference and Workshop on Neural Information Processing Systems，2018：4805-4815.
[15] FOUT A，BYRD J，SHARIAT B.Protein interface prediction using graph convolutional networks[C]//2017 Conference and Workshop on Neural Information Processing Systems，2017：6530-6539.
[16] QI S Y，WANG W G，JIA B X，et al.Learning human-object interactions by graph parsing neural networks[C]//2018 European Conference on Computer Vision，2018：407-423.
[17] CUI Z Y，HENRICKSON K，KE R M，et al.Traffic graph convolutional recurrent neural network：a deep learning framework for network-scale traffic learning and forecasting[J].IEEE Transactions on Intelligent Transportation Systems，2020，21（11）：4883-4894.
[18] DEFFERRARD M，BRESSON X，VANDERGHEYNST P.Convolutional neural networks on graphs with fast localized spectral filtering[C]//2016 Conference and Workshop on Neural Information Processing Systems，2016：3844-3852.
[19] BRUNA J，ZAREMBA W，SZLAM A，et al.Spectral networks and locally connected networks on graphs[C]//3rd International Conference on Learning Representations，2014.