Attention Grading Modeling Based on Schulte Grid and LSTM

doi:10.3778/j.issn.1002-8331.2101-0069

Abstract

Abstract: In the study of attention grading based on EEG signals, there are two technical difficulties that need to be solved urgently. EEG data collection and labeling of different attention types are difficult. EEG feature extraction algorithm ignores the timing characteristics of the original EEG signal. In response to the above problems, the Schulte grid paradigm based on visual search and reaction time technology is designed to realize the collection and automatic labeling of EEG data of different types of attention. The long and short-term memory（LSTM） deep learning network is designed to achieve attention force classification, save the timing characteristics of the original EEG signal. The experimental results show that this attention grading model can distinguish high, medium and low attention levels well. Compared with the existing five attention grading algorithms based on EEG signals：wavelet transform（DWT）, approximate entropy, and total Spatial pattern（CSP）, brain network based on coherence coefficient, and convolutional neural network, on the same EEG data set, this attention classification model has the highest recognition accuracy, 21.49?percentage points higher than DWT algorithm; 25.82?percentage points higher than approximate entropy algorithm; 20.53?percentage points higher than CSP algorithm; 13.32?percentage points higher than brain network algorithm based on coherence coefficient; 9.05?percentage points higher than convolutional neural network（CNN）.

Key words: attention grading model, original EEG signal, long and short-term memory（LSTM）, attention monitoring

摘要： 在基于脑电信号的注意力分级研究中，存在两个亟待解决的技术难点。第一不同注意类型的脑电数据采集及标注困难；第二脑电特征提取算法忽视原始脑电信号时序特征。针对以上问题，设计了基于视觉搜索和反应时技术的舒尔特方格范式，实现对不同注意类型脑电数据的采集以及自动标注；设计长短期记忆深度学习网络（LSTM）实现对注意力分级，保存原始脑电信号的时序特征。实验结果表明，注意力分级模型可以很好区分高中低三种注意力水平；对比现有的五种基于EEG信号的注意力分级算法，小波变换（DWT）、近似熵、共空间模式（CSP）、基于相干系数的脑网络和卷积神经网络（CNN），在相同的EEG数据集上，该注意力分级模型识别准确率最高，高出DWT算法21.49个百分点；高出近似熵算法25.82个百分点；高出CSP算法20.53个百分点；高出基于相干系数的脑网络算法13.32个百分点；高出CNN9.05个百分点。

关键词: 注意力分级模型, 原始脑电信号, 长短期记忆深度学习网络（LSTM）, 注意力监测

WANG Pai, WU Fan, WANG Mei, QIN Xuebin. Attention Grading Modeling Based on Schulte Grid and LSTM[J]. Computer Engineering and Applications, 2022, 58(15): 133-140.

王湃, 吴凡, 汪梅, 秦学斌. 舒尔特方格与LSTM的注意力分级建模[J]. 计算机工程与应用, 2022, 58(15): 133-140.

References

[1] LI C Y，ZHAO W J.Progress in the brain-computer interface：an interview with Bin He[J].National Science Review，2020，7（2）：480-483.
[2] SHIN J W，KWON S B，BAK Y，et al.BCI induces apoptosis via generation of reactive oxygen species and activation of intrinsic mitochondrial pathway in H1299 lung cancer cells[J].Science China（Life Sciences），2018，61（10）：1243-1253.
[3] QI Y，MA F Q，GE T T，et al.A bidirectional brain-computer interface for effective epilepsy control[J].Journal of Zhejiang University-Science C（Computers & Electronics），2014，15（10）：839-847.
[4] CHENG Y，BU J J，LI N，et al.Dysfunctional resting state EEG microstate correlated with the severity of cigarette exposure in nicotine addiction[J/OL].Science China（Information Sciences）：1-3[2020-06-07].http：//kns.cnki.net/kcms/detail/11.5847.TP.20200601.1550.002.html.
[5] 刘素杰.基于脑网络测度的注意力脑电分级研究[D].郑州：郑州大学，2018.
LIU S J.Study on attention EEG classification based on brain network measure[D].Zhengzhou：Zhengzhou University，2018.
[6] 龚琦.脑电信号与注意力的关联研究[D].武汉：武汉工程大学，2017.
GONG Q.Research on extracting mind features from brain wave signal[D].Wuhan：Wuhan University of Engineering，2017.
[7] 吴欢，印想，官金安.频带能量与样本熵在注意力脑电信号中的对比研究[J].计算机与数字工程，2020，48（3）：603-606.
WU H，YIN X，GUAN J A.Comparative research of band energy and sample entropy in attention EEG signals[J].Computer and Digital Engineering，2020，48（3）：603-606.
[8] 陈群，薄华.基于深度森林的脑电注意力识别研究[J].电子设计工程，2018，26（17）：35-39.
CHEN Q，BO H.Based on deep forest attention to identify research of EEG[J].Electronic Design Engineering，2018，26（17）：35-39.
[9] KE Y F，CHEN L，FU L，et al.Visual attention recognition based on nonlinear dynamical parameters of EEG[J].Bio-medical Materials and Engineering，2014，24：349-355.
[10] MING D，ZHANG M.Multiscale entropy analysis of attention ralated EEG based on motor imaginary potential[C]//International Conference on Computational Intelligence for Measurement Systems and Applications，Hong Kong，China，May 11-13，2009：24-27.
[11] 燕楠，王珏，魏娜，等.基于样本熵的注意力相关脑电特征信息提取与分类[J].西安交通大学学报，2007，10（41）：1237-1241.
YAN N，WANG J，WEI N，et al.Feature exaction and classification of attention related electroencephalographic signals based on sample entropy[J].Journal of Xi’an Jiaotong University，2007，10（41）：1237-1241.
[12] LI J Q，SHEN J M，LIU S Q，et al.Responses of patients with disorders of consciousness to habit stimulation：a quantitative EEG study[J].Neuroscience Bulletin，2018，34（4）：691-699.
[13] 王湃，吕元杰，汪梅.一种基于智能手机与脑-机接口技术的环境控制系统[J].西安科技大学学报，2015，35（5）：656-661.
WANG P，LV Y J，WANG M.An environmental control system based on smart phone and brain computer interface technology[J].Journal of Xi’an University of Science and Technology，2015，35（5）：656-661.
[14] ZHANG M，WANG Z Y，HU H L.A new SSVEP-based BCI utilizing frequency and space to encode visual targets[J].Science China（Information Sciences），2020，63（8）：253-255.
[15] 路荣，黄力宇，晋琅.小波包分解脑电复杂性特征提取的注意状态实时识别[J].医疗卫生装备，2013，34（2）：1-4.
LU R，HUANG L Y，JIN L.Realtime recognition of attention state by complexity analysis of EEG signal using wavelet packet decomposition and support vector machine[J].Chinese Medical Equipment Journal，2013，34（2）：1-4.
[16] 梁宁建，吴明证，高旭成.基于反应时范式的内隐社会认知研究方法[J].心理科学，2003，26（2）：208-211.
LIANG N J，WU M Z，GAO X C.The research method of implicit social cognition based on the response-time paradigm[J].Psychological Science，2003，26（2）：208-211.
[17] 小雅.孩子注意力训练法：舒尔特方格法[J].中华家教，2011（9）：36.
Xiaoya.Children’s attention training method：schulte grid method[J].Family Education，2011（9）：36.
[18] ZHANG R，LI Y，YAN Y，et al.Control of a wheelchair in an indoor environment based on a brain-computer interfaceand automated navigation[J].IEEE Transactions on Neural Systems & Rehabilitation Engineering，2016，24：128.
[19] BEAR M F，CONNORS B W，PARADISO M A.Neuroscience：exploring the brain[J].Lippincott Williams & Wilkins，2007，5（138）：377-379.
[20] PFURTSCHELLER G，ALLISON B Z，BRUNNER C，et al.The hybrid BCI[J].Frontiers in Neuroscience，2010，4（30）：3.
[21] MULLER G R，BREITWIESER C，CINCOTTI F，et al.Tools for brain-computer interaction：a general concept for a hybrid BCI[J].Frontiers in Neuroinformatics，2011，5（4）：30.
[22] 余天佑.多模态与多自由度脑机接口研究[D].广州：华南理工大学，2013.
YU T Y.A study on multi-model and multiple degree-of-freedom brain-computer interfaces[D].Guangzhou：South China University of Technology，2013.
[23] PHAM T D，TRAN D.Emotion recognition using the emotiv EPOC device[C]//International Conference on Neural Information Processing.Berlin Heidelberg：Springer，2012.
[24] 潘红光，张奇，米文毓，等.基于长短期记忆网络的解码器设计及闭环脑机接口系统构建[J].西安科技大学学报，2019，39（6）：1057-1064.
PAN H G，ZHANG Q，MI W Y，et al.LSTM-based decoder design and closed-loop BMI system formulation[J].Journal of Xi’an University of Science and Technology，2019，39（6）：1057-1064.
[25] WANG J，LI H，CAI Z，et al.AutoMentor：artificial intelligent mentor in educational game[C]//International Conference on Artificial Intelligence in Education.Berlin Heidelberg：Springer，2013.
[26] 徐鲁强，刘静霞，肖光灿等.脑电注意水平的特征识别[J].计算机应用，2012，32（11）：3268-3270.
XU L Q，LIU J X，XIAO G C，et al.Characterization and classification of EEG attention level[J].Journal of? Computer Applications，2012，32（11）：3268-3270.
[27] 郭孜政，吴志敏，潘雨帆，等.基于EEG的驾驶持续性注意水平PSO-SVM识别模型[J].北京工业大学学报，2016，42（3）：427-432.
GUO Z Z，WU Z M，PAN Y F，et al.PSO-SVM identification model for driving sustained attention level based on EEG[J].Journal of Beijing University of Technology，2016，42（3）：427-432.