Survey on ERP-Based Target Detection

doi:10.3778/j.issn.1002-8331.2107-0279

Abstract

Abstract:

Target detection technology is widely used. The existing target detection methods based on computer vision are difficult to meet the needs of complex scenes due to target occlusion, light intensity and other factors. The advanced cognitive ability and rapid perception ability of the human brain have certain advantages in dealing with complex situations. Event-Related Potentials（ERP） based Brain Computer Interface（BCI） system is related to specific events. This system can detect high-level cognitive activities independent of spontaneous EEG. It has become one of the hotspots of artificial intelligence research. In this paper, the research status of target detection based on ERP signal is summarized. First, the experimental paradigm based on Rapid Serial Visual Presentation task（RSVP） is introduced, including the design factors such as presentation mode, target field of view, target complexity and so on. Then, the preprocessing methods, feature extraction and feature classification algorithms in EEG analysis are summarized, and the applications in face recognition, military operations, medical analysis and other fields are introduced. Finally, the problems and challenges in the current research are discussed, and the future research direction and application prospects are prospected.

Key words: Brain Computer Interface（BCI）, Event-Related Potentials（ERP）, target detection, Rapid Serial Visual Presentation（RSVP）, signal analysis

摘要：

目标检测技术应用广泛，现有的基于计算机视觉的目标检测方法由于目标遮挡、光照强弱等因素难以适应复杂场景的需求。而人脑的高级认知能力和快速感知能力在处理复杂情况时具有一定的优势。基于事件相关电位（Event-Related Potentials，ERP）的脑机接口（Brain Computer Interface，BCI）系统与特定事件相关，可检测独立于自发脑电的高级认知活动，是当前人工智能领域的研究热点之一。针对基于ERP信号的目标检测各个环节进行了研究现状的全面归纳，梳理了以快速串行视觉呈现（Rapid Serial Visual Presentation task，RSVP）为主的实验范式，包括呈现模式、目标视场角、目标复杂度等设计因素。总结了脑电信号分析中的预处理方法、特征提取和特征分类算法，介绍了其在人脸识别、军事作战、医学分析等领域中的初步应用。探讨了目前研究中存在的问题和挑战并展望未来的研究方向与应用前景。

关键词: 脑机接口（BCI）, 事件相关电位（ERP）, 目标检测, 快速串行视觉呈现（RSVP）, 信号分析

LI Zixing, LAN Zhen, TANG Dengqing, YAN Chao, XIANG Xiaojia, ZHOU Han. Survey on ERP-Based Target Detection[J]. Computer Engineering and Applications, 2021, 57(23): 37-49.

李子杏，兰珍，唐邓清，闫超，相晓嘉，周晗. 基于ERP信号的目标检测研究综述[J]. 计算机工程与应用, 2021, 57(23): 37-49.

References

[1] HOFFMANN U，GARCIA G，VESIN J M，et al.A boosting approach to P300 detection with application to brain-computer interfaces[C]//Proceedings of the 2nd International IEEE EMBS Conference on Neural Engineering，2005.
[2] POLICH J，DONCHIN E.P300 and the word frequency effect[J].Electroencephalography and Clinical Neurophysiology，1988，70（1）：33-45.
[3] XU N，GAO X，HONG B，et al.BCI competition 2003—data set IIb：enhancing P300 wave detection using ICA-based subspace projections for BCI applications[J].IEEE Transactions on Bio-medical Engineering，2004，51（6）：1067.
[4] WANG H，SHI B E，WANG Y.Convolutional neural network for target face detection using single-trial EEG signal[C]//Proceedings of the 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society（EMBC），2018.
[5] SAJDA P，POHLMEYER E，WANG J，et al.In a blink of an eye and a switch of a transistor：cortically coupled computer vision[J].Proceedings of the IEEE，2010，98（3）：462-478.
[6] WEIDEN M，KHOSLA D，KEEGAN M.Electroencephalographic detection of visual saliency of motion towards a practical brain-computer interface for video analysis[C]//Proceedings of the 14th ACM International Conference on Multimodal Interaction，2012.
[7] CECOTTI H，ECKSTEIN M P，GIESBRECHT B.Effects of performing two visual tasks on single-trial detection of event-related potentials[C]//Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society，2012.
[8] WANG J，POHLMEYER E，HANNA B，et al.Brain state decoding for rapid image retrieval[C]//Proceedings of the 17th ACM International Conference on Multimedia，2009.
[9] ROSENTHAL D，DEGUZMAN P，PARRA L C，et al.Evoked neural responses to events in video[J].IEEE Journal of Selected Topics in Signal Processing，2014，8（3）：358-365.
[10] 周前祥，姜国华，曲战胜，等.视场角设置对虚拟环境中目标判断的影响[J].航天医学与医学工程，2003（4）：292-295.
ZHOU Q X，JIANG G H，QU Z S，et al.Effects of field viewing angles on object jubgerent in virtual environment[J].Space Medicine & Medical Engineering，2003（4）：292-295.
[11] 周唯，吴成东.视网膜图像中的黄斑中心检测[J].中国图象图形学报，2018，23（3）：442-449.
ZHOU W，WU C D.Detection of macula fovea in a retinal image[J].Journal of Image and Graphics，2018，23（3）：442-449.
[12] DIAS J C，PARRA L C.No EEG evidence for subconscious detection during rapid serial visual presentation[C]//Proceedings of the 2011 IEEE Signal Processing in Medicine and Biology Symposium（SPMB），2011.
[13] 褚凯轩，常天庆，郭理彬，等.目标视场角对P300-RSVP目标检测系统的影响分析[J].计算机工程与应用，2018，54（23）：115-119.
CHU K X，CHANG T Q，GUO L B，et al.Analysis of influence of target perspective on P300-RSVP target detection system[J].Computer Engineering and Applications，2018，54（23）：115-119.
[14] U??UMLI? M，CHAVARRIAGA R，MILLáN J.An iterative framework for EEG-based image search：robust retrieval with weak classifiers[J].Plos One，2013，8（8）：e72018.
[15] CECOTTI H，ECKSTEIN M P，GIESBRECHT B.Single-trial classification of neural responses evoked in rapid serial visual presentation：effects of stimulus onset asynchrony and stimulus repetition[C]//Proceedings of the 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society，2014.
[16] MARATHE A R，RIES A J，LAWHERN V J，et al.The effect of target and non-target similarity on neural classification performance：a boost from confidence[J].Frontiers in Neuroscience，2015，9：270.
[17] HUANG Y，ERDOGMUS D，PAVEL M，et al.A framework for rapid visual image search using single-trial brain evoked responses[J].Neurocomputing，2011，74（12/13）：2041-2051.
[18] OKEN B S，ORHAN U，ROARK B，et al.Brain-computer interface with language model-electroencephalography fusion for locked-in syndrome[J].Neurorehabilitation and Neural Repair，2014，28（4）：387-394.
[19] LEES S，DAYAN N，CECOTTI H，et al.A review of rapid serial visual presentation-based brain-computer interfaces[J].Journal of Neural Engineering，2018，15（2）：1-23.
[20] CECOTTI H，SATO-REINHOLD J，SY J L，et al.Impact of target probability on single-trial EEG target detection in a difficult rapid serial visual presentation task[C]//Proceedings of the 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society，2011.
[21] GONSALVEZ C L，POLICH J.P300 amplitude is determined by target-to-target interval[J].Psychophysiology，2010，39（3）：388-396.
[22] CHAHINE G，KREKELBERG B.Cortical contributions to saccadic suppression[J].PLoS One，2009，4（9）：e6900.
[23] JOHNSON R.A triarchic model of P300 amplitude[J].Psychophysiology，1986：367-384.
[24] TOURYAN J，LAURIE G，HORNE J H，et al.Real-time measurement of face recognition in rapid serial visual presentation[J].Frontiers in Psychology，2011，2：42.
[25] CAI B，XIAO S，JIANG L，et al.A rapid face recognition BCI system using single-trial ERP[C]//Proceedings of the 2013 6th International IEEE/EMBS Conference on Neural Engineering（NER），2013.
[26] FUHRMANN ALPERT G，MANOR R，SPANIER A B，et al.Spatiotemporal representations of rapid visual target detection：a single-trial EEG classification algorithm[J].IEEE Transactions on Biomedical Engineering，2014，61（8）：2290-2303.
[27] MANOR R，GEVA A B.Convolutional neural network for multi-category rapid serial visual presentation BCI[J].Frontiers in Computational Neuroscience，2015，9：146.
[28] HUANG L，ZHAO Y，ZENG Y，et al.BHCR：RSVP target retrieval BCI framework coupling with CNN by a Bayesian method[J].Neurocomputing，2017，238：255-268.
[29] LE S，EPPS J.Classifying EEG for brain-computer interfaces：learning optimal filters for dynamical system features[C]//Proceedings of the 23rd International Conference on Machine Learning，2006.
[30] JOHNSON G D，KRUSIENSKI D J.Ensemble SWLDA classifiers for the P300 speller[C]//Proceedings of the International Conference on Human-Computer Interaction.Berlin，Heidelberg：Springer，2009.
[31] BLANKERTZ B，LOSCH F，KRAULEDAT M，et al.The Berlin brain?computer interface：accurate performance from first-session in BCI-naive subjects[J].IEEE Transactions on Biomedical Engineering，2008，55（10）：2452-2462.
[32] SUBASI A，GURSOY M I.EEG signal classification using PCA，ICA，LDA and support vector machines[J].Expert Systems with Applications，2010，37（12）：8659-8666.
[33] MCFARLAND D J，MCCANE L M，DAVID S V，et al.Spatial filter selection for EEG-based communication[J].Electroencephalography Clinical Neurophysiology，1997，103（3）：386-394.
[34] LINDGREN J T.As above，so below? Towards understanding inverse models in BCI[J].Journal of Neural Engineering，2017，15（1）：1-12.
[35] ALHADDAD M J.Common average reference（CAR） improves P300 speller[J].International Journal of Engineering & Technology，2012，2（3）.
[36] COHEN M X.Comparison of different spatial transformations applied to EEG data：a case study of error processing[J].International Journal of Psychophysiology，2021，97（3）：245-257.
[37] TSUCHIMOTO S，SHIBUSAWA S，IWAMA S，et al.Use of common average reference and large-Laplacian spatial-filters enhances EEG signal-to-noise ratios in intrinsic sensorimotor activity[J].Journal of Neuroscience Methods，2021：109089.
[38] RIVET B，SOULOUMIAC A，ATTINA V，et al.xDAWN algorithm to enhance evoked potentials：application to brain-computer interface[J].IEEE Transactions on Bio-medical Engineering，2009，56（8）：2035-2043.
[39] KACHENOURA A，ALBERA L，SENHADJI L，et al.ICA：a potential tool for BCI systems[J].IEEE Signal Processing Magazine，2007，25（1）：57-68.
[40] COMON P.Independent component analysis，a new concept?[J].Signal Processing，1994，36（3）：287-314.
[41] BELL A J，SEJNOWSKI T J.An information-maximization approach to blind separation and blind deconvolution[J].Neural Computation，1995，7（6）：1129-1159.
[42] HYVARINEN A，OJA E.A fast fixed-point algorithm for independent component analysis[J].Neural Computation，1997，9（7）：1483-1492.
[43] MAMMONE N，LA FORESTA F，MORABITO F C.Automatic artifact rejection from multichannel scalp EEG by wavelet ICA[J].IEEE Sensors Journal，2012，12（3）：533-542.
[44] COSTA J，DA-SILVA P，ALMEIDA R，et al.Validation in principal components analysis applied to EEG data[J].Computational Mathematical Methods in Medicine，2014：413801.
[45] BLANKERTZ B，TOMIOKA R，LEMM S，et al.Optimizing spatial filters for robust EEG single-trial analysis[J].IEEE Signal Processing Magazine，2007，25（1）：41-56.
[46] SPULER M，WALTER A，ROSENTHAL W，et al.Spatial filtering based on canonical correlation analysis for classification of evoked or event-related potentials in EEG data[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering，2014，22（6）：1097-1103.
[47] HOTELLING H.Relations between 2 sets of variants[J].Biometrika，1935，28（3/4）：312-377.
[48] DíAZ C F B，OLAYA A F R.A novel method based on regularized logistic regression and CCA for P300 detection using a reduced number of EEG trials[J].IEEE Latin America Transactions，2020，18（12）：2147-2154.
[49] MAHANTA M S，AGHAEI A S，PLATANIOTIS K N.A Bayes optimal matrix-variate LDA for extraction of spatio-spectral features from EEG signals[C]//Proceedings of the 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society（EMBC），2012.
[50] SHUKLA P，CHAURASIYA R，VERMA S，et al.A thresholding-free state detection approach for home appliance control using P300-based BCI[J].IEEE Sensors Journal，2021：2.
[51] ECH-CHOUDANY Y，SCIDA D，ASSARAR M，et al.Dissimilarity-based time-frequency distributions as features for epileptic EEG signal classification[J].Biomedical Signal Processing and Control，2021，64：102268.
[52] ZHANG Y，CHEN W，LIN C L，et al.Boosting-LDA algriothm with multi-domain feature fusion for motor imagery EEG decoding[J].Biomedical Signal Processing and Control，2021，70：102983.
[53] EGAMBARAM A，BADRUDDIN N，ASIRVADAM V S，et al.Online detection and removal of eye blink artifacts from electroencephalogram[J].Biomedical Signal Processing and Control，2021，69：102887.
[54] LEMM S，BLANKERTZ B，CURIO G，et al.Spatio-spectral filters for improving the classification of single trial EEG[J].IEEE Transactions on Biomedical Engineering，2005，52（9）：1541-1548.
[55] DORNHEGE G.Combined optimization of spatial and temporal filters for improving brain-computer interfacing[J].IEEE Transactions on Bio-medical Engineering，2006，53（11）：2274.
[56] YU K，SHEN K，SHAO S，et al.Common spatio-temporal pattern for single-trial detection of event-related potential in rapid serial visual presentation triage[J].IEEE Transactions on Bio-medical Engineering，2011，58（9）：2513.
[57] YU K，SHEN K，SHAO S，et al.Bilinear common spatial pattern for single-trial ERP-based rapid serial visual presentation triage[J].Journal of Neural Engineering，2012，9（4）：6013.
[58] SAJDA P，GERSON A，PARRA L.High-throughput image search via single-trial event detection in a rapid serial visual presentation task[C]//Proceedings of the First International IEEE EMBS Conference on Neural Engineering，2003.
[59] MARATHE A R，RIES A J，MCDOWELL K.Sliding HDCA：single-trial EEG classification to overcome and quantify temporal variability[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering，2014，22（2）：201-211.
[60] MATHAN S，WHITLOW S，MAZAEVA N.Sensor-based cognitive state assessment in a mobile environment[C]//Proceedings of the 11th International Conference on Human-Computer Interaction，2005.
[61] XIAO X，XU M，JIN J，et al.Discriminative canonical pattern matching for single-trial classification of ERP components[J].IEEE Transactions on Biomedical Engineering，2020，67（8）：2266-2275.
[62] CECOTTI H，GRASER A.Convolutional neural networks for P300 detection with application to brain-computer interfaces[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2011，33（3）：433.
[63] JOSHI R B，GOEL P，SUR M，et al.Single trial P300 classification using convolutional LSTM and deep learning ensembles method[C]//Proceedings of the International Conference on Intelligent Human Computer Interaction，2018.
[64] LAWHERN V J，SOLON A J，WAYTOWICH N R，et al.EEGNet：a compact convolutional network for eeg-based brain-computer interfaces[J].Journal of Neural Engineering，2018，15（5）：1-17.
[65] LIU M F，WU W，GU Z，et al.Deep learning based on batch normalization for P300 signal detection[J].Neurocomputing，2018，275：288-297.
[66] LAN Z，YAN C，LI Z，et al.MACRO：multi-attention convolutional recurrent model for subject-independent ERP detection[J].IEEE Signal Processing Letters，2021，28：1505-1509.
[67] MA R，YU T，ZHONG X，et al.Capsule network for ERP detection in brain-computer interface[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering，2021，29：718-730.
[68] WANG Y，JUNG T P.A collaborative brain-computer interface for improving human performance[J].Plos One，2011，6（5）：e20422.
[69] SANGINETO E，ZEN G，RICCI E，et al.We are not all equal：personalizing models for facial expression analysis with transductive parameter transfer[C]//Proceedings of the 22nd ACM International Conference on Multimedia，Orlando，Florida，USA，2014：357-366.
[70] WAYTOWICH N R，LAWHERN V J，BOHANNON A W，et al.Spectral transfer learning using information geometry for a user-independent brain-computer interface[J].Frontiers in Neuroscience，2016，10（198）：430.
[71] WEI C S，LIN Y P，WANG Y T，et al.Transfer learning with large-scale data in brain-computer interfaces[C]//Proceedings of the 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society（EMBC），2016.
[72] KUNDU S，ARI S.MsCNN：a deep learning framework for P300-based brain-computer interface speller[J].IEEE Transactions on Medical Robotics and Bionics，2020，2（1）：86-93.
[73] XIAO G，MA Y，LIU C L，et al.A machine emotion transfer model for intelligent human-machine interaction based on group division[J].Mechanical Systems and Signal Processing，2020，142：106736.
[74] ZHANG C，QIU S，WANG S，et al.Target detection using ternary classification during a rapid serial visual presentation task using magnetoencephalography data[J].Frontiers in Computational Neuroscience，2021，15（9）：619508.
[75] BOWEN L，YANFEI L，XIAORONG G，et al.Enhancing the EEG classification in RSVP task by combining interval model of ERPs with spatial and temporal regions of interest[J].Journal of Neural Engineering，2021，18（1）：016008.
[76] XU M，CHEN Y，WANG D，et al.Multi-objective optimization approach for channel selection and cross-subject generalization in RSVP-based BCIs[J].Journal of Neural Engineering，2021，18（4）：046076.
[77] XIYU S，BIN Y，LI T，et al.Asynchronous video target detection based on single-trial EEG signals[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering，2020，28（9）：1931-1943.
[78] WEI W，SHUANG Q，XUELIN M，et al.Reducing calibration efforts in RSVP tasks with multi-source adversarial domain adaptation[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering，2020，28（11）：2344-2355.
[79] MIYANO H.Modeling the attentional blink：a simple logistic model with first-order difference system[J].Journal of Mathematical Psychology，2021，101：102507.
[80] JALILPOUR S，SARDOUIE S H.RCTP：regularized common tensor pattern for rapid serial visual presentation spellers[J].Biomedical Signal Processing and Control，2021，70：102930.
[81] QU L，CHEN D，YIN K.Research on EEG feature decoding based on stimulus image[C]//Proceedings of the 2021 IEEE 4th Advanced Information Management，Communicates，Electronic and Automation Control Conference（IMCEC），Chongqing，China，2021.
[82] LEES S，MCCULLAGH P，PAYNE P，et al.Speed of rapid serial visual presentation of pictures，numbers and words affects event-related potential-based detection accuracy[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering，2020，28（1）：113-122.
[83] PANWAR S，RAD P，JUNG T P，et al.Modeling EEG data distribution with a wasserstein generative adversarial network to predict RSVP events[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering，2020，28（8）：1720-1730.
[84] AMRI M F，YULIANI A R，KUSUMANDARI D E，et al.Feature extraction on brain wave activities in rapid serial visual presentation stimulus[C]//Proceedings of the 2020 International Conference on Radar，Antenna，Microwave，Electronics，and Telecommunications（ICRAMET），Tangerang，Indonesia，2020.
[85] SANTAMARIA-VAZQUEZ E，MARTINEZ-CAGIGAL V，VAQUERIZO-VILLAR F，et al.EEG-inception：a novel deep convolutional neural network for assistive ERP-based brain-computer interfaces[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering，2020，28（12）：1.
[86] KAPOOR A，SHENOY P，TAN D S.Combining brain computer interfaces with vision for object categorization[C]//Proceedings of the 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition，2008.
[87] TOURYAN J，GIBSON L，HORNE J H，et al.Real-time measurement of face recognition in rapid serial visual presentation[J].Frontiers in Psychology，2011，2（4）：42.
[88] ZENG Y，WU Q，YANG K，et al.EEG-based identity authentication framework using face rapid serial visual presentation with optimized channels[J].Sensors，2019，19（1）：6.
[89] ROLLINS L，OLSEN A，EVANS M.Social categorization modulates own-age bias in face recognition and ERP correlates of face processing[J].Neuropsychologia，2020，141：107417.
[90] GUILLAUME F，THOMAS é.Recollection and familiarity in schizophrenia：an ERP investigation using face recognition exclusion tasks[J].Psychiatry Research，2021，302：113973.
[91] HERZOG C S.Brain-in-the-loop integration：CT2WS enters a new phase[J].Diplomatic Courier，2012.
[92] BARNGROVER C，ALTHOFF A，DEGUZMAN P，et al.A Brain-Computer Interface（BCI） for the detection of mine-like objects in sidescan sonar imagery[J].IEEE Journal of Oceanic Engineering，2016，41（1）：123-138.
[93] MATRAN-FERNANDEZ A，POLI R.Brain-computer interfaces for detection and localisation of targets in aerial images[J].IEEE Transactions on Bio-medical Engineering，2016，64（4）：959-969.
[94] LEVIN E，ROLAND W，HABIBI R，et al.Rapid visual presentation to support geospatial big data processing[J].The International Archives of the Photogrammetry，Remote Sensing and Spatial Information Sciences，2020：463-470.
[95] IBá?EZ-MOLINA A J，LOZANO V,SORIANO M F，et al.EEG multiscale complexity in schizophrenia during picture naming[J].Frontiers in Physiology，2018，9：1213.
[96] KADAMBI P，LOVELACE J A，BEYETTE F R.Audio based brain computer interfacing for neurological assessment of fatigue[C]//Proceedings of the 6th International IEEE/EMBS Conference on Neural Engineering（NER），2013.
[97] WANG C，RAPP P，DARMON D，et al.Utility of P300 ERP in monitoring post-trauma mental health：a longitudinal study in military personnel returning from combat deployment[J].Journal of Psychiatric Research，2018，101：5-13.
[98] ZOKAEI N，GILLEBERT C R，CHAUVIN J J，et al.Temporal orienting in Parkinson’s disease[J].The European Journal of Neuroscience，2021，53（8）：2713-2725.
[99] HOPE C，STERR A，ELANGOVAN P，et al.High throughput screening for mammography using a human-computer interface with rapid serial visual presentation（RSVP）[C]//Proceedings of the Medical Imaging 2013：Image Perception，Observer Performance，and Technology Assessment International Society for Optics and Photonics，2013.
[100] SHALBAF A，BAGHERZADEH S，MAGHSOUDI A.Transfer learning with deep convolutional neural network for automated detection of schizophrenia from EEG signals[J].Australasian Physical Engineering Sciences in Medicine，2020，43（7）：1229-1239.