Review of Codec Algorithms for Motor Imagery Brain-Computer Interface in Embedded System

doi:10.3778/j.issn.1002-8331.2401-0199

Abstract

Abstract: Brain-machine interface technology establishes a communication pathway between the brain and external devices, enabling users to directly control these devices. In recent years, research on encoding and decoding algorithms for brain-machine interfaces based on the motor imagery paradigm has found increasing applications in healthcare, education, entertainment, and everyday life devices. These algorithms often need to be embedded into hardware devices to meet the requirements of practical applications. Therefore, this paper introduces the research status of brain-computer interface codec algorithms for motion imagination in embedded systems in recent years, and points out their advantages and disadvantages from two perspectives of traditional machine learning algorithms and deep learning algorithms. Then, it focuses on the four types of commonly used embedded platform representative equipment and their advantages and disadvantages, and gives corresponding hardware selection suggestions for different application scenarios. In addition, this paper summarizes the evaluation indicators that are more suitable for embedded brain-computer interface systems, and finally summarizes the existing challenges and future development directions in the field.

Key words: brain-computer interface （BCI）, motor imagery, EEG signal encoding and decoding algorithms, embedded system

摘要： 脑-机接口技术通过在大脑与外部设备之间建立信息传输通路，使用户能够对外部设备进行直接控制。近年来，基于运动想象范式的脑-机接口编解码算法研究在医疗健康、教育娱乐及日常生活设备中的应用范围越来越广，这些算法通常需要嵌入到硬件设备中来满足实际应用的需求。介绍了近年来嵌入式系统中运动想象脑-机接口编解码算法研究现状，从传统机器学习算法和深度学习算法两个角度指出其对应的优缺点。重点介绍四类常用嵌入式平台的代表性设备及其优缺点，并针对不同的应用场景给出相应的硬件选型建议。归纳了更适用于嵌入式脑-机接口系统的评价指标并最终总结了领域内现存的挑战与未来发展方向。

关键词: 脑-机接口, 运动想象, 脑电信号编解码算法, 嵌入式系统

YU Qinwen, ZHOU Wangcheng, DAI Yakang, LIU Yan. Review of Codec Algorithms for Motor Imagery Brain-Computer Interface in Embedded System[J]. Computer Engineering and Applications, 2024, 60(18): 50-65.

于钦雯, 周王成, 戴亚康, 刘燕. 嵌入式系统中运动想象脑-机接口编解码算法综述[J]. 计算机工程与应用, 2024, 60(18): 50-65.

References

[1] ARPAIA P, ESPOSITO A, NATALIZIO A, et al. How to successfully classify EEG in motor imagery BCI: a metrological analysis of the state of the art[J]. Journal of Neural Engineering, 2022, 19(3): 031002.
[2] ALTAHERI H, MUHAMMAD G, ALSULAIMAN M, et al. Deep learning techniques for classification of electroencephalogram (EEG) motor imagery (MI) signals: a review[J]. Neural Computing and Applications, 2023, 35(20): 14681-14722.
[3] PALUMBO A, IELPO N, CALABRESE B. An FPGA-embedded brain-computer interface system to support individual autonomy in locked-in individuals[J]. Sensors, 2022, 22(1): 318.
[4] KRANA M, FARMAKI C, PEDIADITIS M, et al. SSVEP based wheelchair navigation in outdoor environments[C]//2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 2021: 6424-6427.
[5] VILELA M, HOCHBERG L R. Applications of brain-computer interfaces to the control of robotic and prosthetic arms[J]. Handbook of Clinical Neurology, 2020, 168: 87-99.
[6] TAMILARASI S, SUNDARARAJAN J. FPGA based seizure detection and control for brain computer interface[J]. Cluster Computing, 2019, 22(5): 11841-11848.
[7] MANE R, CHOUHAN T, GUAN C. BCI for stroke rehabilitation: motor and beyond[J]. Journal of Neural Engineering, 2020, 17(4): 041001.
[8] SHAO L, ZHANG L, BELKACEM A N, et al. EEG-controlled wall-crawling cleaning robot using SSVEP-based brain-computer interface[J]. Journal of Healthcare Engineering, 2020：6968713.
[9] KOBAYASHI N, ISHIZUKA K. LSTM-based classification of multiflicker-SSVEP in single channel dry-EEG for low-power/high-accuracy quadcopter-BMI system[C]//2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), 2019: 2160-2165.
[10] MKNIERIM M. OpenBCI+cEEGrids[EB/OL].[2021-03-18]. https://openbci.com/community/openbci-ceegrids/.
[11] XIE Y, MAJOROS T, ONIGA S. FPGA-based hardware accelerator on portable equipment for EEG signal patterns recognition[J]. Electronics, 2022, 11(15): 2410.
[12] SOHAL H, JAIN S. FPGA implementation of low power pre-processor design for biomedical signal processing application[M]//Computational and experimental methods in mechanical engineering. Singapore： Springer, 2022: 489-497.
[13] SEOK D, LEE S, KIM M, et al. Motion artifact removal techniques for wearable EEG and PPG sensor systems[J]. Frontiers in Electronics, 2021, 2: 685513.
[14] JAFARI A, GANDHI S, KONURU S H, et al. An EEG artifact identification embedded system using ICA and multi-instance learning[C]//2017 IEEE International Symposium on Circuits and Systems (ISCAS), 2017: 1-4.
[15] LIN C T, LIN F C, CHEN S A, et al. EEG-based brain-computer interface for smart living environmental auto-adjustment[J]. Journal of Medical and Biological Engineering, 2010, 30(4): 237-245.
[16] KHURANA K, GUPTA P, PANICKER R C, et al. Development of an FPGA-based real-time P300 speller[C]//22nd International Conference on Field Programmable Logic and Applications (FPL), 2012: 551-554.
[17] LI G, CHUNG W Y. Combined EEG-gyroscope-tDCS brain machine interface system for early management of driver drowsiness[J]. IEEE Transactions on Human-Machine Systems, 2017, 48(1): 50-62.
[18] BELWAFI K, GHAFFARI F, DJEMAL R, et al. A hardware/software prototype of EEG-based BCI system for home device control[J]. Journal of Signal Processing Systems, 2017, 89: 263-279.
[19] SHAKEEL A, TANAKA T, KITAJO K. Time-series prediction of the oscillatory phase of EEG signals using the least mean square algorithm-based AR model[J]. Applied Sciences, 2020, 10(10): 3616.
[20] MALEKMOHAMMADI A, MOHAMMADZADE H, CHAMANZAR A, et al. An efficient hardware implementation for a motor imagery brain computer interface system[J]. Scientia Iranica, 2019, 26: 72-94.
[21] MARCOS R C L, CHAILLOUX P J D, ALBA BLANCO E. Real time identification of motor imagery actions on EEG signals[J]. Ingeniería Electrónica, Automáticay Comunicaciones, 2020, 41(1): 101-117.
[22] ASANZA V, CONSTANTINE A, VALAREZO S, et al. Implementation of a classification system of EEG signals based on FPGA[C]//2020 Seventh International Conference on Democracy & eGovernment (ICEDEG), 2020: 87-92.
[23] PRONO L, MARCHIONI A, MANGIA M, et al. An MCU implementation of PCA/PSA streaming algorithms for EEG features extraction[C]//2021 IEEE Biomedical Circuits and Systems Conference (BioCAS), 2021: 1-5.
[24] VASEI T, SABER M A, NAHVY A, et al. An efficient RTL design for a wearable brain-computer interface[J]. IET Computers & Digital Techniques, 2024.
[25] DORNHEGE G, BLANKERTZ B, KRAULEDAT M, et al. Combined optimization of spatial and temporal filters for improving brain-computer interfacing[J]. IEEE Transactions on Biomedical Engineering, 2006, 53(11): 2274-2281.
[26] WU W, GAO X, HONG B, et al. Classifying single-trial EEG during motor imagery by iterative spatio-spectral patterns learning (ISSPL)[J]. IEEE Transactions on Biomedical Engineering, 2008, 55(6): 1733-1743.
[27] ZHANG H, CHIN Z Y, ANG K K, et al. Optimum spatio-spectral filtering network for brain-computer interface[J]. IEEE Transactions on Neural Networks, 2010, 22(1): 52-63.
[28] 王亮. 基于FPGA的运动想象脑电信号的分类识别研究[D].淮南：安徽理工大学, 2020.
WANG L. Based on the movement of the FPGA imagine EEG classification research[D]. Huainan：Anhui University of Science and Technology, 2020.
[29] ELLAWALA N, THAYAPARAN S. Hardware implementation of EEG classifier using LDA[C]//2019 2nd International Conference on Bioinformatics, Biotechnology and Biomedical Engineering (BioMIC)-Bioinformatics and Biomedical Engineering, 2019: 1-5.
[30] RIZAL A, HADIYOSO S, RAMDANI A Z. FPGA-based implementation for real-time epileptic EEG classification using hjorth descriptor and KNN[J]. Electronics, 2022, 11(19): 3026.
[31] LYU S, CHOWDHURY M H, CHEUNG R C C. Efficient hardware and software co-design for EEG signal classification based on extreme learning machine[C]//2023 International Conference on Electrical, Computer and Communication Engineering (ECCE), 2023: 1-5.
[32] MOHEBBANAAZ, RAJANI KUMARI L V, PADMA SAI Y. Classification of arrhythmia beats using optimized K-nearest neighbor classifier[C]//Intelligent Systems: Proceedings of ICMIB 2020. Singapore :Springer, 2021: 349-359.
[33] EKAPUTRI C, FUADAH Y N, PRATIWI N K C, et al. Drowsiness detection based on EEG signal using discrete wavelet transform (DWT) and K-Nearest Neighbors (K-NN) methods[C]//Proceedings of the 1st International Conference on Electronics, Biomedical Engineering, and Health Informatics (ICEBEHI 2020), Surabaya, Indonesia, October 8-9 2020. Singapore :Springer, 2021: 487-498.
[34] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
[35] NIKITAKIS A, MAKANTASIS K, TAMPOURATZIS N, et al. A unified novel neural network approach and a prototype hardware implementation for ultra-low power EEG classification[J]. IEEE Transactions on Biomedical Circuits and Systems, 2019, 13(4): 670-681.
[36] WANG W, HUANG Y, WANG Y, et al. Generalized autoencoder: a neural network framework for dimensionality reduction[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2014: 490-497.
[37] LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521(7553): 436-444.
[38] MWATA-VELU T, RUIZ-PINALES J, ROSTRO-GONZALEZ H, et al. Motor imagery classification based on a recurrent-convolutional architecture to control a hexapod robot[J]. Mathematics, 2021, 9(6): 606.
[39] MA X, ZHENG W, PENG Z, et al. FPGA-based rapid electroencephalography signal classification system[C]//2019 IEEE 11th International Conference on Advanced Infocomm Technology (ICAIT), 2019: 223-227.
[40] MAITI S, ANAMIKA, MANDAL A S, et al. Classification of motor imagery EEG signal for navigation of brain controlled drones[C]//Proceedings of 11th International Conference on Intelligent Human Computer Interaction (IHCI 2019), Allahabad, India, December 12-14, 2019. [S.l.]: Springer International Publishing, 2020: 3-12.
[41] LAWHERN V J, SOLON A J, WAYTOWICH N R, et al. EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces[J]. Journal of Neural Engineering, 2018, 15(5): 056013.
[42] MWATA-VELU T, NIYONSABA-SEBIGUNDA E, AVINA-CERVANTES J G, et al. Motor imagery multi-tasks classification for BCIs using the NVIDIA Jetson TX2 board and the EEGNet network[J]. Sensors, 2023, 23(8): 4164.
[43] TORRES-VALVERDE L, IMAMOGLU N, GONZALEZ-TORRES A, et al. Evaluation of neural networks with data quantization in low power consumption devices[C]//2020 IEEE 11th Latin American Symposium on Circuits & Systems (LASCAS), 2020: 1-4.
[44] INGOLFSSON T M, HERSCHE M, WANG X, et al. EEG-TCNet: an accurate temporal convolutional network for embedded motor-imagery brain-machine interfaces[C]//2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2020: 2958-2965.
[45] SALAMI A, ANDREU-PEREZ J, GILLMEISTER H. EEG-ITNet: an explainable inception temporal convolutional network for motor imagery classification[J]. IEEE Access, 2022, 10: 36672-36685.
[46] MA W, GONG Y, XUE H, et al. A lightweight and accurate double-branch neural network for four-class motor imagery classification[J]. Biomedical Signal Processing and Control, 2022, 75: 103582.
[47] JIA Z, LIN Y, WANG J, et al. MMCNN: a multi-branch multi-scale convolutional neural network for motor imagery classification[C]//Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Cham:Springer, 2020: 736-751.
[48] ZOU X, GUO Z, CHENG Q, et al. An enhanced convolution neural network model based on depthwise-separable convolutions for pneumonia classification[C]//2022 12th International Conference on Information Technology in Medicine and Education (ITME), 2022: 423-428.
[49] ALTUWAIJRI G A, MUHAMMAD G. A multibranch of convolutional neural network models for electroencephalogram-based motor imagery classification[J]. Biosensors, 2022, 12(1): 22.
[50] LEE H K, MYOUNG J S, CHOI Y S. A lightweight end-to-end neural networks for decoding of motor imagery brain signal[C]//2022 Thirteenth International Conference on Ubiquitous and Future Networks (ICUFN), 2022: 411-413.
[51] PULLINI A, ROSSI D, LOI I, et al. Mr. Wolf: an energy-precision scalable parallel ultra low power SoC for IoT edge processing[J]. IEEE Journal of Solid-State Circuits, 2019, 54(7): 1970-1981.
[52] WANG X, HERSCHE M, T?MEKCE B, et al. An accurate eegnet-based motor-imagery brain-computer interface for low-power edge computing[C]//2020 IEEE International Symposium on Medical Measurements and Applications (MeMeA), 2020: 1-6.
[53] 唐旭东. 基于卷积神经网络与微控制器的便携式运动想象脑电信号解码器研究[D].杭州: 杭州电子科技大学, 2023.
TANG X D. Research on portable motion imagination EEG signal decoder based on convolutional neural networks and microcontrollers[D]. Hangzhou: Hangzhou Dianzi University, 2023.
[54] SUDHARSAN B, SALERNO S, NGUYEN D D, et al. TinyML benchmark: executing fully connected neural networks on commodity microcontrollers[C]//World Forum on Internet of Things (WF-IoT), 2021:883-884.
[55] RIYADI M A, PRAKOSO T, WHAILLAN F O, et al. Classification of EEG-based brain waves for motor imagery using support vector machine[C]//2019 International Conference on Electrical Engineering and Computer Science (ICECOS), 2019: 422-425.
[56] SUBRATA A C, RIYADI M A, PRAKOSO T. EEG-based BMI using multi-class motor imagery for bionic arm[C]//2020 3rd International Conference on Mechanical, Electronics, Computer, and Industrial Technology (MECnIT), 2020: 255-260.
[57] SCHNEIDER T, WANG X, HERSCHE M, et al. Q-EEGNet: an energy-efficient 8-bit quantized parallel EEGNet implementation for edge motor-imagery brain-machine interfaces[C]//2020 IEEE International Conference on Smart Computing (SMARTCOMP), 2020: 284-289.
[58] WANG X, HERSCHE M, MAGNO M, et al. MI-BMInet: an efficient convolutional neural network for motor imagery brain--machine interfaces with EEG channel selection[J]. arXiv:2203.14592, 2022.
[59] KORHAN N, DOKUR Z, OLMEZ T. Motor imagery based EEG classification by using common spatial patterns and convolutional neural networks[C]//2019 Scientific Meeting on Electrical-Electronics & Biomedical Engineering and Computer Science (EBBT), 2019: 1-4.
[60] LIN J S, SHE B H. A BCI system with motor imagery based on bidirectional long-short term memory[C]//IOP Conference Series: Materials Science and Engineering.[S.l.]: IOP Publishing, 2020: 012026.
[61] DARVISH R B, LO D, ZHAO R, et al. Pushing the limits of narrow precision inferencing at cloud scale with microsoft floating point[C]//Advances in Neural Information Processing Systems, 2020: 10271-10281.
[62] LIN Q, SONG S, CASTRO I D, et al. Wearable multiple modality bio-signal recording and processing on chip: a review[J]. IEEE Sensors Journal, 2020, 21(2): 1108-1123.
[63] HERNANDEZ-RUIZ A C, ENéRIZ D, MEDRANO N, et al. Motor-imagery EEGNet-based processing on a low-Spec SoC hardware[C]//2021 IEEE Sensors, 2021: 1-4.
[64] TSUKAHARA A, ANZAI Y, TANAKA K, et al. A design of EEGNet‐based inference processor for pattern recognition of EEG using FPGA[J]. Electronics and Communications in Japan, 2021, 104(1): 53-64.
[65] FLOOD D, ROBINSON N, SHREEJITH S. FPGA-based deep-learning accelerators for energy efficient motor imagery EEG classification[C]//2022 IEEE International Conference on Omni-Layer Intelligent Systems (COINS), 2022: 1-6.
[66] 任超. 基于深度学习的脑电识别算法研究及FPGA实现[D]. 西安:西安理工大学, 2021.
REN C. Research on EEG recognition algorithm based on deep learning and FPGA implementation[D]. Xi’an: Xi’an University of Technology, 2021.
[67] LOTTE F, CONGEDO M, LéCUYER A, et al. A review of classification algorithms for EEG-based brain-computer interfaces[J]. Journal of Neural Engineering, 2007, 4(2): R1.
[68] KüBLER A, NEUMANN N, KAISER J, et al. Brain-computer communication: self-regulation of slow cortical potentials for verbal communication[J]. Archives of Physical Medicine and Rehabilitation, 2001, 82(11): 1533-1539.
[69] BELWAFI K, GANNOUNI S, ABOALSAMH H. Embedded brain computer interface: state-of-the-art in research[J]. Sensors, 2021, 21(13): 4293.
[70] SPEIER W, ARNOLD C, POURATIAN N. Evaluating true BCI communication rate through mutual information and language models[J]. PLoS One, 2013, 8(10): e78432.