Progress in Human Lower Extremity Motion Recognition and Prediction Based on Surface Electromyography

doi:10.3778/j.issn.1002-8331.2209-0145

Abstract

Abstract: Surface electromyography is an important way to analyze human behavior intention. Driven by deep learning, surface electromyography has made great progress in human lower limb action recognition and prediction. However, EMG signals face the defects of poor anti-interference and inability to record deep muscles, which brings huge difficulties to the later study of limb movements under surface EMG. This paper summarizes and summarizes the research progress of body movements under surface EMG by scholars at home and abroad in recent years, and analyzes it from four aspects：lower limb EMG data acquisition, signal processing method, feature extraction development, and training model. The experimental results of related methods are compared comprehensively, and relevant conclusions are put forward. Finally, the shortcomings of the current research are summarized and personal opinions are put forward, in order to provide more theoretical basis for the application of limb recognition under surface electromyography.

Key words: surface electromyography, lower body movement prediction, deep learning

摘要： 表面肌电技术是人体行为意图分析的重要方式。在深度学习的推动下，表面肌电在人体下肢动作识别预测上取得了很大的进展。然而，肌电信号面临着抗干扰性差、无法直接提取等缺陷，从而给后期的表面肌电下肢体动作研究带来巨大的困难。对近年来国内外学者在表面肌电下肢体动作研究进展总结归纳，从下肢体肌电数据采集、信号处理方式、特征提取发展、训练模型四个方面进行分析。对相关方法的实验结果进行综合比较，并提出归纳总结。最后对当前研究的不足之处进行了总结并提出建议，以期为表面肌电下肢体识别的应用提供更多的理论依据。

关键词: 表面肌电, 下肢体动作预测, 深度学习

DONG Zeping, QIU Dawei, LIU Jing. Progress in Human Lower Extremity Motion Recognition and Prediction Based on Surface Electromyography[J]. Computer Engineering and Applications, 2023, 59(14): 39-50.

董泽萍, 仇大伟, 刘静. 表面肌电人体下肢动作识别预测进展[J]. 计算机工程与应用, 2023, 59(14): 39-50.

References

[1] OSHIMA A，NAKAMURA Y，KAMIBAYASHI K.Modulation of muscle synergies in lower-limb muscles associated with split-belt locomotor adaptation[J].Frontiers in Human Neuroscience，2022，16：852530.
[2] 雷东威，陈彬，雷静桃.穿戴式外骨骼机器人肌电信号采集与处理[J].计量与测试技术，2021，48（12）：11-15.
LEI D W，CHEN B，LEI J T.Wearable exoskeleton robot EMG signal acquisition and processing[J].Metrology & Measurement Technique，2021，48（12）：11-15.
[3] 杜义浩，邱石，谢平，等.下肢康复机器人的自适应人机交互控制策略[J].自动化学报，2018，44（4）：743-750.
DU Y H，QIU S，XIE P，et al.Adaptive interaction control for lower limb rehabilitation robots[J].Acta Automatica Sinica，2018，44（4）：734-750.
[4] HONG J C，YASUDA K，OHASHI H，et al.Optimal assistance timing to induce voluntary dorsiflexion movements：a preliminary study in healthy participants[J].Applied Sciences，2022，2（4）：2248.
[5] 张龙娇，曾晓勤.基于深度神经网络的sEMG手势识别研究[J].计算机工程与应用，2019，55（23）：113-119.
ZHANG L J，ZENG X Q.Research on gesture recognition of sEMG based on deep neural network[J].Computer Engineering and Applications，2019，55（23）：113-119.
[6] JARAMILLO-YáNEZ A，BENALCáZAR M E，MENA-MALDONADO E.Real-time hand gesture recognition using surface electromyography and machine learning：a systematic literature review[J].Sensors，2020，20（9）：2467.
[7] XIONG D，ZHANG D，ZHAO X，et al.Deep learning for EMG-based human-machine interaction：a review[J].IEEE/CAA Journal of Automatica Sinica，2021，8（3）：512-533.
[8] 孙影影，贾振堂，朱昊宇.多模态深度学习综述[J].计算机工程与应用，2020，56（21）：1-10.
SUN Y Y，JIA Z T，ZHU H Y.Survey of multimodal deep learning[J].Computer Engineering and Applications，2020，56（21）：1-10.
[9] WANG J，WANG L，XI X，et al.Estimation and correlation analysis of lower limb joint angles based on surface electromyography[J].Electronics，2020，9（4）：556.
[10] KUANG Y，WU Q，SHAO J，et al.Extreme learning machine classification method for lower limb movement recognition[J].Cluster Computing，2017，20（4）：3051-3059.
[11] ZHANG F，LI P，HOU Z G，et al.sEMG-based continuous estimation of joint angles of human legs by using BP neural network[J].Neurocomputing，2012，78（1）：139-148.
[12] 刘万阳，李晓欧.基于虚拟现实的肌电-计算机接口技术[J].生物医学工程研究，2018，37（2）：238-242.
LIU W Y，LI X O.Emg-computer interface technology based on virtual reality[J].Biomedical Engineering Research，2018，37（2）：238-242.
[13] MEYER A J，PATTEN C，FREGLY B J.Lower extremity EMG-driven modeling of walking with automated adjustment of musculoskeletal geometry[J].PloS One，2017，12（7）：e0179698.
[14] ZHOU B，WANG H，HU F，et al.Accurate recognition of lower limb ambulation mode based on surface electromyography and motion data using machine learning[J].Computer Methods and Programs in Biomedicine，2020，193：105486.
[15] SARTORI M，LLOYD D G，FARINA D.Corrections to “neural data-driven musculoskeletal modeling for personalized neurorehabilitation technologies”[May 16 879-893][J].IEEE Transactions on Biomedical Engineering，2016，63（6）：1341.
[16] WONG C，ZHANG Z Q，LO B，et al.Wearable sensing for solid biomechanics：a review[J].IEEE Sensors Journal，2015，15（5）：2747-2760.
[17] CHOWDHURY R H，REAZ M B I，ALI M A B M，et al.Surface electromyography signal processing and classification techniques[J].Sensors，2013，13（9）：12431-12466.
[18] KWON H，RUTKOVE S B，SANCHEZ B.Recording characteristics of electrical impedance myography needle electrodes[J].Physiological Measurement，2017，38（9）：1748.
[19] LEWIS S，HAHN M，KLEIN C，et al.Implantable silicone electrode for measurement of muscle activity：results of first in vivo evaluation[J].Biomedical Engineering/Biomedizinische Technik，2013，58：24043079.
[20] 李金铭.基于表面肌电信号的下肢康复机器人控制方法研究[D].哈尔滨：哈尔滨工业大学，2013.
LI J M.Research on controlling methods of lower limb rehabilitation robot based on sEmg[D].Harbin：Harbin Institute of Technology，2013.
[21] 史昊楠，桂沛君，徐大卫，等.健康青年下台阶时股内侧肌，股直肌和股外侧肌间的表面肌电变化[J].中国康复理论与实践，2017，23（10）：1162-1165.
SHI H N，GUI P J，XU D W，et al.Surface electromyography changes of vastus medialis，rectus femoris and vastus lateralis during stair descent in healthy youth[J].Chinese Journal of Rehabilitation Theory and Practice，2017，23（10）：1162-1165.
[22] 吴月峰，李建华，张芳，等.表面肌电结合三维步态分析观察脑卒中患者步行支撑相期下肢肌肉激活时序的研究[J].中华物理医学与康复杂志，2019，41（6）：424-428.
WU Y F，LI J H，ZHANG F，et al.Surface electromyography combined with three-dimensional gait analysis to observe the timing of lower limb muscle activation during the walking support phase in stroke patients[J].Chinese Journal of Physical Medicine and Rehabilition，2019，41（6）：424-428.
[23] 陈秀秀，吴庆文，郭子梦，等.镜像疗法对脑卒中偏瘫患者下肢肌肉表面肌电的影响[J].中华物理医学与康复杂志，2018，40（12）：895-898.
CHEN X X，WU Q W，GUO Z M，et al.Effects of mirror therapy on the surface electromyography of lower limb muscles in stroke patients with hemiplegia[J].Chinese Journal of Physical Medicine and Rehabilition，2018，40（12）：895-898.
[24] SHI X，QIN P，ZHU J，et al.Feature extraction and classification of lower limb motion based on sEMG signals[J].IEEE Access，2020，8：132882-132892.
[25] WANG X，DONG D，CHI X，et al.sEMG-based consecutive estimation of human lower limb movement by using multi-branch neural network[J].Biomedical Signal Processing and Control，2021，68：102781.
[26] YAO T，GAO F，ZHANG Q，et al.Multi-feature gait recognition with DNN based on sEMG signals[J].Math Biosci Eng，2021，18：3521-3542.
[27] TOLEDO-PéREZ D C，MARTíNEZ-PRADO M A，GóMEZ-LOENZO R A，et al.A study of movement classification of the lower limb based on up to 4-EMG channels[J].Electronics，2019，8（3）：259.
[28] KYEONG S，SHIN W，YANG M，et al.Recognition of walking environments and gait period by surface electromyography[J].Frontiers of Information Technology & Electronic Engineering，2019，20（3）：342-352.
[29] HU F，WANG H，FENG N，et al.A novel fusion strategy for locomotion activity recognition based on multimodal signals[J].Biomedical Signal Processing and Control，2021，67：102524.
[30] WANG J，TANG L，BRONLUND J E.Surface EMG signal amplification and filtering[J].International Journal of Computer Applications，2013，82（1）：15-22.
[31] 王静.表面肌电信号在上肢康复训练中的应用研究[D].曲阜：曲阜师范大学，2019.
WANG J.Application of surface EMG signal in upper limb rehabilitation training[D].Qufu：Qufu Normal University，2019.
[32] 吴平平，徐剑华，杜明家，等.基于单肌肉表面肌电-加速度融合的步态识别[J].合肥工业大学学报（自然科学版），2020，43（7）：884-889.
WU P P，XU J H，DU M J，et al.Gait recognition based on single muscle surface EMG-acceleration fusion[J].Journal of Hefei University of Technology（Natural Science），2020，43（7）：884-889.
[33] 刘明君，董曾寿.基于改进小波变换的手臂肌电信号去噪算法的研究[J].电子技术应用，2018，44（3）：122-125.
LIU M J，DONG Z S.Research on denoising algorithm of arm EMG signal based on improved wavelet transform[J].Application of Electronic Technique，2018，44（3）：122-125.
[34] HUSSAIN M S，REAZ M B I，IBRAHIMY M I，et al.Wavelet based noise removal from EMG signals[J].Informacije MIDEM-Journal of Microelectronics Electronic Components and Materials，2007，37（2）：94-97.
[35] SUN Z，XI X，YUAN C，et al.Surface electromyography signal denoising via EEMD and improved wavelet thresholds[J].Mathematical Biosciences and Engineering，2020，17（6）：6945-6962.
[36] ZHANG C，ZOU J，MA Z.Identification and analysis of limb rehabilitation signal based on wavelet transform[J].Traitement du Signal，2021，38（3）：689-697.
[37] LI X，TIAN L，ZHENG Y，et al.A new strategy based on feature filtering technique for improving the real-time control performance of myoelectric prostheses[J].Biomedical Signal Processing and Control，2021，70：102969.
[38] JARRAH Y A，ASOGBON M G，SAMUEL O W，et al.High-density surface EMG signal quality enhancement via optimized filtering technique for amputees’ motion intent characterization towards intuitive prostheses control[J].Biomedical Signal Processing and Control，2022，74：103497.
[39] ZONG X，WANG F，SHE J，et al.Denoising method for surface electromyography signals combining CEEMDAN and interval total variation[J].Circuits，Systems，and Signal Processing，2022：1-20.
[40] MA S，LV B，LIN C，et al.EMG signal filtering based on variational mode decomposition and sub-band thresholding[J].IEEE Journal of Biomedical and Health Informatics，2020，25（1）：47-58.
[41] VIJAYVARGIYA A，SINGH B，KUMARI N，et al.sEMG-based deep learning framework for the automatic detection of knee abnormality[J].Signal，Image and Video Processing，2022：1-9.
[42] ZHOU Y，CHEN C，CHENG M，et al.Comparison of machine learning methods in sEMG signal processing for shoulder motion recognition[J].Biomedical Signal Processing and Control，2021，68：102577.
[43] AI Q，ZHANG Y，QI W，et al.Research on lower limb motion recognition based on fusion of sEMG and accele-
rometer signals[J].Symmetry，2017，9（8）：147.
[44] MENG L，PANG J，WANG Z，et al.The role of surface electromyography in data fusion with inertial sensors to enhance locomotion recognition and prediction[J].Sensors，2021，21（18）：6291.
[45] 郑海鹏，李玉榕.基于双通道表面肌电信号的在线手势识别系统研究[J].物联网技术，2022，12（3）：17-20.
ZHENG H P，LI Y R.Research on online gesture recognition system based on dual-channel surface EMG signal[J].Internet of Things Technology，2022，12（3）：17-20.
[46] 汪胜佩，杨惠，肖姝源，等.基于表面肌电信号的手腕动作意图在线识别方法[J].北京生物医学工程，2013，32（5）：467-471.
WANG S P，YANG H，XIAO S Y，et al.Online recognition method of wrist motion intention based on surface EMG signal[J].Beijing Biomedical Engineering，2013，32（5）：467-471.
[47] SAMUEL O W，ZHOU H，LI X，et al.Pattern recognition of electromyography signals based on novel time domain features for amputees' limb motion classification[J].Computers & Electrical Engineering，2018，67：646-655.
[48] AL-QURAISHI M S，ISHAK A J，AHMAD S A，et al.Classification of ankle joint movements based on surface electromyography signals for rehabilitation robot applications[J].Medical & Biological Engineering & Computing，2017，55（5）：747-758.
[49] HAJIAN G，ETEMAD A，MORIN E.Generalized EMG-based isometric contact force estimation using a deep learning approach[J].Biomedical Signal Processing and Control，2021，70：103012.
[50] CHAI G，WANG Y，WU J，et al.Study on the recognition of exercise intensity and fatigue on runners based on subjective and objective information[J].Healthcare，2019，7（4）：150.
[51] TUNCER T，DOGAN S，SUBASI A.Surface EMG signal classification using ternary pattern and discrete wavelet transform based feature extraction for hand movement recognition[J].Biomedical Signal Processing and Control，2020，58：101872.
[52] SHE H，ZHU J，TIAN Y，et al.SEMG feature extraction based on stockwell transform improves hand movement recognition accuracy[J].Sensors，2019，19（20）：4457.
[53] ZHAO K，WEN H，ZHANG Z，et al.Fractal characteristics-based motor dyskinesia assessment[J].Biomedical Signal Processing and Control，2021，68：102707.
[54] JUNIOR J J A M，FREITAS M L B，SIQUEIRA H V，et al.Feature selection and dimensionality reduction：an extensive comparison in hand gesture classification by sEMG in eight channels armband approach[J].Biomedical Signal Processing and Control，2020，59：101920.
[55] LV Z，XIAO F，WU Z，et al.Hand gestures recognition from surface electromyogram signal based on self-organizing mapping and radial basis function network[J].Biomedical Signal Processing and Control，2021，68：102629.
[56] RABIN N，KAHLON M，MALAYEV S，et al.Classification of human hand movements based on EMG signals using nonlinear dimensionality reduction and data fusion techniques[J].Expert Systems with Applications，2020，149：113281.
[57] 段有康，陈小刚，桂剑，等.基于相位划分的下肢连续运动预测[J].浙江大学学报（工学版），2021，55（1）：89-95.
DUAN Y K，CHEN X G，GUI J，et al.Continuous kinematics prediction of lower limbs based on phase division[J].Journal of Zhejiang University（Engineering Edition），2021，55（1）：89-95.
[58] CHEN J，SUN Y，SUN S.Muscle synergy of lower limb motion in subjects with and without knee pathology[J].Diagnostics，2021，11（8）：1318.
[59] 隋修武，石峰.基于改进 MPSO-SVM 算法的下肢连续运动预测模型[J].天津工业大学学报，2019，38（6）：69-75.
SUI X W，SHI F.Prediction model of lower limb continuous motion based on improved MPSO-SVM algorithm[J].Journal of the Tianjin Polytechnic University，2019，38（6）：69-75.
[60] XIE H，LI G，ZHAO X，et al.Prediction of limb joint angles based on multi-source signals by GS-GRNN for exoskeleton wearer[J].Sensors，2020，20（4）：1104.
[61] ZHANG X F，LI X，DAI J T，et al.The design of a hemiplegic upper limb rehabilitation training system based on surface EMG signals[J].Journal of Advanced Mechani-
cal Design，Systems，and Manufacturing，2018，12（1）：1-12.
[62] DE JESUS-RODRIGUEZ H J，MORGAN M A，SAGREIYA H.Deep learning in kidney ultrasound：overview，frontiers，and challenges[J].Advances in Chronic Kidney Disease，2021，28（3）：262-269.
[63] XU Y，ZHOU Y，SEKULA P，et al.Machine learning in construction：from shallow to deep learning[J].Developments in the Built Environment，2021，6：100045.
[64] MORBIDONI C，CUCCHIARELLI A，FIORETTI S，et al.A deep learning approach to EMG-based classification of gait phases during level ground walking[J].Electronics，2019，8（8）：894.
[65] CHEN Z，YANG J，XIE H.Surface-electromyography-based gesture recognition using a multistream fusion strategy[J].IEEE Access，2021，9：50583-50592.
[66] WEI W，DAI Q，WONG Y，et al.Surface-electromyography-based gesture recognition by multi-view deep learning[J].IEEE Transactions on Biomedical Engineering，2019，66（10）：2964-2973.
[67] MACHADO J，MACHADO A，BALBINOT A.Deep learning for surface electromyography artifact contamination type detection[J].Biomedical Signal Processing and Control，2021，68：102752.
[68] XIA P，HU J，PENG Y.EMG‐based estimation of limb movement using deep learning with recurrent convolutional neural networks[J].Artificial Organs，2018，42（5）：67-77.
[69] AMERI A，AKHAEE M A，SCHEME E，et al.A deep transfer learning approach to reducing the effect of electrode shift in EMG pattern recognition-based control[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering，2019，28（2）：370-379.
[70] SHIM H，AN H，LEE S，et al.EMG pattern classification by split and merge deep belief network[J].Symmetry，2016，8（12）：148.
[71] DE SILVA A，PERERA M V，WICKRAMASINGHE K，et al.Real-time hand gesture recognition using temporal muscle activation maps of multi-channel sEMG signals[C]//2020 IEEE International Conference on Acoustics，Speech and Signal Processing（ICASSP），2020：1299-1303.
[72] PANCHOLI S，JOSHI A M.Electromyography-based hand gesture recognition system for upper limb amputees[J].IEEE Sensors Letters，2019，3（3）：1-4.
[73] XIAO F，MU J，LU J，et al.Real-time modeling and feature extraction method of surface electromyography signal for hand movement classification based on oscillatory theory[J].Journal of Neural Engineering，2022，19（2）：026011.
[74] WARIS A，ZIA UR REHMAN M，NIAZI I K，et al.A multiday evaluation of real-time intramuscular EMG usability with ANN[J].Sensors，2020，20（12）：3385.
[75] GUO B，MA Y，YANG J，et al.Lw-CNN-based myoelectric signal recognition and real-time control of robotic arm for upper-limb rehabilitation[J].Computational Intelligence and Neuroscience，2020：8846021.
[76] WEN Y，AVRILLON S，HERNANDEZ-PAVON J C，et al.A convolutional neural network to identify motor units from high-density surface electromyography signals in real time[J].Journal of Neural Engineering，2021，18（5）：056003.