计算机工程与应用 ›› 2024, Vol. 60 ›› Issue (3): 29-43.DOI: 10.3778/j.issn.1002-8331.2305-0269
李振江,魏德健,冯妍妍,于丰帆,马一凡
出版日期:
2024-02-01
发布日期:
2024-02-01
LI Zhenjiang, WEI Dejian, FENG Yanyan, YU Fengfan, MA Yifan
Online:
2024-02-01
Published:
2024-02-01
摘要: 表面肌电信号(surface electromyography,sEMG)是一种测量肌肉活动的非侵入式方法,蕴含着关联人体运动的丰富信息,可用于手部动作识别。基于sEMG手部动作识别是指通过分析手部肌肉的sEMG信号,实现对手部动作的分类和识别。在神经网络发展的推动下,sEMG在手部动作识别领域取得了显著进展,但sEMG面临着噪声大、稳定性差等缺陷,难以有效利用,给高精度手部动作识别模型的获取带来了巨大困难,阻碍了研究成果的转化应用。详细归纳了sEMG手部动作识别方法的研究进展;介绍了常用于动作识别领域的公开肌电数据集,并介绍了自测肌电数据集采集流程;根据研究方法不同将现有的sEMG手部动作识别模型分为基于机器学习的手部动作识别、基于深度学习的手部动作识别和基于混合网络结构的手部动作识别三类,分别对相关模型进行总结分析,对不足之处提出建议;最后对手部动作识别研究需要解决的问题和未来发展方向进行了展望。
李振江, 魏德健, 冯妍妍, 于丰帆, 马一凡. 表面肌电手部动作识别的研究进展[J]. 计算机工程与应用, 2024, 60(3): 29-43.
LI Zhenjiang, WEI Dejian, FENG Yanyan, YU Fengfan, MA Yifan. Research Progress of Surface Electromyography Hand Motion Recognition[J]. Computer Engineering and Applications, 2024, 60(3): 29-43.
[1] LI W, SHI P, YU H. Gesture recognition using surface electromyography and deep learning for prostheses hand: state-of-the-art, challenges, and future[J]. Frontiers in Neuroscience, 2021, 15: 621885. [2] CHEN J, BI S, ZHANG G, et al. High-density surface EMG-based gesture recognition using a 3D convolutional neural network[J]. Sensors, 2020, 20(4): 1201. [3] QI J, JIANG G, LI G, et al. Intelligent human-computer interaction based on surface EMG gesture recognition[J]. IEEE Access, 2019, 7: 61378-61387. [4] PARK K H, LEE S W. Movement intention decoding based on deep learning for multiuser myoelectric interfaces[C]// Proceedings of the 2016 4th International Winter Conference on Brain-Computer Interface (BCI), 2016. [5] XIONG D, ZHANG D, ZHAO X, et al. Deep learning for EMG-based human-machine interaction: a review[J]. IEEE/CAA Journal of Automatica Sinica, 2021, 8(3): 512-533. [6] MEATTINI R, BENATTI S, SCARCIA U, et al. An sEMG-based human-robot interface for robotic hands using machine learning and synergies[J]. IEEE Transactions on Components, Packaging and Manufacturing Technology, 2018, 8(7): 1149-1158. [7] AL HARRACH M, BOUDAOUD S, CARRIOU V, et al. Multi-muscle force estimation using data fusion and HD?sEMG: an experimental study[C]//Proceedings of the Fourth International Conference on Advances in Biomedical Engineering (ICABME), 2017: 1-4. [8] 李自由, 赵新刚, 张弼, 等. 基于表面肌电的意图识别方法在非理想条件下的研究进展[J]. 自动化学报, 2021, 47(5): 955-969. LI Z Y, ZHANG X G, ZHANG B, et al. Review of sEMG-based motion intent recognition methods in non-ideal conditions[J]. Acta Automatica Sinica, 2021, 47(5): 955-969. [9] 王逸桐. 基于表面肌电信号的手势动作识别技术的应用研究[D]. 延吉:延边大学, 2021. WANG Y T. Application research on gesture recognition technology based on surface electromyography[D]. Yanji:Yanbian University, 2021. [10] DIRACO G, RESCIO G, SICILIANO P, et al. Review on human action recognition in smart living: sensing technology, multimodality, real-time processing, interoperability, and resource-constrained processing[J]. Sensors, 2023, 23(11): 5281. [11] SOHN R, SON J, HWANG H, et al. Steady-state visual evoked potential (SSVEP)-based rehabilitation training system with functional electrical stimulation[J]. Journal of Biomedical Engineering Research, 2010, 31(5): 359-364. [12] KONAR A, SAHA S, KONAR A, et al. EEG-gesture based artificial limb movement for rehabilitative applications[J]. Gesture Recognition: Principles, Techniques and Applications, 2018, 724: 243-268. [13] MU D, LI F, YU L, et al. Study on exercise muscle fatigue based on sEMG and ECG data fusion and temporal convolutional network[J]. Plos One, 2022, 17(12): e0276921. [14] FRIGO C, CRENNA P. Multichannel SEMG in clinical gait analysis: a review and state-of-the-art[J]. Clinical Biomechanics, 2009, 24(3): 236-245. [15] SAGAYAM K M, HEMANTH D J. Hand posture and gesture recognition techniques for virtual reality applications: a survey[J]. Virtual Reality, 2017, 21: 91-107. [16] JOSELLI M, CLUA E. Grmobile: a framework for touch and accelerometer gesture recognition for mobile games[C]//2009 VIII Brazilian Symposium on Games and Digital Entertainment, 2009: 141-150. [17] RAHIM M A, SHIN J, ISLAM M R. Hand gesture recognition-based non-touch character writing system on a virtual keyboard[J]. Multimedia Tools and Applications, 2020, 79: 11813-11836. [18] LI F, FEI J. Gesture recognition algorithm based on image information fusion in virtual reality[J]. Personal and Ubiquitous Computing, 2019, 23: 487-497. [19] 赵新刚, 杨毕, 吴成东, 等. 基于表面肌电信号的人体运动意图识别方法研究[C]//2009全国虚拟仪器大会论文集 (二), 2009. ZHAO X G, YANG B, WU C D, et al. Research on the recognition method of human motion intention based on surface electromyography signal[C]//Proceedings of the 2009 National Conference on Virtual Instrumentation (II), 2009. [20] CAMPBELL E, PHINYOMARK A, SCHEME E. Deep cross-user models reduce the training burden in myoelectric control[J]. Frontiers in Neuroscience, 2021, 15: 657958. [21] LUU D K, NGUYEN A T, JIANG M, et al. Deep learning-based approaches for decoding motor intent from peripheral nerve signals[J]. Frontiers in Neuroscience, 2021, 15: 667907. [22] ZHAI X, JELFS B, CHAN R H, et al. Self-recalibrating surface EMG pattern recognition for neuroprosthesis control based on convolutional neural network[J]. Frontiers in Neuroscience, 2017, 11: 379. [23] SULTANA A, AHMED F, ALAM M S. A systematic review on surface electromyography-based classification system for identifying hand and finger movements[J]. Healthcare Analytics, 2022: 100126. [24] ATZORI M, GIJSBERTS A, KUZBORSKIJ I, et al. Characterization of a benchmark database for myoelectric movement classification[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2014, 23(1): 73-83. [25] 付梦龙. 基于表面肌电信号的实时在线肌肉疲劳估计方法研究[D]. 合肥:中国科学技术大学, 2020. FU M L. Research on real-time online muscle fatigue estimation method based on surface electromyography[D]. Hefei:University of Science and Technology of China, 2020. [26] ORTIZ-CATALAN M, BRANEMARK R, HAKANSSON B. BioPatRec: a modular research platform for the control of artificial limbs based on pattern recognition algorithms[J]. Source Code Biol Med, 2013, 8: 11. [27] FATIMAH B, SINGH P, SINGHAL A, et al. Hand movement recognition from sEMG signals using Fourier decomposition method[J]. Biocybernetics and Biomedical Engineering, 2021, 41(2): 690-703. [28] AMMA C, KRINGS T, B?ER J, et al. Advancing muscle-computer interfaces with high-density electromyography[C]//Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 2015: 929-938. [29] GENG W, DU Y, JIN W, et al. Gesture recognition by instantaneous surface EMG images[J]. Scientific Reports, 2016, 6: 36571. [30] DAI Q, LI X, GENG W, et al. CAPG-MYO: a muscle-computer interface supporting user-defined gesture recognition[C]//Proceedings of the 9th International Conference on Computer and Communications Management, 2021: 52-58. [31] DU Y, JIN W, WEI W, et al. Surface EMG-based inter-session gesture recognition enhanced by deep domain adaptation[J]. Sensors, 2017, 17(3): 458. [32] 丁其川, 熊安斌, 赵新刚, 等. 基于表面肌电的运动意图识别方法研究及应用综述[J]. 自动化学报, 2016, 42(1): 13-25. DING Q C, XIONG A B, ZHAO X G, et al. Research and application review of motion intention recognition method based on surface electromyography[J]. Acta Automatica Sinica, 2016, 42(1): 13-25. [33] KANG S, KIM H, PARK C, et al. sEMG-based hand gesture recognition using binarized neural network[J]. Sensors, 2023, 23(3): 1436. [34] 董泽萍, 仇大伟, 刘静. 表面肌电人体下肢动作识别预测进展[J]. 计算机工程与应用, 2023, 59(14): 39-50. DONG Z P, QIU D W, LIU J. Progress in human lower extremity motion recognition and prediction based on surface electromyography[J]. Computer Engineering and Applications, 2023, 59(14): 39-50. [35] CHOWDHURY R H, REAZ M B, ALI M A B M, et al. Surface electromyography signal processing and classification techniques[J]. Sensors, 2013, 13(9): 12431-12466. [36] XUE J, SUN Z, DUAN F, et al. Underwater sEMG-based recognition of hand gestures using tensor decomposition[J]. Pattern Recognition Letters, 2023, 165: 39-46. [37] ZHANG Z, HE C, YANG K. A novel surface electromyographic signal-based hand gesture prediction using a recurrent neural network[J]. Sensors (Basel), 2020, 20(14): 3994. [38] TEPE C, DEMIR M C. Real-time classification of EMG myo armband data using support vector machine[J]. IRBM, 2022, 43(4): 300-308. [39] BAHADOR A, YOUSEFI M, MARASHI M, et al. High accurate lightweight deep learning method for gesture recognition based on surface electromyography[J]. Computer Methods and Programs in Biomedicine, 2020, 195: 105643. [40] PHINYOMARK A, PHUKPATTARANONT P, LIMSAKUL C. Feature reduction and selection for EMG signal classification[J]. Expert Systems with Applications, 2012, 39(8): 7420-7431. [41] UR REHMAN M Z, GILANI S O, WARIS A, et al. A novel approach for classification of hand movements using surface EMG signals[C]//2017 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), 2017: 265-269. [42] ATZORI M, GIJSBERTS A, CASTELLINI C, et al. Electromyography data for non-invasive naturally-controlled robotic hand prostheses[J]. Scientific Data, 2014, 1(1): 1-13. [43] NARAYAN Y. SEMG signal classification using KNN classifier with FD and TFD features[J]. Materials Today: Proceedings, 2021, 37: 3219-3225. [44] AL-TIMEMY A H, BUGMANN G, ESCUDERO J, et al. Classification of finger movements for the dexterous hand prosthesis control with surface electromyography[J]. IEEE Journal of Biomedical and Health Informatics, 2013, 17(3): 608-618. [45] NOEMI GOZZI L M, MERCORIO F, PEDROCCHI A. XAI for myo-controlled prosthesis: explaining EMG data for hand gesture classification[J]. Knowledge-Based Systems, 2022, 240. [46] SHAO J, NIU Y, XUE C, et al. Single-channel SEMG using wavelet deep belief networks for upper limb motion recognition[J]. International Journal of Industrial Ergonomics, 2020, 76: 1-10. [47] DUAN F, REN X, YANG Y. A gesture recognition system based on time domain features and linear discriminant analysis[J]. IEEE Transactions on Cognitive and Developmental Systems, 2021, 13(1): 200-208. [48] NAIK G R, ACHARYYA A, NGUYEN H T. Classification of finger extension and flexion of EMG and Cyberglove data with modified ICA weight matrix[C]//Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2014: 3829-3832. [49] BENALCAZAR M E, MOTOCHE C, ZEA J A, et al. Real-time hand gesture recognition using the Myo armband and muscle activity detection[C]//2017 IEEE Second Ecuador Technical Chapters Meeting (ETCM), 2017: 1-6. [50] BERGIL E, ORAL C, ERGUL E U. Efficient hand movement detection using k-means clustering and k-nearest neighbor algorithms[J]. Journal of Medical and Biological Engineering, 2020, 41(1): 11-24. [51] CHEN X, WANG Z J. Pattern recognition of number gestures based on a wireless surface EMG system[J]. Biomedical Signal Processing and Control, 2013, 8(2): 184-192. [52] MALEKI A N A A. Artificial neural network classifier in comparison with LDA and LS-SVM classifiers to recognize 52 hand postures and movements[C]//2014 4th International Conference on Computer and Knowledge Engineering (ICCKE), 2014. [53] 蔡芝蔚, 吴淑燕, 宋俊锋. 基于 SVM 和组合优化模型的手势识别[J]. 系统仿真学报, 2016, 28(8): 1812-1817. CAI Z W, WU S Y, SONG J F. Study on hand gesture recognition and portfolio optimization model based on SVM[J]. Journal of System Simulation, 2016, 28(8): 1812-1817. [54] 朱旭, 刘静, 董泽萍, 等. 基于表面肌电图手势动作意图识别的系统综述[J]. 中国康复理论与实践, 2022, 28(9): 1032-1038. ZHU X, LIU J, DONG Z P, et al. Gesture action intent recognition based on surface electromyography:a systematic review[J]. Chinese Rehabilitation Theory and Practice, 2022, 28(9): 1032-1038. [55] DING Z, YANG C, TIAN Z, et al. sEMG-based gesture recognition with convolution neural networks[J]. Sustainability, 2018, 10(6): 1865. [56] CHEN L, FU J, WU Y, et al. Hand gesture recognition using compact CNN via surface electromyography signals[J]. Sensors, 2020, 20(3): 672. [57] C?Té-ALLARD U, FALL C L, CAMPEAU-LECOURS A, et al. Transfer learning for sEMG hand gestures recognition using convolutional neural networks[C]//2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2017: 1663-1668. [58] TAM S, BOUKADOUM M, CAMPEAU-LECOURS A, et al. Intuitive real-time control strategy for high-density myoelectric hand prosthesis using deep and transfer learning[J]. Scientific Reports, 2021, 11(1): 11275. [59] QI S, WU X, CHEN W H, et al. sEMG-based recognition of composite motion with convolutional neural network[J]. Sensors and Actuators A: Physical, 2020, 311:1-11. [60] SAMADANI A. Gated recurrent neural networks for EMG-based hand gesture classification. A comparative study[C]//2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2018: 1-4. [61] SIM?O M, NETO P, GIBARU O. EMG-based online classification of gestures with recurrent neural networks[J]. Pattern Recognition Letters, 2019, 128: 45-51. [62] WANG C, GUO W, ZHANG H, et al. sEMG-based continuous estimation of grasp movements by long-short term memory network[J]. Biomedical Signal Processing and Control, 2020, 59:101774. [63] TSINGANOS B C P, CORNELIS J, JANSEN B, et al. Improved gesture recognition based on sEMG signals and TCN[C]//2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019: 1169-1173. [64] BETTHAUSER J L, KRALL J T, BANNOWSKY S G, et al. Stable responsive EMG sequence prediction and adaptive reinforcement with temporal convolutional networks[J]. IEEE Transactions on Biomedical Engineering, 2020, 67(6): 1707-1717. [65] BETTHAUSER J L, KRALL J T, KALIKI R R, et al. Stable electromyographic sequence prediction during movement transitions using temporal convolutional networks[C]//2019 9th International IEEE/EMBS Conference on Neural Engineering (NER), 2019:1046-1049. [66] ZANGHIERI M, CONTI F, BURRELLO A,et al. Temporal variability analysis in sEMG hand grasp recognition using temporal convolutional networks[C]//2020 2nd IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS), 2020: 228-232. [67] ZANGHIERI M, BENATTI S, BURRELLO A, et al. Robust real-time embedded EMG recognition framework using temporal convolutional networks on a multicore IoT processor[J]. IEEE Transactions on Biomedical Circuits and Systems, 2020, 14(2): 244-256. [68] COTE-ALLARD U, CAMPBELL E, PHINYOMARK A, et al. Interpreting deep learning features for myoelectric control: a comparison with handcrafted features[J]. Frontiers in Bioengineering and Biotechnology, 2020, 8: 158. [69] CHEN H, TONG R, CHEN M, et al. A hybrid CNN-SVM classifier for hand gesture recognition with surface emg signals[C]//2018 International Conference on Machine Learning and Cybernetics (ICMLC), 2018: 619-624. [70] HU Y, WONG Y, WEI W, et al. A novel attention-based hybrid CNN-RNN architecture for sEMG-based gesture recognition[J]. PLoS One, 2018, 13(10): e0206049. [71] FARINA D, JIANG N, REHBAUM H, et al. The extraction of neural information from the surface EMG for the control of upper-limb prostheses: emerging avenues and challenges[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2014, 22(4): 797-809. [72] OVUR S E, ZHOU X, QI W, et al. A novel autonomous learning framework to enhance sEMG-based hand gesture recognition using depth information[J]. Biomedical Signal Processing and Control, 2021, 66: 102444. [73] RAHIMIAN E, ZABIHI S, ASIF A, et al. TEMGNet: deep transformer-based decoding of upperlimb sEMG for hand gestures recognition[J]. arXiv:2109.12379, 2021. [74] MCINTOSH J, MARZO A, FRASER M, et al. Echoflex: hand gesture recognition using ultrasound imaging[C]//Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 2017: 1923-1934. [75] YANG X, SUN X, ZHOU D, et al. Towards wearable a-mode ultrasound sensing for real-time finger motion recognition[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2018, 26(6): 1199-1208. |
[1] | 包吉祥, 李林, 赵梦鸽. 基于考虑滞后性LSTM模型的电商需求预测[J]. 计算机工程与应用, 2022, 58(24): 276-283. |
[2] | 姚远,张朝阳. 基于HP-LSTM模型的股指价格预测方法[J]. 计算机工程与应用, 2021, 57(24): 296-304. |
[3] | 薛文龙,于炯,郭志琦,李梓杨. 基于特征融合卷积神经网络的端到端加密流量分类[J]. 计算机工程与应用, 2021, 57(18): 114-121. |
[4] | 张驰,郭媛,黎明. 人工神经网络模型发展及应用综述[J]. 计算机工程与应用, 2021, 57(11): 57-69. |
[5] | 李杰其,胡良兵. 基于机器学习的设备预测性维护方法综述[J]. 计算机工程与应用, 2020, 56(21): 11-19. |
[6] | 隋修武,牛佳宝,李昊天,乔明敏. 基于NMF-SVM模型的上肢sEMG手势识别方法[J]. 计算机工程与应用, 2020, 56(17): 161-166. |
[7] | 张文婧,陈治亚,冯芬玲,李 万. 基于稀疏理论的DAE在公路事故伤亡预测应用[J]. 计算机工程与应用, 2019, 55(7): 241-247. |
[8] | 王海燕1,2,袁雪琴1,宋 超1,张正勇1,2,刘 军1,2,沙 敏1,2. 相似度分析结合人工神经网络鉴别苹果香精[J]. 计算机工程与应用, 2018, 54(4): 141-147. |
[9] | 宋 倩,郭 宇,王发麟. 基于人工神经网络的线缆敷设质量综合评判[J]. 计算机工程与应用, 2018, 54(14): 256-263. |
[10] | 颜 丹,蒋加伏. 基于栈式去噪自动编码器的边际Fisher分析算法[J]. 计算机工程与应用, 2017, 53(5): 134-139. |
[11] | 马正华1,李 雷2,乔玉涛2,戎海龙3,曹海婷2. 基于多传感器融合的动态手势识别研究分析[J]. 计算机工程与应用, 2017, 53(17): 153-159. |
[12] | 柏 丛1,2,彭仲仁1,3. 基于动态模型的公交车行程时间预测[J]. 计算机工程与应用, 2016, 52(3): 103-107. |
[13] | 张明光,张 钰. 基于ANN伪量测建模的配电网状态估计[J]. 计算机工程与应用, 2016, 52(17): 253-256. |
[14] | 刘 殷1,吕学强1,刘 坤2. 条件随机场与多层算法模型的实体自动识别[J]. 计算机工程与应用, 2016, 52(11): 141-147. |
[15] | 彭丽蓉1,周 磊2. 认知行为计算模型结合DM的教学质量提升[J]. 计算机工程与应用, 2014, 50(9): 237-241. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||