融合VIT与CNN注意力机制的面部疼痛评估算法研究

doi:10.3778/j.issn.1002-8331.2305-0311

摘要/Abstract

摘要： 准确的疼痛评估可以为病人镇痛提供一定指导，为解决传统人工疼痛评估效率低、费时费力等问题，提出一种基于VIT与CNN注意力机制的面部疼痛评估算法，对疼痛进行多级预测。搭建面部疼痛表情采集平台，提取视频帧序列并进行数据预处理，建立疼痛表情数据库；提出一种改进的多尺度通道注意力模块关注关键特征信息，将CNN和VIT作为主干网络并行连接，提取更高级的面部局部-全局特征，以时序方式输入长短期记忆网络（LSTM）进行疼痛评估；在疼痛表情数据库上进行模型性能验证，实验结果表明，该算法在精确率、召回率、F1分数、准确率指标方面分别达到96.8%、96.7%、0.97、96.8%，与其他深度学习模型相比可更有效识别疼痛，为康复领域疼痛评估研究做出一定贡献。

关键词: 疼痛评估, 面部表情, VIT网络, 卷积神经网络, 注意力机制

Abstract: Accurate pain assessment can provide valuable guidance for patient analgesia. To address the problems of low efficiency, time-consuming, and laborious nature of traditional artificial pain assessment, a facial pain assessment algorithm is proposed based on VIT and CNN attention mechanisms to predict pain at multiple levels. Firstly, a facial pain expression acquisition platform is built to extract the video frame sequence and preprocess the data, establishing a pain expression database. Secondly, an improved multi-scale channel attention module is proposed to focus on key feature information. CNN and VIT are connected in parallel as the backbone network to extract more advanced facial local-global features, which are then fed to the long short-term memory network (LSTM) in a temporal manner for pain assessment. Finally, the model performance is verified on the pain expression database. The experimental results show that the precision, recall, F1-score and accuracy of the algorithm are 96.8%, 96.7%, 0.97 and 96.8%, respectively. Compared with other deep learning models, it can more effectively identify pain and make a certain contribution to pain assessment research in the field of rehabilitation.

Key words: pain assessment, facial expression, VIT network, convolutional neural network, attention mechanism

郭士杰, 卢世杰, 耿艳利, 顾博文, 孙浩. 融合VIT与CNN注意力机制的面部疼痛评估算法研究[J]. 计算机工程与应用, 2024, 60(15): 277-283.

GUO Shijie, LU Shijie, GENG Yanli, GU Bowen, SUN Hao. Facial Pain Assessment Algorithm Fusing VIT and CNN Attention Mechanism[J]. Computer Engineering and Applications, 2024, 60(15): 277-283.

参考文献

[1] 支瑞聪，周才霞. 疼痛自动识别综述[J]. 计算机系统应用, 2020, 29(2): 9-27.
ZHI R C, ZHOU C X. Review of automatic pain recognition[J]. Computer Systems & Applications, 2020, 29(2): 9-27.
[2] TOMPKINS D A, HOBELMANN J G, COMPTON P, et al. Providing chronic pain management in the “fifth vital sign” era: historical and treatment perspectives on a modern-day medical dilemma[J]. Drug Alcohol Depend, 2017, 173: 11-21.
[3] 彭进业, 杨瑞靖, 冯晓毅, 等. 人脸疼痛表情识别综述[J]. 数据采集与处理, 2016, 31(1): 43-55.
PENG J Y, YANG R J, FENG X Y, et al. Survey on facial expression recognition of pain[J]. Journal of Data Acquisition and Processing, 2016, 31(1): 43-55.
[4] HUANG Y, QING L, XU S, et al. HybNet: a hybrid network structure for pain intensity estimation[J]. The Visual Computer, 2022, 38(3): 871-882.
[5] WERNER P, AL-HAMADI A, LIMBRECHT-ECKLUNDT K, et al. Automatic pain assessment with facial activity descriptors[J]. IEEE Transactions on Affective Computing, 2017, 8(3): 286-299.
[6] KRUMHUBER E G, SKORA L, KüSTER D, et al. A review of dynamic datasets for facial expression research[J]. Emotion Review, 2017, 9(3): 280-292.
[7] YE J J, LEE K T, CHOU Y Y, et al. Assess pain intensity using photoplethysmography signal in chronic myofascial pain syndrome[J]. Pain Practice, 2018, 18(3): 296-304.
[8] 刘定玺, 蒋国璋, 章花, 等. 基于深度学习的面部疼痛智能评估方法研究[J]. 生物医学工程研究, 2022, 41(3): 268-274.
LIU D X, JIANG G Z, ZHANG H, et al. Research on intelligent facial pain assessment method based on deep learning[J]. Journal of Biomedical Engineering Research, 2022, 41(3): 268-274.
[9] 郭文强, 赵艳, 徐紫薇, 等. 基于多模态的贝叶斯网络疼痛识别方法[J]. 科学技术与工程, 2022, 22(28): 12505-12511.
GUO W Q, ZHAO Y, XU Z W, et al. Pain recognition method based on multimodal Bayesian network[J]. Science Technology and Engineering, 2022, 22(28): 12505-12511.
[10] RODRIGUEZ P, CUCURULL G, GONZALEZ J, et al. Deep pain: exploiting long short-term memory networks for facial expression classification[J]. IEEE Transactions on Cybernetics, 2022, 52(5): 3314-3324.
[11] BARGSHADY G, ZHOU X, DEO R C, et al. Enhanced deep learning algorithm development to detect pain intensity from facial expression images[J]. Expert Systems with Applications, 2020, 149: 113305.
[12] XU X, HUANG J S, SA V R. Pain evaluation in video using extended multitask learning from multidimensional measurements[J]. Proceedings of Machine Learning Research, 2020, 116: 141-154.
[13] 郑建伟, 刘新妹, 殷俊龄. 基于LBP和SVM的疼痛表情识别[J]. 计算机系统应用, 2021, 30(4): 111-117.
ZHENG J W, LIU X M, YIN J L. Pain expression recognition based on LBP and SVM[J]. Computer Systems & Applications, 2021, 30(4): 111-117.
[14] TAVAKOLIAN M, LOPEZ M B, LIU L. Self-supervised pain intensity estimation from facial videos via statistical spatiotemporal distillation[J]. Pattern Recognition Letters, 2020, 140: 26-33.
[15] WANG F, XIANG X, LIU C, et al. Regularizing face verification nets for pain intensity regression[C]//2017 24th IEEE International Conference on Image Processing (ICIP), Bejing, Sep 17-20, 2017. New York: IEEE, 2017: 1087-1091.
[16] ZAMZMI G, GOLDGOF D, KASTURI R, et al. Neonatal pain expression recognition using transfer learning[J]. arXiv:1807.01631, 2018.
[17] DUTTA P, NACHAMAI M. Facial pain expression recognition in real-time videos[J]. Journal of Healthcare Engineering, 2018: 7961427.
[18] TAVAKOLIAN M, HADID A. Deep spatiotemporal representation of the face for automatic pain intensity estimation[C]//2018 24th IEEE International Conference on Pattern Recognition (ICPR), Bejing, Aug 20-24, 2018. New York: IEEE, 2018: 350-354.
[19] BARGSHADY G, ZHOU X, DEO R C, et al. Ensemble neural network approach detecting pain intensity from facial expressions[J]. Artificial Intelligence in Medicine, 2020, 109: 101954.