渐进式特征融合的高效轻量级人体姿态估计

doi:10.3778/j.issn.1002-8331.2405-0028

摘要/Abstract

摘要： 针对当前人体姿态估计模型在追求检测性能时所面临的巨大计算负载问题，以高分辨率网络为基线提出高效渐进式特征融合网络（efficient progressive feature fusion network，EPFFNet）。移除了高分辨率网络的第四阶段，达到平衡模型复杂度和检测能力的目的。构建了一种混合高效通道注意力，能够同时关注全局和局部通道特征信息，并在此基础上提出了一种新型注意力特征融合模块，实现特征之间的强融合。设计了一种高效坐标注意力，增强模型对不同区域特征信息的关注，同时与多样归一化和其他轻量化模块相结合提出了GENeck和GEBlock模块，替换高分辨率网络的残差块，保证了模型性能并实现了模型轻量化。设计了一种渐进式特征交融模块，增强模型对不同分辨率特征的融合能力。实验结果表明，EPFFNet在COCO和MPII数据集上分别取得了75.1%和90.8%的准确率，不仅在轻量化模型中实现了最优精度，而且与大模型相比具有相当或更佳的性能。

关键词: 计算机视觉, 人体姿态估计, 轻量化, 高分辨率网络, 注意力机制

Abstract: In response to the significant computational load faced by current human pose estimation models in pursuit of detection performance, this paper proposes efficient progressive feature fusion network(EPFFNet) based on a high-resolution network as the baseline. The fourth stage of the high-resolution network is removed to achieve a balance between model complexity and detection capability. A hybrid efficient channel attention mechanism is constructed, which can simultaneously focus on both global and local channel feature information. Based on this, a novel attention feature fusion module is proposed to achieve strong fusion between features. Furthermore, an efficient coordinate attention mechanism is designed to enhance the model’s attention to feature information in different regions. Additionally, combined with various normalization and other lightweight modules, GENeck and GEBlock modules are proposed to replace the residual blocks of the high-resolution network, ensuring model performance while achieving model lightweighting. A progressive feature fusion module is designed to enhance the model’s ability to fuse features of different resolutions. The experimental results indicate that EPFFNet achieves accuracy rates of 75.1% and 90.8% on the COCO and MPII datasets respectively. EPFFNet not only achieves the best accuracy among lightweight models but also demonstrates comparable or better performance than larger models.

Key words: computer vision, human pose estimation, lightweight, high-resolution network, attention mechanism

肖俊, 赵骥. 渐进式特征融合的高效轻量级人体姿态估计[J]. 计算机工程与应用, 2025, 61(15): 218-228.

XIAO Jun, ZHAO Ji. Efficient Lightweight Human Pose Estimation with Progressive Feature Fusion[J]. Computer Engineering and Applications, 2025, 61(15): 218-228.

参考文献

[1] ZHENG C, WU W, CHEN C, et al. Deep learning-based human pose estimation: a survey[J]. ACM Computing Surveys, 2023, 56(1): 1-37.
[2] CHEN Y L, WANG Z C, PENG Y X, et al. Cascaded pyramid network for multi-person pose estimation[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7103-7112.
[3] XIAO B, WU H P, WEI Y C. Simple baselines for human pose estimation and tracking[C]//Proceedings of the 15th European Conference on Computer Vision. Cham: Springer International Publishing, 2018: 472-487.
[4] SUN K, XIAO B, LIU D, et al. Deep high-resolution representation learning for human pose estimation[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 5686-5696.
[5] YANG S, QUAN Z B, NIE M, et al. TransPose: keypoint localization via transformer[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2021: 11782-11792.
[6] XU Y, ZHANG J, ZHANG Q, et al. ViTPose: simple vision transformer baselines for human pose estimation[J]. arXiv: 2204.12484, 2022.
[7] WANG J, SUN K, CHENG T, et al. Deep high-resolution representation learning for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 43(10): 3349-3364.
[8] NEFF C, SHETH A, FURGURSON S, et al. EfficientHRNet[J]. Journal of Real-Time Image Processing, 2021, 18(4): 1037-1049.
[9] TAN M, LE Q. EfficientNet: rethinking model scaling for convolutional neural networks[C]//Proceedings of the International Conference on Machine Learning, 2019: 6105-6114.
[10] YU C Q, XIAO B, GAO C X, et al. Lite-HRNet: a lightweight high-resolution network[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 10435-10445.
[11] MA N N, ZHANG X Y, ZHENG H T, et al. ShuffleNet V2: practical guidelines for efficient CNN architecture design[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer International Publishing, 2018: 122-138.
[12] LI Q, ZHANG Z, XIAO F, et al. Dite-HRNet: dynamic lightweight high-resolution network for human pose estimation[J]. arXiv:2204.10762, 2022.
[13] WANG Y H, LI M Y, CAI H, et al. Lite pose: efficient architecture design for 2D human pose estimation[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 13116-13126.
[14] DAI Y M, GIESEKE F, OEHMCKE S, et al. Attentional feature fusion[C]//Proceedings of the 2021 IEEE Winter Conference on Applications of Computer Vision. Piscataway: IEEE, 2021: 3559-3568.
[15] HOU Q B, ZHOU D Q, FENG J S. Coordinate attention for efficient mobile network design[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 13713-13722.
[16] TANG Y, HAN K, GUO J, et al. GhostNetv2: enhance cheap operation with long-range attention[C]//Advances in Neural Information Processing Systems, 2022: 9969-9982.
[17] LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft COCO: common objects in context[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer, 2014: 740-755.
[18] ANDRILUKA M, PISHCHULIN L, GEHLER P, et al. 2D human pose estimation: new benchmark and state of the art analysis[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2014: 3686-3693.
[19] WANG Q L, WU B G, ZHU P F, et al. ECA-net: efficient channel attention for deep convolutional neural networks[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 11531-11539.
[20] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer International Publishing, 2018: 3-19.
[21] LI C, ZHOU A, YAO A. Omni-dimensional dynamic convolution[J]. arXiv:2209.07947, 2022.
[22] PAN X, LUO P, SHI J, et al. Two at once: enhancing-learning and generalization capacities via IBN-Net[C]//Proceedings of the European Conference on Computer Vision, 2018: 464-479.
[23] 林远强, 郜辉, 王鹏, 等. 引入级联通道注意力的轻量化人体姿态估计[J]. 计算机工程与应用, 2024, 60(13): 219-227.
LIN Y Q, GAO H, WANG P, et al. Lightweight human pose estimation with cascaded channel attention[J]. Computer Engineering and Applications, 2024, 60(13): 219-227.
[24] ZHANG F, ZHU X, DAI H, et al. Distribution-aware coordinate representation for human pose estimation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 7093-7102.
[25] SUN X F, ADAMU M J, ZHANG R F, et al. Pixel-coordinate-induced human pose high-precision estimation method[J]. Electronics, 2023, 12(7): 1648.
[26] YUAN Y, FU R, HUANG L, et al. HRFormer: high-resolution Transformer for dense prediction[J]. arXiv:2110.09408, 2021.
[27] HUANG J J, ZHU Z, GUO F, et al. The devil is in the details: delving into unbiased data processing for human pose estimation[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 5699-5708.
[28] LI Y J, ZHANG S K, WANG Z C, et al. Tokenpose: learning keypoint tokens for human pose estimation[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2021: 11293-11302.
[29] ZHANG T, LIAN J X, WEN J T, et al. Multi-person pose estimation in the wild: using adversarial method to train a top-down pose estimation network[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2023, 53(7): 3919-3929.
[30] WANG X Y, TONG J W, WANG R. Attention refined network for human pose estimation[J]. Neural Processing Letters, 2021, 53(4): 2853-2872.
[31] SANDLER M, HOWARD A, ZHU M L, et al. MobileNetV2: inverted residuals and linear bottlenecks[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 4510-4520.
[32] YU W H, LUO M, ZHOU P, et al. MetaFormer is actually what you need for vision[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 10809-10819.
[33] CHEN Y P, DAI X Y, LIU M C, et al. Dynamic convolution: attention over convolution kernels[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 11027-11036.
[34] CHEN Y, DAI X, LIU M, et al. Dynamic ReLU[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer International Publishing, 2020: 351-367.