Lightweight Human Pose Estimation with Cascaded Channel Attention

doi:10.3778/j.issn.1002-8331.2307-0385

Abstract

Abstract: Aiming at the problem of serious loss of accuracy in the lightweighting process of the current human pose estimation model, a lightweight human pose estimation model that introduces cascaded channel attention is proposed using the high resolution network (HRNet) as a baseline. Firstly, a cascading channel attention that maintains internal high-resolution features is constructed so as to learn the importance of each channel of the input features to improve the model representation. Secondly, the residual module with high arithmetic complexity in HRNet stages 2, 3, and 4 is replaced by designing a lightweight deepwise convolutional transform module based on the structure of the MetaFormer. Furthermore, a multi-scale feature fusion method is designed to reduce the loss of semantic information of multi-dimensional features in the original fusion method of HRNet. Finally, unbiased data processing is used to eliminate offset errors caused by the process of encoding the heat map at key points. Experimental results from the COCO2017 validation set show that the proposed model reduces the number of model parameters and floating-point operations by 90.2% and 83.1%, respectively, compared to the benchmark model with a 2 percentage points decrease in AP, and achieves the optimal accuracy among the lightweight models with an AP of 71.4%.

Key words: human pose estimation, lightweight, channel attention, MetaFormer structure, multi-scale feature fusion

摘要： 针对当前人体姿态估计模型在轻量化过程中精度损失严重的问题，以高分辨率网络（HRNet）为基线提出一种引入级联通道注意力的轻量化人体姿态估计模型。构建一种保持内部高分辨率特征的级联通道注意力，学习输入特征各通道的重要性来提高模型表征能力；通过设计一种基于MetaFormer结构的轻量级深度卷积变换模块来替换HRNet阶段2、3、4中运算复杂度较高的残差模块；设计一种多尺度特征融合方法减少HRNet原融合方法中的多维特征语义信息损失；采用无偏数据处理来消除关键点热力图编码过程中导致的偏移误差。COCO2017验证集的实验结果表明，所提出的模型同基准模型相比，在AP降低2个百分点的情况下，模型参数量和浮点运算量分别减少了90.2%和83.1%，并且以AP为71.4%的表现在轻量化模型中达到精度最优。

关键词: 人体姿态估计, 轻量化, 通道注意力, MetaFormer结构, 多尺度特征融合

LIN Yuanqiang, GAO Hui, WANG Peng, LYU Zhigang, LI Xiaoyan, WANG Chu. Lightweight Human Pose Estimation with Cascaded Channel Attention[J]. Computer Engineering and Applications, 2024, 60(13): 219-227.

林远强, 郜辉, 王鹏, 吕志刚, 李晓艳, 王储. 引入级联通道注意力的轻量化人体姿态估计[J]. 计算机工程与应用, 2024, 60(13): 219-227.

References

[1] DUAN H, ZHAO Y, CHEN K, et al. Revisiting skeleton-based action recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 2969-2978.
[2] LIU H, LIU T, ZHANG Z, et al. ARHPE: asymmetric relation-aware representation learning for head pose estimation in industrial human-computer interaction[J]. IEEE Transactions on Industrial Informatics, 2022, 18(10): 7107-7117.
[3] WEI W L, LIN J C, LIU T L, et al. Capturing humans in motion: temporal-attentive 3D human pose and shape estimation from monocular video[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 13211-13220.
[4] 孙琪翔, 张睿哲, 何宁, 等. 基于非局部高分辨率网络的人体姿态估计方法[J]. 计算机工程与应用, 2022, 58(13): 227-234.
SUN Q X, ZHANG R Z, HE N, et al. Human pose estimation method based on non-local high-resolution networks[J]. Computer Engineering and Applications, 2022, 58(13): 227-234.
[5] CHEN Y, WANG Z, PENG Y, et al. Cascaded pyramid network for multi-person pose estimation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7103-7112.
[6] XIAO B, WU H, WEI Y. Simple baselines for human pose estimation and tracking[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 466-481.
[7] SUN K, XIAO B, LIU D, et al. Deep high-resolution representation learning for human pose estimation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 5693-5703.
[8] YANG S, QUAN Z, NIE M, et al. Transpose: keypoint locali-
zation via transformer[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 11802-11812.
[9] XU Y, ZHANG J, ZHANG Q, et al. ViTPose: simple vision Transformer baselines for human pose estimation[J]. arXiv:2204.12484, 2022.
[10] WANG J, SUN K, CHENG T, et al. Deep high-resolution representation learning for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 43(10): 3349-3364.
[11] YU C, XIAO B, GAO C, et al. Lite-HRNet: a lightweight high-resolution network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 10440-10450.
[12] LI Q, ZHANG Z, XIAO F, et al. Dite-HRNet: dynamic lightweight high-resolution network for human pose estimation[J]. arXiv:2204.10762, 2022.
[13] 邓辉, 徐杨. 融入注意力和密集连接的轻量型人体姿态估计[J]. 计算机工程与应用, 2022, 58(16): 265-273.
DENG H, XU Y. Lightweight human pose estimation based on attention and dense connection[J]. Computer Engineering and Applications, 2022, 58(16): 265-273.
[14] HUANG J, ZHU Z, GUO F, et al. The devil is in the details: delving into unbiased data processing for human pose estimation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 5700-5709.
[15] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7132-7141.
[16] WANG Q, WU B, ZHU P, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 11534-11542.
[17] WANG X, GIRSHICK R, GUPTA A, et al. Non-local neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7794-7803.
[18] SHEN Z, ZHANG M, ZHAO H, et al. Efficient attention: attention with linear complexities[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021: 3531-3539.
[19] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 3-19.
[20] LIU H, LIU F, FAN X, et al. Polarized self-attention: towards high-quality pixel-wise regression[J]. arXiv:2107.00782, 2021.
[21] YU W, LUO M, ZHOU P, et al. MetaFormer is actually what you need for vision[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 10819-10829.