Human Pose Estimation Method Based on Non-Local High-Resolution Networks

doi:10.3778/j.issn.1002-8331.2012-0217

Abstract

Abstract: Human pose estimation is the basic task in computer vision. It can be used in action recognition, games, animation production and so on. Inspired by the nonlocal mean method, a non-local high-resolution（NLHR） network is designed. Fusion of non-local network modules in the network phase of the original image 1/32 resolution, so that the network has the ability to obtain global features, in order to improve the accuracy of human posture estimation. The NLHR network is trained on MPII datasets and tested on the MPII validation set, the average accuracy under PCKh@0.5 evaluation criteria is 90.5%, 0.2 percentage points higher than the HRNet baseline. Training on the COCO human key point detection dataset, test result on the COCO validation set AP is 76.7%, 2.3 percentage points higher than HRNet baseline. Three ablation experiments verify that the NLHR network for human pose estimation can exceed the existing human pose estimation network in accuracy.

Key words: human pose estimation, non-local means, non-local network module, HRNet（high-resolution network） baseline

摘要： 人体姿态估计是计算机视觉中的基础任务，其可应用于动作识别、游戏、动画制作等。受非局部均值方法的启发，设计了非局部高分辨率网络（non-local high-resolution，NLHR），在原始图像1/32分辨率的网络阶段融合非局部网络模块的，使网络有了获取全局特征的能力，从而提高人体姿态估计的准确率。NLHR网络在MPII数据集上训练，在MPII验证集上测试，PCKh@0.5评价标准下的平均准确率为90.5%，超过HRNet基线0.2个百分点；在COCO人体关键点检测数据集上训练，在COCO验证集上测试，平均准确率为76.7%，超过HRNet基线2.3个百分点。通过3组消融实验，验证NLHR网络针对人体姿态估计在精度上能够超过现有的人体姿态估计网络。

关键词: 人体姿态估计, 非局部均值, 非局部网络模块, HRNet基线

SUN Qixiang, ZHANG Ruizhe, HE Ning, ZHANG Congcong. Human Pose Estimation Method Based on Non-Local High-Resolution Networks[J]. Computer Engineering and Applications, 2022, 58(13): 227-234.

孙琪翔, 张睿哲, 何宁, 张聪聪. 基于非局部高分辨率网络的人体姿态估计方法[J]. 计算机工程与应用, 2022, 58(13): 227-234.

References

[1] 王冉.基于深度卷积神经网络的人体姿势估计研究[D].成都：电子科技大学，2016.
WANG R.Human posture estimation based on deep convolutional neural network[D].Chengdu：University of Electronic Science and Technology of China，2016.
[2] FANG H S，XIE S Q，TAI Y W，et al.RMPE：Regional multi-person pose estimation[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision.Piscataway：IEEE，2017：2334-2343.
[3] CHEN Y L，WANG Z C，PENG Y X，et al.Cascaded pyramid network for multi-person pose estimation[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway：IEEE，2018：7103-7112.
[4] HUANG S，GONG M，TAO D.A coarse-fine network for key point localization[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision.Piscataway：IEEE，2017：3028-3037.
[5] XIAO B，WU H，WEI Y.Simple baselines for human pose estimation and tracking[C]//Proceedings of the 2018 European Conference on Computer Vision.Cham：Springer，2018：466-481.
[6] NEWELL A，YANG K，DENG J.Stacked hourglass networks for human pose estimation[C]//Proceedings of the 2016 European Conference on Computer Vision.Cham：Springer，2016：483-499.
[7] YANG W，SHUAN L，OUYANG W L，et al.Learning feature pyramids for human pose estimation[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision.Piscataway：IEEE，2017：1281-1290.
[8] PISHCHULIN L，ELDAR I，SIYU T，et al.DeepCut：Joint subset partition and labeling for multi person pose estimation[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway：IEEE，2016：4929-4937.
[9] IQBAL U，GALL J.Multi-person pose estimation with local joint-to-person associations[C]//Proceedings of the 2016 European Conference on Computer Vision.Cham：Springer，2016：627-642.
[10] INSAFUTDINOV E，LEONID P，BJOERN A，et al.DeeperCut：A deeper，stronger，and faster multi-person pose estimation model[C]//Proceedings of the 2016 European Conference on Computer Vision.Cham：Springer，2016：34-50.
[11] HE K M，ZHANG X Y，REN S Q，et al.Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway：IEEE，2016：770-778.
[12] LEVINKOV E，JONAS U，SIYU T，et al.Joint graph decomposition & node labeling：Problem，algorithms，applications[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway：IEEE，2017：6012-6020.
[13] VARADARAJAN S，PARUAL D，OMESH T.A greedy part assignment algorithm for real-time multi-person 2D pose estimation[C]//Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision.Piscataway：IEEE，2018：418-426.
[14] CAO Z，TOMAS S，WEI S E，et al.Realtime multi-person 2D pose estimation using part affinity fields[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway：IEEE，2017：7291-7299.
[15] NEWELL A，HUANG Z，JIA D.Associative embedding：End-to-end learning for joint detection and grouping[C]//Advances in Neural Information Processing Systems，2017：2277-2287.
[16] XIA F T，PENG W，CHEN X J，et al.Joint multi-person posee estimation and semantic part segmentation[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway：IEEE，2017：6769-6778.
[17] PAPANDREOU G，ZHU T，CHEN L C，et al.PersonLab：Person pose estimation and instance segmentation with a bottom-up，part-based，geometric embedding model[C]//Proceedings of the 2018 European Conference on Computer Vision.Cham：Springer，2018：269-286.
[18] TANG W，PEI Y，YING W.Deeply learned compositional models for human pose estimation[C]//Proceedings of the 2018 European Conference on Computer Vision.Cham：Springer，2018：190-206.
[19] ANDRILUKA M，LEONID P，PETER G，et al.2D human pose estimation：new benchmark and state of the art analysis[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway：IEEE，2014：3686-3693.
[20] LIN T Y，MICHAEL M，SERGE B，et al.Microsoft COCO：Common objects in context[C]//Proceedings of the 2014 European Conference on Computer Vision.Cham：Springer，2014：740-755.
[21] SUN K，BIN X，DONG L，et al.Deep high-resolution representation learning for human pose estimation[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway：IEEE，2019：5693-5703.
[22] BUADES A，BARTOMEU C，MOREL J M.A non-local algorithm for image denoising[C]//Proceeding of the 2005 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway：IEEE，2005：60-65.
[23] WANG X L，ROSS G，ABHINAV G，et al.Non-local neural networks[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway：IEEE，2018：7794-7803.
[24] WEI S E，VARUN R，TAKEO K，et al.Convolutional pose machines[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway：IEEE，2016：4724-4732.
[25] KINGMA D P，JIMMY B.Adam：A method for stochastic optimization[J].arXiv：1412.6980，2014.