Multi-Scale Lightweight Human Pose Estimation with Dense Connections

doi:10.3778/j.issn.1002-8331.2208-0209

Abstract

Abstract: To solve the problem of how to improve the performance of the network while being lightweight, a multi-scale lightweight human pose estimation（LDMNet） combined with densely connected network is proposed on the basis of simple and lightweight human pose estimation（LPN）. The bottleneck structure of downsampling is redesigned, and the dense connection is combined with deep separable convolution and multi-scale feature extraction. A lightweight and efficient feature extraction structure is constructed, and the multi-feature extraction of the hollow space convolution pool pyramid is improved. Experiments on the MPII dataset and COCO dataset show that compared with the baseline method LPN, the average accuracy on the MPII dataset is increased by 1.9 percentage points and the average accuracy on the COCO dataset is increased by 3.2 percentage points when the complexity of the model is slightly increased by LDMNet. In addition, compared with the latest lightweight network LiteHRNet, the average accuracy of the COCO validation set and MPII validation set is also improved by 2.9 and 1.5 percentage points, indicating that the network effectively improves the recognition accuracy of the network on the basis of lightweight.

Key words: human pose estimation, dense connection network, multi-scale feature extraction, lightweight network

摘要： 针对如何在保证轻量化的同时提升网络的性能问题，以轻量级简单基线（LPN）为基础提出的融入密集连接的多尺度轻量级人体姿态估计（LDMNet），重新设计了下采样的瓶颈结构，将密集连接与深度可分离卷积以及多尺度特征提取相结合，构建了一个轻量高效的特征提取结构，同时改进了空洞空间卷积池化金字塔多特征进行再提取。在MPII数据集以及COCO数据集上的实验表明，与基线方法LPN相比，LDMNet在少量增加参数量和计算量的情况下，在MPII验证集上的平均准确率提升了1.9个百分点，在COCO验证集上的平均准确率提升了3.2个百分点，另外与最新的轻量级网络LiteHRNet相比在COCO验证集与MPII验证集上平均准确率也取得了2.9和1.5个百分点的提升，该网络在轻量化的基础上有效地提升了网络的识别精度。

关键词: 人体姿态估计, 密集连接网络, 多尺度特征提取, 轻量级网络

GAO Kun, LI Wanggen, SHU Yang, WANG Zhige, GE Yingkui. Multi-Scale Lightweight Human Pose Estimation with Dense Connections[J]. Computer Engineering and Applications, 2022, 58(24): 196-204.

高坤, 李汪根, 束阳, 王志格, 葛英奎. 融入密集连接的多尺度轻量级人体姿态估计[J]. 计算机工程与应用, 2022, 58(24): 196-204.

References

[1] 钱慧芳，易剑平，付云虎.基于深度学习的人体动作识别综述[J].计算机科学与探索，2021，15（3）：438-455.
QIAN H F，YI J P，FU Y H.Review of human action recognition based on deep learning[J].Journal of Frontiers of Computer Science and Technology，2021，15（3）：438-455.
[2] LUVIZON D C，PICARD D，TABIA H.2d/3d pose estimation and action recognition using multitask deep learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：5137-5146.
[3] 苏江毅，宋晓宁，吴小俊，等.多模态轻量级图卷积人体骨架行为识别方法[J].计算机科学与探索，2021，15（4）：733-742.
SU J Y，SONG X N，WU X J，et al.Skeleton based action recognition algorithm on multi-modal lightweight graph convolutional network[J].Journal of Frontiers of Computer Science and Technology，2021，15（4）：733-742.
[4] VULETIC T，DUFFY A，HAY L，et al.Systematic literature review of hand gestures used in human computer interaction interfaces[J].International Journal of Human-Computer Studies，2019，129：74-94.
[5] 何坚，郭泽龙，刘乐园，等.基于滑动窗口和卷积神经网络的可穿戴人体活动识别技术[J].电子与信息学报，2022，44（1）：168-177.
HE J，GUO Z L，LIU L Y，et al.Human activity recognition technology based on sliding window and convolutional neural network[J].?Journal of Electronics & Information Technology，2022，44（1）：168-177.
[6] 邓益侬，罗健欣，金凤林.基于深度学习的人体姿态估计方法综述[J].计算机工程与应用，2019，55（19）：22-42.
DENG Y N，LUO J X，JIN F L.Overview of human pose estimation methods based on deep learning[J].Computer Engineering Applications，2019，55（19）：22-42.
[7] NEWELL A，YANG K，DENG J.Stacked hourglass networks for human pose estimation[C]//European Conference on Computer Vision.Cham：Springer，2016：483-499.
[8] CHEN Y，WANG Z，PENG Y，et al.Cascaded pyramid network for multi-person pose estimation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：7103-7112.
[9] XIAO B，WU H，WEI Y.Simple baselines for human pose estimation and tracking[C]//Proceedings of the European Conference on Computer Vision，2018：466-481.
[10] SUN K，XIAO B，LIU D，et al.Deep high-resolution representation learning for human pose estimation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：5693-5703.
[11] ZHANG W Q，FANG J M，WANG X G，et al.Efficient pose：efficient human pose estimation with neural architecture search[J].Computational Visual Media，2021，7（3）：335-347.
[12] ZHANG Z，TANG J，WU G.Simple and lightweight human pose estimation[J].arXiv：1911.10346，2019.
[13] YU C，XIAO B，GAO C，et al.Lite-hrnet：a lightweight high-resolution network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：10440-10450.
[14] LEE Y，HWANG J，LEE S，et al.An energy and GPU-computation efficient backbone network for real-time object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops，2019.
[15] HUANG G，LIU Z，VAN DER MAATEN L，et al.Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：4700-4708.
[16] ZHANG X，ZHOU X，LIN M，et al.Shufflenet：an extremely efficient convolutional neural network for mobile devices[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：6848-6856.
[17] WANG R J，LI X，LING C X.Pelee：a real-time object detection system on mobile devices[J].Advances in Neural Information Processing Systems，2018，31.
[18] 邓辉，徐杨.融入注意力和密集连接的轻量型人体姿态估计[J].计算机工程与应用，2022，58（16）：265-273.
DENG H，XU Y.Lightweight human pose estimation based on attention and dense connection[J].Computer Engineering and Applications，2022，58（16）：265-273.
[19] HOU Q，ZHOU D，FENG J.Coordinate attention for efficient mobile network design[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：13713-13722.
[20] HU J，SHEN L，SUN G.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：7132-7141.
[21] WANG Q，WU B，ZHU P，et al.ECA-net：efficient channel attention for deep convolutional neural networks[C]//CVF Conference on Computer Vision and Pattern Recognition，2020.
[22] WANG Y，LI M，CAI H，et al.Lite pose：efficient architecture design for 2d human pose estimation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2022：13126-13136.
[23] HOWARD A G，ZHU M，CHEN B，et al.Mobilenets：efficient convolutional neural networks for mobile vision applications[J].arXiv：1704.04861，2017.
[24] LI H，LI J，WEI H，et al.Slim-neck by GSConv：a better design paradigm of detector architectures for autonomous vehicles[J].arXiv：2206.02424，2022.
[25] CHEN L C，PAPANDREOU G，SCHROFF F，et al.Rethinking atrous convolution for semantic image segmentation[J].arXiv：1706.05587，2017.
[26] ANDRILUKA M，PISHCHULIN L，GEHLER P，et al.2d human pose estimation：new benchmark and state of the art analysis[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2014：3686-3693.
[27] LIN T Y，MAIRE M，BELONGIE S，et al.Microsoft coco：common objects in context[C]//European Conference on Computer Vision.Cham：Springer，2014：740-755.