Adaptive Feature Fusion Network for Crowd Counting

doi:10.3778/j.issn.1002-8331.2006-0320

Abstract

Abstract:

In an attempt to solve the problems of scale change and multi-level feature fusion in population counting method, inspired by U-Net encoder decoder structure network, an adaptive feature fusion network is proposed to carry out accurate population counting. The Adaptive Feature Fusion Module（AFFM） is proposed to efficiently aggregate the high-level semantic information and low-level spatial detail extracted by the encoder branch according to the needs of decoder branch. The Adaptive Context Extractor（ACE） is proposed to extract multi-scale context information on multiple effective field-of-views, then these features are adaptively fused to improve the robustness of the network to scale changes. By conducting exhaustive experiments on Shanghai Tech, UCF-CC-50 and UCF-QNRF, the results show that the network has high accuracy and robustness.

Key words: crowd counting, Convolutional Neural Network（CNN）, density estimation, multi?level features, scale variation, feature fusion

摘要：

针对人群计数方法中存在的尺度变化和多层级特征融合不佳的问题，基于U-Net的编码器-解码器网络结构，提出一种自适应特征融合网络，来进行精准的人群计数。提出自适应特征融合模块，根据解码器分支的需要，高效地聚合编码器分支提取的高层语义信息和底层的边缘信息；提出自适应上下文信息提取器，从不同感受野下提取多尺度的上下文信息并自适应加权融合，提高网络对于人头尺度变化的鲁棒性。在ShanghaiTech、UCF-CC-50和UCG-QNRF上的实验表明，与目前主流的人群计数算法相比，该算法具有更强的准确性和鲁棒性。

关键词: 人群计数, 卷积神经网络, 密度估计, 多层级特征, 尺度变化, 特征融合

ZUO Jianhao, JIANG Wengang. Adaptive Feature Fusion Network for Crowd Counting[J]. Computer Engineering and Applications, 2021, 57(21): 203-208.

左健豪，姜文刚. 自适应融合特征的人群计数网络[J]. 计算机工程与应用, 2021, 57(21): 203-208.

References

[1] FOOKES C，DENMAN S，LAKEMOND R，et al.Semi-supervised intelligent surveillance system for secure environments[C]//2010 IEEE International Symposium on Industrial Electronics，2010：2815-2820.
[2] SHU C，HAMPAPUR A，LU Z，et al.Intelligent surveillance system and method for integrated event based surveillance：U.S.Patent Application 11/455，251[P].2007-12-20.
[3] CASTRO J L，DELGADO M，MEDINA J，et al.Intelligent surveillance system with integration of heterogeneous information for intrusion detection[J].Expert Systems with Applications，2011，38（9）：11182-11192.
[4] DALAL N，TRIGGS B.Histograms of oriented gradients for human detection[C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition（CVPR’05），2005：886-893.
[5] ENZWEILER M，GAVRILA D M.Monocular pedestrian detection：Survey and experiments[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2008，31（12）：2179-2195.
[6] LEIBE B，SEEMANN E，SCHIELE B.Pedestrian detection in crowded scenes[C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition（CVPR’05），2005：878-885.
[7] TUZEL O，PORIKLI F，MEER P.Pedestrian detection via classification on riemannian manifolds[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2008，30（10）：1713-1727.
[8] LEMPITSKY V，ZISSERMAN A.Learning to count objects in images[C]//Proceedings of Advances in Neural Information Processing Systems，2010：1324-1332.
[9] PHAM V Q，KOZAKAYA T，YAMAGUCHI O，et al.Count forest：co-voting uncertain number of targets using random forest for crowd density estimation[C]//Proceedings of the IEEE International Conference on Computer Vision，2015：3253-3261.
[10] HINTON G E，SALAKHUTDINOV R R.Reducing the dimensionality of data with neural networks[J].Science，2006，313（5786）：504-507.
[11] LECUN Y，BOTTOU L，BENGIO Y，et al.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE，1998，86（11）：2278-2324.
[12] ZHANG Y，ZHOU D，CHEN S，et al.Single-image crowd counting via multi-column convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：589-597.
[13] LI Y，ZHANG X，CHEN D.Csrnet：dilated convolutional neural networks for understanding the highly congested scenes[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：1091-1100.
[14] VALLOLI V K，MEHTA K.W-net：reinforced u-net for density map estimation[J].arXiv：1903.11249，2019.
[15] SINDAGI V A，PATEL V M.Multi-level bottom-top and top-bottom feature fusion for crowd counting[C]//Proceedings of the IEEE International Conference on Computer Vision，2019：1002-1012.
[16] JIANG X，XIAO Z，ZHANG B，et al.Crowd counting and density estimation by trellis encoder-decoder networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2019：6133-6142.
[17] SIMONYAN K，ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].arXiv：1409. 1556，2014.
[18] RONNEBERGER O，FISCHER P，BROX T.U-net：Convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer-assisted Intervention.Cham：Springer，2015：234-241.
[19] SINDAGI V A，PATEL V M.Multi-level bottom-top and top-bottom feature fusion for crowd counting[C]//Proceedings of the IEEE International Conference on Computer Vision，2019：1002-1012.
[20] HE K，ZHANG X，REN S，et al.Spatial pyramid pooling in deep convolutional networks for visual recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，37（9）：1904-1916.
[21] ZHAO H，SHI J，QI X，et al.Pyramid scene parsing network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：2881-2890.
[22] CAO X K.Scale aggregation network for accurate and efficient crowd counting[C]//European Conference on Computer Vision，2018：757-773.
[23] ZHANG C，LI H，WANG X，et al.Cross-scene crowd counting via deep convolutional neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2015：833-841.
[24] CAO X，WANG Z，ZHAO Y，et al.Scale aggregation network for accurate and efficient crowd counting[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：734-750.
[25] GUO D，LI K，ZHA Z J，et al.Dadnet：dilated-attention-deformable convnet for crowd counting[C]//Proceedings of the 27th ACM International Conference on Multimedia，2019：1823-1832.
[26] JIANG X，XIAO Z，ZHANG B，et al.Crowd counting and density estimation by trellis encoder-decoder networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2019：6133-6142.
[27] WAN J，CHAN A.Adaptive density map generation for crowd counting[C]//Proceedings of the IEEE International Conference on Computer Vision，2019：1130-1139.
[28] ZHANG A，YUE L，SHEN J，et al.Attentional neural fields for crowd counting[C]//Proceedings of the IEEE International Conference on Computer Vision，2019：5714-5723.