Weakly Supervised Person Search Combining Dual-Path Network and Multi-Label Classification

doi:10.3778/j.issn.1002-8331.2201-0024

Abstract

Abstract: Supervised person search relies entirely on person bounding boxes and person identity labels. It is easy to annotate person bounding boxes in large-scale datasets, but it’s extremely difficult to collect person identity association information cross multi-camera. In order to get rid of the dependence on person identity label, a weakly supervised person search combining dual-path network and multi-label classification with only person bounding box label method is proposed. In order to reduce the background information interference caused by person detection error, the combination of panoramic image branch and the cutting image branch is used to study the dual-path person instance feature, and to enhance the representation of the semantic information of the person area by minimizing the feature of the same instances in the two paths. At the same time, for the learning of the person re-identification feature, the single class label is assigned to each instance, then prediction multi-label by feature similarity threshold and mutual neighbor methods, and learning feature by multi-label based on the non-parametric classifier. The experimental results show that the mAP and top-1 of CUHK-SYSU dataset are 84.2% and 86.0%, respectively, and the mAP and top-1 of PRW dataset are 38.8% and 85.1%, respectively, showing excellent performance compared with the latest method.

Key words: person search, weakly supervised learning, person re-identification, multi-label classification

摘要： 有监督的行人搜索方法依赖于行人框和行人身份的精细标记，而大规模数据集下行人框的标注较易实现，但跨图像的行人身份标记却非常困难。为了摆脱对行人身份标签的依赖，只借助行人框标注，设计了结合双路网络和多标签分类的弱监督行人搜索方法，同时对行人定位和再识别任务进行联合优化。为减少行人定位误差引起的背景信息干扰，融合全景图像分支和裁剪图像分支进行双路特征学习，通过最小化两分支中同行人实例的特征差异来增强网络对行人区域语义信息的表征能力。同时，为解决无身份标签监督下行人可辨识特征的学习问题，设计了在线多标签预测，通过相似度阈值和互近邻原则来提升标签的可靠性。最后利用基于特征存储的非参数化分类器进行多标签分类学习，鼓励相似度高的特征聚合，相似度低的特征分离。实验评估在CUHK-SYSU数据集的mAP和top-1分别达到84.2%和86.0%，在PRW数据集的mAP和top-1分别达到38.8%和85.1%，与最新的方法相比性能表现突出。

关键词: 行人搜索, 弱监督学习, 行人再识别, 多标签分类

ZHANG Jianhe, JIANG Xiaoyan. Weakly Supervised Person Search Combining Dual-Path Network and Multi-Label Classification[J]. Computer Engineering and Applications, 2023, 59(9): 159-166.

张建贺, 姜晓燕. 结合双路网络和多标签分类的弱监督行人搜索[J]. 计算机工程与应用, 2023, 59(9): 159-166.

References

[1] XU Y，MA B，HUANG R，et al.Person search in a scene by jointly modeling people commonness and person uniqueness[C]//Proceedings of the 22nd ACM International Conference on Multimedia，2014：937-940.
[2] XIAO T，LI S，WANG B，et al.Joint detection and identification feature learning for person search[C]//Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition，2017：3376-3385.
[3] ZHENG L，ZHANG H，SUN S，et al.Person re-identification in the wild[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：1367-1376.
[4] HAN C，YE J，ZHONG Y，et al.Re-id driven localization refinement for person search[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：9814-9823.
[5] CHEN D，ZHANG S，YANG J，et al.Norm-aware embedding for efficient person search[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：12615-12624.
[6] LI Z，MIAO D.Sequential end-to-end network for efficient person search[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2021：2011-2019.
[7] SONG L，WANG C，ZHANG L，et al.Unsupervised domain adaptive re-identification：theory and practice[J].Pattern Recognition，2020，102：73-82.
[8] ZHANG X，CAO J，SHEN C，et al.Self-training with progressive augmentation for unsupervised cross-domain person re-identification[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：8222-8231.
[9] GE Y，ZHU F，CHEN D，et al.Self-paced contrastive learning with hybrid memory for domain adaptive object re-id[C]//34th Conference on Neural Information Processing Systems，2021.
[10] DENG W，ZHENG L，YE Q，et al.Image-image domain adaptation with preserved self-similarity and domain-dissimilarity for person re-identification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：994-1003.
[11] GE Y，ZHU F，ZHAO R，et al.Structured domain adaptation with online relation regularization for unsupervised person re-id[J].arXiv：2003.06650，2020.
[12] YU H X，ZHENG W S，WU A，et al.Unsupervised person re-identification by soft multilabel learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：2143-2152.
[13] ZHONG Z，ZHENG L，LUO Z，et al.Invariance matters：exemplar memory for domain adaptive person re-identification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：598-607.
[14] WANG D，ZHANG S.Unsupervised person re-identification via multi-label classification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：10981-10990.
[15] CAO S，LIU Y.An iterative unsupervised person search algorithm on natural scene images[C]//Proceedings of IEEE 2019 Chinese Automation Congress，2019：3779-3783.
[16] YAN Y，LI J，LIAO S，et，al.Exploring visual context for weakly supervised person search[J].arXiv：2106.10506，2021.
[17] YAN L，ZHENG W，WANG F，et al.Weakly supervised person search[C]//Proceedings of IEEE 7th International Conference on Data Science and Advanced Analytics，2020：188-196.
[18] REN S，HE K，GIRSHICK R，et al.Faster R-CNN：towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2017：1137-1149.
[19] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[20] ZHANG X，WANG X，BIAN J W，et al.Diverse knowledge distillation for end-to-end person search[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2021：3412-3420.
[21] WU Z，XIONG Y，YU S X，et al.Unsupervised feature learning via non-parametric instance discrimination[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition，2018：3733-3742.
[22] ZHONG Z，ZHENG L，CAO D，et al.Re-ranking person re-identification with k-reciprocal encoding[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition，2017：3652-3661.
[23] XIAO J，XIE Y，TILLO T，et al.Ian：the individual aggregation network for person search[J].Pattern Recognition，2019，87：332-340.
[24] LIU H，FENG J，JIE Z，et al.Neural person search machines[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：493-501.
[25] MUNJAL B，AMIN S，TOMBARI F，et al.Query-guided end-to-end person search[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：811-820.
[26] DONG W，ZHANG Z，SONG C，et al.Bi-directional interaction network for person search[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：2839-2848.