Pedestrian Multi-Object Tracking Algorithm with Strengthened Re-Identification

doi:10.3778/j.issn.1002-8331.2110-0231

Abstract

Abstract: In multi-object tracking tasks, identity re-identification（Re-ID） generally depends on the quality of detection performance. Detection deviations cause the Re-ID feature to blur, thereby reducing recognition accuracy. Especially in complex scenes such as scale transformation and frequent occlusion, Re-ID is not robust, and the effect of multi-object tracking is poor. To strengthen the Re-ID performance, a pedestrian multi-object tracking algorithm is proposed. This algorithm uses CenterNet to detect objects by predicting center point heatmaps, and designs a detection deviation loss to further constraint response value on the predicted heatmaps, which alleviates the blurry problem of Re-ID features bring by inaccurate detection. Moreover, to strengthen Re-ID robustness, the algorithm implements a novel strategy to adaptively increase the learnable Re-ID features around objects center. As a result, it improves the quality of Re-ID features, and makes Re-ID task less dependent on the detection accuracy. Experimental results on MOT16 and MOT17 benchmarks show that the proposed algorithm can effectively improve the Re-ID performance in comparison with other SOTA methods, which achieves real-time tracking with 25.6 FPS.

Key words: multi-object tracking, re-identification（Re-ID）, center point detection, real-time

摘要： 在多目标跟踪任务中，重识别（re-identification，Re-ID）效果通常依赖于检测性能的好坏，检测偏差会导致Re-ID特征模糊，从而降低重识别精度。特别是在尺度变化和频繁遮挡等复杂场景下，Re-ID鲁棒性不高，多目标跟踪效果较差。针对该问题，提出一种加强重识别的行人多目标跟踪算法。该算法以CenterNet为检测器，通过预测目标中心点热力图来检测目标位置，并设计检测偏差损失加强对预测热力图响应值的约束，以缓解因检测不准确导致的Re-ID特征模糊问题。为提高Re-ID鲁棒性，提出Re-ID可学习特征动态扩充策略。该策略通过自适应扩充目标中心的Re-ID可学习特征来提高特征质量，并减小Re-ID对中心点检测精度的依赖。在MOT16和MOT17测试集上进行验证，结果表明，算法能有效提升Re-ID性能，与主流算法相比具有更好的跟踪效果，且兼顾了实时性，达到25.6 FPS。

关键词: 多目标跟踪, 重识别, 中心点检测, 实时

WANG Liming, SUN Jun, CHEN Qidong. Pedestrian Multi-Object Tracking Algorithm with Strengthened Re-Identification[J]. Computer Engineering and Applications, 2022, 58(21): 213-222.

王黎明, 孙俊, 陈祺东. 加强重识别的行人多目标跟踪算法[J]. 计算机工程与应用, 2022, 58(21): 213-222.

References

[1] CIAPARRONE G，SANCHEZ F，TABIK S，et al.Deep learning in video multi-object tracking：a survey[J].Neurocomputing，2020，381：61-88.
[2] REN S，HE K，GIRSHICK R，et al.Faster r-cnn：towards real-time object detection with region proposal networks[J].Advances in Neural Information Processing Systems，2015，28：91-99.
[3] HE K，GKIOXARI G，DOLLAR P，et al.Mask r-cnn[C]//Proceedings of the IEEE International Conference on Computer Vision，Venice，Oct 22-29，2017.Piscataway：IEEE，2017：2961-2969.
[4] REDMON J，FARHADI A.Yolov3：an incremental improve-ment[J].arXiv：1804.02767，2018.
[5] ZHOU X，WANG D，KRAHENBUHL P.Objects as points[J].arXiv：1904.07850，2019.
[6] TIAN Z，SHEN C，CHEN H，et al.Fcos：a simple and strong anchor-free object detector[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2022，44（4）：1922-1933.
[7] CHEN L，AI H，ZHUANG Z，et al.Real-time multiple people tracking with deeply learned candidate selection and person re-identification[C]//Proceedings of the 2018 IEEE International Conference on Multimedia and Expo，San Diego，July 23-27，2018.Washington：IEEE Computer Society，2018：1-6.
[8] WANG Z，ZHENG L，LIU Y，et al.Towards real-time multi-object tracking[C]//LNCS 12356：Proceedings of the 16th European Conference on Computer Vision，Glasgow，August 23-28，2020.Cham：Springer，2020：107-122.
[9] ZHANG Y，WANG C，WANG X，et al.Fairmot：on the fairness of detection and re-identification in multiple object tracking[J].arXiv：2004.01888，2020.
[10] WOJKE N，BEWLEY A，PAULUS D.Simple online and real-time tracking with a deep association metric[C]//Proceedings of the 2017 IEEE International Conference on Image Processing，Beijing，Sep 17-20，2017.Piscataway：IEEE，2017：3645-3649.
[11] BEWLEY A，GE Z Y，OTT L，et al.Simple online and realtime tracking[C]//Proceedings of the 2016 IEEE International Conference on Image Processing，Phoenix，Sep 25-28，2016.Piscataway：IEEE，2016：3464-3468.
[12] YU F，LI W，LI Q，et al.Poi：multiple object tracking with high performance detection and appearance feature[C]//European Conference on Computer Vision，Amsterdam，October 8-16，2016.Cham：Springer，2016：36-42.
[13] VOIGTLAENDER P，KRAUSE M，OSEP A，et al.Mots：multi-object tracking and segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，Long Beach，June 15-20，2019.Washington：IEEE Computer Society，2019：7942-7951.
[14] YE M，SHEN J，LIN G，et al.Deep learning for person re-identification：a survey and outlook[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2022，44（6）：2872-2893.
[15] 罗浩，姜伟，范星，等.基于深度学习的行人重识别研究进展[J].自动化学报，2019，45（11）：2032-2049.
LUO H，JIANG W，FAN X，et al.A survey on deep learning based person re-identification[J].Acta Automatica Sinica，2019，45（11）：2032-2049.
[16] 张磊，吴晓富，张索非，等.基于多分支协作的行人重识别网络[J].模式识别与人工智能，2021，34（9）：853-862.
ZHANG L，WU X F，ZHANG S F，et al.Multi-branch cooperative network for person re-identification[J].Pattern Recognition and Artificial Intelligence，2021，34（9）：853-862.
[17] 刘玉杰，周彩云，李宗民，等.基于增强特征融合网络的行人重识别方法[J].计算机辅助设计与图形学学报，2021，33（2）：232-240.
LIU Y J，ZHOU C Y，LI Z M，et al.Strong feature fusion networks for person re-identification[J].Journal of Computer-Aided Design & Computer Graphics，2021，33（2）：232-240.
[18] 陈璠，彭力.多层级重叠条纹特征融合的行人重识别[J].计算机科学与探索，2021，15（9）：1753-1761.
CHEN F，PENG L.Person re-identification based on multi-level feature fusion with overlapping[J].Journal of Frontiers of Computer Science and Technology，2021，15（9）：1753-1761.
[19] 苑春苗，牛瑛，郭涛，等.融合衣服特征迁移的行人重识别方法[J].计算机科学与探索，2021，15（9）：1740-1752.
YUAN C M，NIU Y，GUO T，et al.Person re-identification combined with clothing information transfer[J].Journal of Frontiers of Computer Science and Technology，2021，15（9）：1740-1752.
[20] 姚足，龚勋，陈锐，等.面向行人重识别的局部特征研究进展、挑战与展望[J].自动化学报，2021，47（12）：1-19.
YAO Z，GONG X，CHEN R，et al.Research progress，challenge and prospect of local features for person re-identification[J].Acta Automatica Sinica，2021，47（12）：1-19.
[21] 肖雅妮，范馨月，陈文峰.多分支融合局部特征的行人重识别算法[J].计算机工程与应用，2021，57（18）：213-219.
XIAO Y N，FAN X Y，CHEN W F.Research on person re-identification based on integrating local features undermulti-branches[J].Computer Engineering and Applications，2021，57（18）：213-219.
[22] 宋丽丽，李彬，赵俊雅，等.正态重采样的改进行人再识别度量学习算法[J].计算机工程与应用，2020，56（8）：158-165.
SONG L L，LI B，ZHAO J Y，et al.Normality resampling of improved metric learning method for person re-identification[J].Computer Engineering and Applications，2020，56（8）：158-165.
[23] CHEN H，WANG Y，LAGADEC B，et al.Joint generative and contrastive learning for unsupervised person re-identification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：2004-2013.
[24] 杨婉香，严严，陈思，等.基于多尺度生成对抗网络的遮挡行人重识别方法[J].软件学报，2020，31（7）：1943-1958.
YANG W X ，YAN Y，CHEN S，et al.Multi-scale generative adversarial network for person re-identification under occlusion[J].Journal of Software，2020，31（7）：1943-1958.
[25] YU F，WANG D，SHELHAMER E，et al.Deep layer aggregation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Salt Lake，June 18-23，2018.Piscataway：IEEE，2018：2403-2412.
[26] LIN T，GOYAL P，GIRSHICK R，et al.Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision，Venice，Oct 22-29，2017.Piscataway：IEEE，2017：2980-2988.
[27] KENDALL A，GAL Y，CIPOLLA R.Multi-task learning using uncertainty to weigh losses for scene geometry and semantics[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Salt Lake City，June 18-23，2018.Piscataway：IEEE，2018：7482-7491.
[28] KALMAN R E.A new approach to linear filtering and prediction problems[J].Transactions of the ASME-Journal of Basic Engineering，1960，82（D）：35-45.
[29] KUHN H W.The Hungarian method for the assignment 10 computer engineering and applications problem[J].Naval Research Logistics Quarterly，1955，2（1/2）：83-97.
[30] ANTON M，LAURA L，IAN R，et al.MOT16：a benchmark for multi-object tracking[J].arXiv：1603.00831，2016.
[31] LIN T，MAIRE M，BELONGIE S，et al.Microsoft coco：common objects in context[C]//LNCS 8693：European Conference on Computer Vision，Zurich，October 7-13，2014.Cham：Springer，2014：740-755.
[32] PANG B，LI Y，ZHANG Y，et al.Tubetk：adopting tubes to track multi-object in a one-step training model[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，Seattle，June 13-19，2020.Washington：IEEE Computer Society，2020：6308-6318.
[33] MAHMOUDI N，AHADI S，RAHMATI M.Multi-target tracking using CNN-based features：CNNMTT[J].Multimedia Tools and Applications，2019，78（6）：7077-7096.
[34] PENG J，WANG C，WAN F，et al.Chained-tracker：chaining paired attentive regression results for end-to-end joint multiple-object detection and tracking[C]//European Conference on Computer Vision，Glasgow，August 23-28，2020.Cham：Springer，2020：145-161.
[35] SUN S，AKHTAR N，SONG H，et al.Deep affinity network for multiple object tracking[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2019，43（1）：104-119.
[36] ZHOU X，KOLTUN V，KRAHENBUHL P.Tracking objects as points[C]//European Conference on Computer Vision，Glasgow，August 23-28，2020.Cham：Springer，2020：474-490.