结合检测修复和特征平滑更新的多目标追踪

doi:10.3778/j.issn.1002-8331.2401-0459

摘要/Abstract

摘要： 多目标跟踪可以分为两个子任务，目标检测任务和数据关联任务。随着多目标跟踪数据集的日益复杂，密集人群引起的遮挡问题严重影响了检测和关联的准确性。为此提出了两个改进模块。检测修复模块，对目标检测的结果进行修复。前一帧轨迹的外观特征和当前帧特征作互相关运算得到的最高相应点作为潜在的检测框，将其与初始检测框融合，实现检测修复，并引入特征局部扩展思想减少该方式产生的误检。平滑的特征更新模块，如何确定和衡量遮挡程度是进一步优化的前提。为此提出特征独立性作为衡量遮挡指标，利用检测框下边界判断遮挡关系，以检测框重合部分比上自身计算特征独立性，并在此基础上设计平滑的外观特征更新算法。在密集多目标跟踪数据集MOT20上，用单阶段模型实现了最先进的性能，HOTA提升到55.3%和MOTA提升到70.7%。

关键词: 多目标跟踪, 检测修复, 遮挡, 外观特征更新

Abstract: Multi-object tracking can be divided into two sub-tasks： target detection and data association. With the increasing complexity of multi-object tracking datasets, the problem of occlusion caused by dense crowds significantly affects the accuracy of detection and association. To address this, two innovative modules are proposed. One is detection repair module, it repairs the results of target detection. The highest response point obtained by cross-correlating the appearance features of the previous frame trajectory with the current frame features serves as a potential detection box. It is fused with the initial detection box to achieve detection restoration. The concept of local feature expansion is introduced to mitigate false positives generated by this approach. Another is smooth feature update module,how to determine and measure the degree of occlusion is a prerequisite for further optimization. For this reason, feature independence is proposed as a measure of occlusion. The lower boundary of the detection frame is used to determine the occlusion relationship. The overlapped part of the detection frame is compared to itself to calculate the feature independence. On this basis, a smooth appearance feature update algorithm is designed. On the dense multi-object tracking dataset MOT20, state-of-the-art performance is achieved on a single-stage model,with HOTA improving to 55.3% and MOTA improving to 70.7%.

Key words: multi-object tracking, repair detection, occlusion, appearance update

李宗民, 杨少波, 王君伍. 结合检测修复和特征平滑更新的多目标追踪[J]. 计算机工程与应用, 2025, 61(9): 202-210.

LI Zongmin, YANG Shaobo, WANG Junwu. Multi-Object Tracking Combining Detection Restoration and Feature Smooth Update[J]. Computer Engineering and Applications, 2025, 61(9): 202-210.

参考文献

[1] BEWLEY A, GE Z, OTT L, et al. Simple online and realtime tracking[C]//Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), 2016: 3464-3468.
[2] SUN S J, AKHTAR N, SONG H S, et al. Deep affinity network for multiple object tracking[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 43(1): 104-119.
[3] XU J, CAO Y, ZHANG Z, et al. Spatial-temporal relation networks for multi-object tracking[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019: 3988-3998.
[4] BERGMANN P, MEINHARDT T, LEAL-TAIZE L. Tracking without bells and whistles[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019: 941-951.
[5] MEINHARDT T, KIRILLOV A, LEAL-TAIXE L, et al. Trackformer: multi-object tracking with transformers[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 8844-8854.
[6] ZHOU X, KOLTUN V, KRAHENBUHL P. Tracking objects as points[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer International Publishing, 2020: 474-490.
[7] ZHANG Y, SUN P, JIANG Y, et al. Bytetrack: multi-object tracking by associating every detection box[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 1-21.
[8] GE Z, LIU S, WANG F, et al. YOLOX: Exceeding YOLO series in 2021[J]. arXiv:2107.08430, 2021.
[9] 袁姮, 赵肖祎. 流形背景感知的相关滤波目标跟踪[J]. 计算机科学与探索, 2023, 17(6): 1373-1386.
YUAN H, ZHAO X Y. Manifold background-aware correlation filter target tracking[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(6): 1373-1386.
[10] ZOU Z, HUANG J, LUO P. Compensation tracker: reprocessing lost object for multi-object tracking[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2022: 307-317.
[11] KALMAN R E. A new approach to linear filtering and prediction problems[J]. Journal of Basic Engineering, 1960, 82(1): 35-45.
[12] LIANG C, ZHANG Z, ZHOU X, et al. One more check: making “fake background” be tracked again[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2022: 1546-1554.
[13] HYUN J, KANG M, WEE D, et al. Detection recovery in online multi-object tracking with sparse graph tracker[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023: 4850-4859.
[14] 汪强, 卢先领. 时空模板更新的Transformer目标跟踪算法[J]. 计算机科学与探索, 2023, 17(9): 2161-2173.
WANG Q, LU X L. Transformer object tracking algorithm based on spatio-temporal template update[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(9): 2161-2173.
[15] PANG H, SU J, MA R, et al. Multiple templates transformer for visual object tracking[J]. Knowledge-Based Systems, 2023(25): 280.
[16] WANG Z, ZHENG L, LIU Y, et al. Towards real-time multi-object tracking[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer International Publishing, 2020: 107-122.
[17] DU Y, ZHAO Z, SONG Y, et al. StrongSORT: make deepSORT great again[J]. IEEE Transactions on Multimedia, 2023.
[18] DENDORFER P, REZATOFIGHI H, MILAN A, et al. MOT20: a benchmark for multi object tracking in crowded scenes[J]. arXiv:2003.09003, 2020.
[19] CHEN L, AI H, ZHUANG Z, et al. Real-time multiple people tracking with deeply learned candidate selection and person re-identification[C]//Proceedings of the 2018 IEEE International Conference on Multimedia and Expo (ICME), 2018: 1-6.
[20] WOJKE N, BEWLEY A, PAULUS D. Simple online and realtime tracking with a deep association metric[C]//Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP), 2017: 3645-3649.
[21] YU F, LI W, LI Q, et al. POI: multiple object tracking with high performance detection and appearance feature[J]. arXiv:1610.06136, 2016.
[22] LI C, LI L, JIANG H, et al. YOLOv6: a single-stage object detection framework for industrial applications[J]. arXiv:2209. 02976, 2022.
[23] REN S, HE K, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[C]//Advances in Neural Information Processing Systems, 2015.
[24] LU Z, RATHOD V, VOTEL R, et al. RetinaTrack: online single stage joint detection and tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 14668-14678.
[25] PANG J, QIU L, LI X, et al. Quasi-dense similarity learning for multiple object tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 164-173.
[26] ZHANG Y, WANG C, WANG X, et al. FairMOT: on the fairness of detection and re-identification in multiple object tracking[J]. International Journal of Computer Vision, 2021, 129: 3069-3087.
[27] WU J, CAO J, SONG L, et al. Track to detect and segment: an online multi-object tracker[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 12352-12361.
[28] WANG Q, ZHENG Y, PAN P, et al. Multiple object tracking with correlation learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 3876-3886.
[29] YASEEN S A, SASI S. Robust algorithm for object detec tion and tracking in a dynamic scene[J]. Journal of Image and Graphics, 2014, 2(1): 41-45.
[30] MUDARISOV S, GAINULLIN I, GABITOV I, et al. Soil compaction management: reduce soil compaction using a chain-track tractor[J]. Journal of Terramechanics, 2020, 89: 1-12.
[31] PENG J, WANG C, WAN F, et al. Chained-tracker: chaining paired attentive regression results for end-to-end joint multiple-object detection and tracking[J]. arXiv:2007. 14557, 2020.
[32] LIANG C, ZHANG Z, ZHOU X, et al. Rethinking the competition between detection and reid in multiobject tracking[J]. IEEE Transactions on Image Processing, 2022, 31: 3182-3196.
[33] KUHN H W. The Hungarian method for the assignment problem[J]. Naval Research Logistics Quarterly, 1955, 2(1/2): 83-97.
[34] ESS A, LEIBE B, SCHINDLER K, et al. A mobile vision system for robust multi-person tracking[C]//Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, 2008: 1-8.
[35] ZHANG S, BENENSON R, SCHIELE B. Citypersons: a diverse dataset for pedestrian detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 3213-3221.
[36] ZHENG L, ZHANG H, SUN S, et al. Person re-identification in the wild[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 1367-1376.
[37] DOLLAR P, WOJEK C, SCHIELE B, et al. Pedestrian detection: a benchmark[C]//Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009: 304-311.
[38] XIAO T, LI S, WANG B, et al. Joint detection and identification feature learning for person search[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 3415-3424.
[39] STANDLER D, BEYERER J. Improving multiple pedestrian tracking by track management and occlusion handling[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 10958-10967.
[40] BERNARDIN K, STIEFELHAGEN R. Evaluating multiple object tracking performance: the clear MOT metrics[J]. EURASIP Journal on Image and Video Processing, 2008: 2463090.
[41] LUITEN J, OSEP A, DENDORFER P, et al. HOTA: a higher order metric for evaluating multi-object tracking[J]. International Journal of Computer Vision, 2021, 129: 548-578.
[42] ZHANG Y, SHENG H, WU Y, et al. Multiplex labeling graph for near-online tracking in crowded scenes[J]. IEEE Internet of Things Journal, 2020, 7(9): 7892-7902.
[43] IBRAHIM N, DARLIS A R, KUSUMOPUTRO B. Performance analysis of fuzzy-weighted multiple instance learning on thermal video-based visual tracking[J]. Journal of Image and Graphics (United Kingdom), 2022, 10(2): 88-94.
[44] HORNAKOVA A, KAISER T, SWOBODA P, et al. Making higher order mot scalable: an efficient approximate solver for lifted disjoint paths[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 6330-6340.
[45] DAI P, WENG R, CHOI W, et al. Learning a proposal classifier for multiple object tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 2443-2452.
[46] KAWANISHI Y. Label-based multiple object ensemble tracking with randomized frame dropping[C]//Proceedings of the 2022 26th International Conference on Pattern Recognition (ICPR), 2022: 900-906.
[47] SHAO S, ZHAO Z, LI B, et al. Crowdhuman: a benchmark for detecting human in a crowd[J]. arXiv:1805.00123, 2018.
[48] CHEN M, LIAO Y, LIU S, et al. TR-MOT: multi-object tracking by reference[J]. arXiv:2203.16621, 2022.
[49] WANG Y, KITANI K, WENG X. Joint object detection and multi-object tracking with graph neural networks[C]//Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), 2021: 13708-13715.
[50] CAI J, XU M, LI W, et al. MeMOT: multi-object tracking with memory[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 8090-8100.