基于改进FairMOT的多目标跟踪算法

doi:10.3778/j.issn.1002-8331.2302-0314

摘要/Abstract

摘要： 针对复杂环境下多目标跟踪出现漏检与数据关联算法不友好导致目标之间频繁发生切换等问题，提出了基于FairMOT框架的多目标跟踪算法MFMOT。设计了轻量化多支路注意力模块，利用通道分组降低复杂度，从三个维度进行特征增强，使网络筛选提取到特征信息。重识别分支采用PolyLoss损失函数，增强同类目标之间的语义信息以区分同类不同目标对象。提出多特征融合相似度矩阵，通过融合多种特征相似度矩阵得到最优的相似度矩阵，降低目标之间身份切换次数。实验结果表明，在MOT17与MOT20数据集中HOTA分别为61.5%与56.1%，相比原有FairMOT模型分别提升2.2个百分点与2.3个百分点。与此同时，将多特征融合相似度矩阵应用至与FairMOT相同模式的多目标跟踪方法中，HOTA、MOTA与IDF1均得到提升。

关键词: 多目标跟踪, 重识别, 注意力机制, 相似度矩阵

Abstract: In response to problems such as missed detections and unfriendly data association algorithms leading to frequent switching among objects in complex environments, a multi-object tracking algorithm MFMOT that utilizes the FairMOT framework as its foundation is proposed. Firstly, a lightweight multi-branch attention module is designed, which utilizes channel grouping to reduce complexity and enhances features from three dimensions, enabling the network to select and extract feature information. Secondly, the re-identification branch uses the PolyLoss loss function to enhance the semantic information between similar objects to distinguish different objects of the same type. Finally, a multi-feature fusion similarity matrix is proposed to obtain the optimal similarity matrix by fusing multiple feature similarity matrices, reducing the number of identity switches between targets. The experimental results show that the HOTA scores are 61.5% and 56.1% in the MOT17 and MOT20 datasets respectively, which improves by 2.2 percentage points and 2.3 percentage points compared to the original FairMOT model. Furthermore, when applying the multi-feature fusion similarity matrix to a multi-object tracking method with the same mode as FairMOT, improvements in HOTA, MOTA, and IDF1 are observed.

Key words: multi-object tracking, re-identification, attention mechanism, similarity matrix

李旺, 张娜娜. 基于改进FairMOT的多目标跟踪算法[J]. 计算机工程与应用, 2024, 60(11): 139-146.

LI Wang, ZHANG Nana. Multi-Object Tracking Algorithm Based on Improved FairMOT[J]. Computer Engineering and Applications, 2024, 60(11): 139-146.

参考文献

[1] CIAPARRONE G, SáNCHEZ F L, TABIK S, et al. Deep learning in video multi-object tracking: a survey[J]. Neurocomputing, 2020, 381: 61-88.
[2] 晏康, 曾凤彩, 何宁, 等. 引入注意力机制的JDE多目标跟踪方法[J]. 计算机工程与应用, 2022, 58(21): 189-196.
YAN K, ZENG F C, HE N, et al. JDE multi object tracking method with attention mechanism[J]. Computer Engineering and Applications, 2022, 58(21): 189-196.
[3] 彭嘉淇, 王涛, 陈柯安, 等. 结合时空一致性的FairMOT跟踪算法优化[J]. 中国图象图形学报, 2022, 27(9): 2749-2760.
PENG J Q, WANG T, CHEN K A, et al. Spatio-temporal consistency based FairMOT tracking algorithm optimization[J]. Journal of Image and Graphics, 2022, 27(9): 2749-2760.
[4] 王黎明, 孙俊, 陈祺东. 加强重识别的行人多目标跟踪算法[J]. 计算机工程与应用, 2022, 58(21): 213-222.
WANG L M, SUN J, CHEN Q D. Pedestrain multi-object tracking algorithm with strengthened re-identification[J]. Computer Engineering and Applications, 2022, 58(21): 213-222.
[5] WANG Z, ZHENG L, LIU Y, et al. Towards real-time multi-object tracking[C]//Proceedings of the European Conference on Computer Vision, 2020: 107-122.
[6] ZHANG Y, WANG C, WANG X, et al. FairMOT: on the fairness of detection and re-identification in multiple object tracking[J]. International Journal of Computer Vision, 2021, 129(11): 3069-3087.
[7] TOKMAKOV P, LI J, BURGARD W, et al. Learning to track with object permanence[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 10860-10869.
[8] LIU W, SONG Y, CHEN D, et al. Deformable object tracking with gated fusion[J]. IEEE Transactions on Image Processing, 2019, 28(8): 3766-3777.
[9] WU J, CAO J, SONG L, et al. Track to detect and segment: an online multi-object tracker[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 12352-12361.
[10] PANG J, QIU L, LI X, et al. Quasi-dense similarity learning for multiple object tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 164-173.
[11] LIANG C, ZHANG Z, ZHOU X, et al. Rethinking the competition between detection and ReID in multiobject tracking[J]. IEEE Transactions on Image Processing, 2022, 31: 3182-3196.
[12] WANG Q, ZHENG Y, PAN P, et al. Multiple object tracking with correlation learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 3876-3886.
[13] YU E, LI Z, HAN S, et al. RelationTrack: relation-aware multiple object tracking with decoupled representation[J]. IEEE Transactions on Multimedia, 2023, 25: 2686-2697.
[14] BEWLEY A, GE Z, OTT L, et al. Simple online and realtime tracking[C]///Proceedings of the 2016 IEEE International Conference on Image Processing, 2016: 3464-3468.
[15] ZHANG Y, SUN P, JIANG Y, et al. ByteTrack: multi-object tracking by associating every detection box[J]. arXiv:2110.06864, 2021.
[16] CAO J, WENG X, KHIRODKAR R, et al. Observation-centric SORT: rethinking SORT for robust multi-object tracking[J]. arXiv:2203.14360, 2022.
[17] WOJKE N, BEWLEY A, PAULUS D. Simple online and realtime tracking with a deep association metric[C]//Proceedings of the 2017 IEEE International Conference on Image Processing, 2017: 3645-3649.
[18] 贺愉婷, 车进, 吴金蔓. 基于YOLOv5和重识别的行人多目标跟踪方法[J]. 液晶与显示, 2022, 37(7): 880-890.
HE Y T, CHE J, WU J M. Pedestrian multi-target tracking method based on YOLOv5 and person re-identification[J]. Chinese Journal of Liquid Crystals and Displays, 2022, 37(7): 880-890.
[19] LENG Z, TAN M, LIU C, et al. PolyLoss: a polynomial expansion perspective of classification loss functions[J]. arXiv:2204.12511, 2022.
[20] ESS A, LEIBE B, SCHINDLER K, et al. A mobile vision system for robust multi-person tracking[C]//Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, 2008: 1-8.
[21] ZHANG S, BENENSON R, SCHIELE B. CityPersons: a diverse dataset for pedestrian detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 3213-3221.
[22] DOLLáR P, WOJEK C, SCHIELE B, et al. Pedestrian detection: a benchmark[C]//Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009: 304-311.
[23] XIAO T, LI S, WANG B, et al. Joint detection and identification feature learning for person search[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 3415-3424.
[24] ZHENG L, ZHANG H, SUN S, et al. Person re-identification in the wild[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 1367-1376.
[25] SHAO S, ZHAO Z, LI B, et al. CrowdHuman: a benchmark for detecting human in a crowd[J]. arXiv:1805.00123, 2018.
[26] LUITEN J, OSEP A, DENDORFER P, et al. HOTA: a higher order metric for evaluating multi-object tracking[J]. International Journal of Computer Vision, 2021, 129(2): 548-578.
[27] ZHENG L, TANG M, CHEN Y, et al. Improving multiple object tracking with single object tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 2453-2462.
[28] XU Y, BAN Y, DELORME G, et al. TransCenter: transformers with dense representations for multiple-object tracking[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(6): 7820-7835.
[29] SHAN C, WEI C, DENG B, et al. Tracklets predicting based adaptive graph tracking[J]. arXiv:2010.09015, 2020.
[30] LI W, XIONG Y, YANG S, et al. Semi-TCL: semi-supervised track contrastive representation learning[J]. arXiv:2107.02396, 2021.
[31] STADLER D, BEYERER J. On the performance of crowd-specific detectors in multi-pedestrian tracking[C]//Proceedings of the 2021 17th IEEE International Conference on Advanced Video and Signal Based Surveillance, 2021: 1-12.
[32] WANG Y, KITANI K, WENG X. Joint object detection and multi-object tracking with graph neural networks[C]//Proceedings of the 2021 IEEE International Conference on Robotics and Automation, 2021: 13708-13715.
[33] SUN P, CAO J, JIANG Y, et al. TransTrack: multiple object tracking with transformer[J]. arXiv:2012.15460, 2020.