Multi-Object Tracking Based on Key Point Detection and Association

doi:10.3778/j.issn.1002-8331.2203-0425

Abstract

Abstract: Aiming at the problem of poor correlation of object identity caused by low correlation of object information in the multi-object tracking, this paper proposes a multi-object tracking algorithm based on key point detection and correlation. Firstly, it models the object by central key point, and uses CenterNet to detect and locate the point. Combining the depth features of the object with the scale features, it constructs a joint feature extractor based on the explicit and hidden relationship between their observations. Then taking the joint feature as the object state, and the object state of the next frame is estimated by hidden Markov model. Finally, it uses the object motion information and key point scale information to propose a “secondary correlation” matching mechanism, which is proposed to realize the correlation between the estimated state and the detected object, obtains the optimal correlation matching result. Experiments are carried out on the public MOT17 dataset, after compared with some mainstream algorithms, the results show that this algorithm performs better in tracking accuracy and has good robustness to the problem of identity exchange.

Key words: machine vision, multi-object tracking, key point detection, object information correlation

摘要： 针对多目标跟踪领域中由目标信息关联性低引起的目标身份关联性差的问题，提出了一种基于关键点检测和关联的多目标跟踪算法。对目标的中心关键点建模，利用CenterNet对该点进行检测定位；将目标的深度特征与关键点尺度特征相结合，基于二者观测的显隐性关系构建一个联合特征提取器；将该联合特征作为目标的状态，通过隐马尔可夫模型估计下一帧的目标状态；利用目标的运动信息和关键点尺度信息提出“二级关联”的匹配机制，实现对该估计状态与检测目标的关联，得到最优的关联匹配结果。在公开的MOT17数据集上进行了仿真实验，并与一些主流算法进行了对比，结果表明，该算法在跟踪准确度指标表现较优，并对身份互换问题有较好的鲁棒性。

关键词: 机器视觉, 多目标跟踪, 关键点检测, 目标信息关联性

LIU Yibo, XI Zhenghao. Multi-Object Tracking Based on Key Point Detection and Association[J]. Computer Engineering and Applications, 2023, 59(13): 156-163.

刘艺博, 奚峥皓. 基于关键点检测和关联的多目标跟踪[J]. 计算机工程与应用, 2023, 59(13): 156-163.

References

[1] LI Z X，LIU J M，LI S，et al.Group targets tracking algorithm based on box particle filter[J].Acta Automatica Sinica，2015，41（4）：785-798.
[2] CIAPARRONE G，SáNCHEZ F L，TABIK S，et al.Deep learning in video multi-object tracking：a survey[J].Neurocomputing，2020，381：61-88.
[3] PANICKER S，GOSTAR A K，BAB-HADIASHAR A，et al.Recent advances in stochastic sensor control for multi-object tracking[J].Sensors，2019，19（17）：3790.
[4] EMAMI P，PARDALOS P M，ELEFTERIADOU L，et al.Machine learning methods for data association in multi-object tracking[J].ACM Computing Surveys（CSUR），2020，53（4）：1-34.
[5] XU Y，ZHOU X，CHEN S，et al.Deep learning for multiple object tracking：a survey[J].IET Computer Vision，2019，13（4）：355-368.
[6] REDMON J，DIVVALA S，GIRSHICK R，et al.You only look once：unified，real?time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：779-788.
[7] REN S，HE K，GIRSHICK R，et al.Faster R-CNN：towards real-time object detection with region proposal networks[C]//Advances in Neural Information Processing Systems，2015.
[8] ZHOU X，WANG D，KR?HENBüHL P.Objects as points[J].arXiv：1904.07850，2019.
[9] BEWLEY A，GE Z，OTT L，et al.Simple online and realtime tracking[C]//Proceedings of 2016 IEEE International Conference on Image Processing（ICIP），2016：3464-3468.
[10] WOJKE N，BEWLEY A，PAULUS D.Simple online and realtime tracking with a deep association metric[C]//Proceedings of 2017 IEEE International Conference on Image Processing（ICIP），2017：3645-3649.
[11] CHEN L，AI H，ZHUANG Z，et al.Real-time multiple people tracking with deeply learned candidate selection and person re-identification[C]//Proceedings of 2018 IEEE International Conference on Multimedia and Expo（ICME），2018：1-6.
[12] ZHOU X，KOLTUN V，KR?HENBüHL P.Tracking objects as points[C]//Proceedings of European Conference on Computer Vision，2020：474-490.
[13] ZHANG Y，WANG C，WANG X，et al.A simple baseline for multi-object tracking[J].arXiv：2004.01888，2020.
[14] LOWE D G.Distinctive image features from scale-invariant keypoints[J].International Journal of Computer Vision，2004，60（2）：91-110.
[15] GHAHRAMANI Z.An introduction to hidden Markov models and Bayesian networks[J].International Journal of Pattern Recognition and Artifical Intelligence，2001，15（1）：9-42.
[16] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[17] WANG Z.Markov chain Monte Carlo methods for lattice Gaussian sampling：convergence analysis and enhancement[J].IEEE Transactions on Communications，2019，67（10）：6711-6724.
[18] LINDBERG D V，OMRE H.Inference of the transition matrix in convolved hidden Markov models and the generalized Baum-Welch algorithm[J].IEEE Transactions on Geoscience and Remote Sensing，2015，53（12）：6443-6456.
[19] MILAN A，LEAL-TAIXé L，REID I，et al.MOT16：a benchmark for multi-object tracking[J].arXiv：1603. 00831，2016.
[20] DENDORFER P，REZATOFIGHI H，MILAN A，et al.Mot20：a benchmark for multi object tracking in crowded scenes[J].arXiv：2003.09003，2020.
[21] ZHENG L，SHEN L，TIAN L，et al.Scalable person re-identification：a benchmark[C]//Proceedings of the IEEE International Conference on Computer Vision，2015：1116-1124.
[22] LI W，ZHAO R，XIAO T，et al.Deepreid：deep filter pairing neural network for person re-identification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2014：152-159.
[23] BERNARDIN K，STIEFELHAGEN R.Evaluating multiple object tracking performance：the clear mot metrics[J].EURASIP Journal on Image and Video Processing，2008（1）：1-10.
[24] LIU J，CAO X，LI Y，et al.Online multi-object tracking using hierarchical constraints for complex scenarios[J].IEEE Transactions on Intelligent Transportation Systems，2017，19（1）：151-161.
[25] TIAN W，LAUER M，CHEN L.Online multi-object tracking using joint domain information in traffic scenarios[J].IEEE Transactions on Intelligent Transportation Systems，2019，21（1）：374-384.