Design of Improved ECO Multi-Object Tracker Based on Camera Motion Estimation

doi:10.3778/j.issn.1002-8331.2303-0006

Abstract

Abstract: Multiple object tracking (MOT) is susceptible to the motion and random jitter of the camera in dynamic scenes due to the linear assumption of the motion model, leading to tracking errors. A camera motion aware filter multi-object tracker (CMAFT) is designed to solve the above problems. Firstly, a new model combines camera motion estimation with the regional search characteristics of single object tracking (SOT) to compensate for the drift caused by camera motion and improve the accuracy of prediction. Secondly, an improved cascade matching method for this model is proposed to deal with the mutual occlusion and identity switching between different objects by fusing the SOT prediction. Finally, experiments are run on the MOT17 dataset to verify the effectiveness of the proposed method.

Key words: multiple object tracking（MOT）, camera motion estimation, single object tracking（SOT）

摘要： 由于运动模型采用线性假设，使得多目标跟踪（multiple object tracking，MOT）在动态场景下容易受到相机运动和随机抖动的影响，导致跟踪错误。为解决上述问题，设计了一种相机运动感知滤波多目标跟踪器（camera motion aware filter multi-object tracker，CMAFT）。首先提出一种新模型，将相机运动估计和单目标跟踪（single object tracking，SOT）的区域搜索特性相结合，以补偿由相机运动引起的偏移并提高预测的准确度；其次针对该模型提出一个改进的级联匹配方法，通过融合SOT预测以处理不同目标间的相互遮挡和身份切换问题；最后在MOT17数据集上进行实验以验证提出方法的有效性。

关键词: 多目标跟踪（MOT）, 相机运动估计, 单目标跟踪（SOT）

CHEN Jianchao, XI Zhenghao, LIU Xiang. Design of Improved ECO Multi-Object Tracker Based on Camera Motion Estimation[J]. Computer Engineering and Applications, 2024, 60(10): 285-291.

陈健超, 奚峥皓, 刘翔. 基于相机运动估计的改进ECO多目标跟踪器设计[J]. 计算机工程与应用, 2024, 60(10): 285-291.

References

[1] SHUAI B, BERNESHAWI A, LI X, et al. SiamMOT: Siamese multi-object tracking[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 12372-12382.
[2] ZHOU X, KOLTUN V, KR?HENBüHL P. Tracking objects as points[C]//Proceedings of the 16th European Conference on Computer Vision. Cham: Springer, 2020: 474-490.
[3] ZHANG Y, WANG C, WANG X, et al. FairMOT: on the fairness of detection and re-identification in multiple object tracking[J]. International Journal of Computer Vision, 2021, 129(11): 3069-3087.
[4] ZHANG Y, SUN P, JIANG Y, et al. ByteTrack: multi-object tracking by associating every detection box[J]. arXiv:2110. 06864, 2021.
[5] BEWLEY A, GE Z, OTT L, et al. Simple online and realtime tracking[C]//Proceedings of the 2016 IEEE International Conference on Image Processing, 2016: 3464-3468.
[6] KALMAN R E. A new approach to linear filtering and prediction problems[J]. Journal of Basic Engineering, 1960, 82D: 35-45.
[7] WOJKE N, BEWLEY A, PAULUS D. Simple online and realtime tracking with a deep association metric[C]//Proceedings of the 2017 IEEE International Conference on Image Processing, 2017: 3645-3649.
[8] WANG Z, ZHENG L, LIU Y, et al. Towards real-time multi-object tracking[C]//Proceedings of the 16th European Conference on Computer Vision. Cham: Springer, 2020: 107-122.
[9] VOIGTLAENDER P, KRAUSE M, OSEP A, et al. MOTs: multi-object tracking and segmentation[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 7942-7951.
[10] PENG J, WANG C, WAN F, et al. Chained-tracker: chaining paired attentive regression results for end-to-end joint multiple-object detection and tracking[C]//Proceedings of the 16th European Conference on Computer Vision. Cham: Springer, 2020: 145-161.
[11] ZHU J, YANG H, LIU N, et al. Online multi-object tracking with dual matching attention networks[C]//Proceedings of the 15th European Conference on Computer Vision, 2018: 366-382.
[12] DANELLJAN M, BHAT G, SHAHBAZ KHAN F, et al. ECO: efficient convolution operators for tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 6638-6646.
[13] FENG W, HU Z, WU W, et al. Multi-object tracking with multiple cues and switcher-aware classification[J]. arXiv: 1901.06129, 2019.
[14] YIN J, WANG W, MENG Q, et al. A unified object motion and affinity model for online multi-object tracking[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 6768-6777.
[15] ZHENG L, TANG M, CHEN Y, et al. Improving multiple object tracking with single object tracking[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 2453-2462.
[16] ZHOU X, WANG D, KR?HENBüHL P. Objects as points[J]. arXiv:1904.07850, 2019.
[17] SRIGRAROM S, CHEW K H. Hybrid motion-based object detection for detecting and tracking of small and fast moving drones[C]//Proceedings of the 2020 International Conference on Unmanned Aircraft Systems, 2020: 615-621.
[18] LI W, MU J, LIU G. Multiple object tracking with motion and appearance cues[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshops, 2019.
[19] BERGMANN P, MEINHARDT T, LEAL-TAIXE L. Tracking without bells and whistles[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, 2019: 941-951.
[20] HAN S, HUANG P, WANG H, et al. MAT: motion-aware multi-object tracking[J]. Neurocomputing, 2022, 476: 75-86.
[21] WANG S, HAN R, FENG W, et al. Multiple human tracking in non-specific coverage with wearable cameras[C]//Proceedings of the 2021 IEEE International Conference on Acoustics, Speech and Signal Processing, 2021: 2200-2204.
[22] AHARON N, ORFAIG R, BOBROVSKY B Z. BoT-SORT: robust associations multi-pedestrian tracking[J]. arXiv:2206. 14651, 2022.
[23] MUR-ARTAL R, TARDóS J D. ORB-SLAM2: an open-source SLAM system for monocular, stereo, and RGB-D cameras[J]. IEEE Transactions on Robotics, 2017, 33(5): 1255-1262.
[24] CAMPOS C, ELVIRA R, RODRíGUEZ J J G, et al. ORB-SLAM3: an accurate open-source library for visual, visual-inertial, and multimap SLAM[J]. IEEE Transactions on Robotics, 2021, 37(6): 1874-1890.
[25] HYUN J, KANG M, WEE D, et al. Detection recovery in online multi-object tracking with sparse graph tracker[J]. arXiv:2205.00968, 2022.
[26] BRASó G, LEAL-TAIXé L. Learning a neural solver for multiple object tracking[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 6247-6257.
[27] CHU P, FAN H, TAN C C, et al. Online multi-object tracking with instance-aware tracker and dynamic model refreshment[C]//Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision, 2019: 161-170.
[28] MILAN A, LEAL-TAIXé L, REID I, et al. MOT16: a benchmark for multi-object tracking[J]. arXiv:1603.00831, 2016.
[29] FISCHLER M A, BOLLES R C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM, 1981, 24(6): 381-395.
[30] BAY H, TUYTELAARS T, GOOL L V. SURF: speeded up robust features[C]//Proceedings of the 9th European Conference on Computer Vision. Berlin, Heidelberg: Springer, 2006: 404-417.
[31] LUITEN J, OSEP A, DENDORFER P, et al. HOTA: a higher order metric for evaluating multi-object tracking[J]. International Journal of Computer Vision, 2021, 129(2): 548-578.
[32] BERNARDIN K, STIEFELHAGEN R. Evaluating multiple object tracking performance: the clear MOT metrics[J]. EURASIP Journal on Image and Video Processing, 2008. DOI: 10.1155/2008/246309.
[33] KUHN H W. The Hungarian method for the assignment problem[J]. Naval Research Logistics Quarterly, 1955, 2(1/2): 83-97.
[34] CHU Q, OUYANG W, LIU B, et al. DASOT: a unified framework integrating data association and single object tracking for online multi-object tracking[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence, 2020: 10672-10679.
[35] CHU P, LING H. FAMNet: joint learning of feature, affinity and multi-dimensional assignment for online multiple object tracking[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, 2019: 6172-6181.
[36] HE Y, WEI X, HONG X, et al. Identity-quantity harmonic multi-object tracking[J]. IEEE Transactions on Image Processing, 2022, 31: 2201-2215.
[37] DAI P, WENG R, CHOI W, et al. Learning a proposal classifier for multiple object tracking[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 2443-2452.
[38] STADLER D, BEYERER J. Multi-pedestrian tracking with clusters[C]//Proceedings of the 2021 17th IEEE International Conference on Advanced Video and Signal Based Surveillance, 2021: 1-10.