JDE Multi-Object Tracking Method with Attention Mechanism

doi:10.3778/j.issn.1002-8331.2104-0049

Abstract

Abstract: Multi-object tracking is an important research content in the field of computer vision. JDE（joint detection and embedding） multi-object tracking algorithm has high inference speed and accuracy, but when the object overlap or scale is small, the tracking performance of the algorithm is bad. To solve the above problems, this paper proposes Attention-JDE, which combines attention mechanism, multi-scale fusion and other ideas, uses feature pyramid and spatial pyramid pooling to improve the detection and tracking ability of the model for small-scale objects. The spatial attention mechanism and channel attention mechanism are combined to improve the tracking performance when the objects overlap. In addition, this paper also introduces the Mish activation function to effectively reduce the number of ID switching during tracking. Compared with the original JDE method and other SOTA methods, Attention-JDE has higher tracking accuracy（MOTA）, inference speed can reach to 19.5 FPS which is high real-time performance.

Key words: multi-object tracking, attention mechanism, multi-scale fusion, feature enhancement, JDE algorithm

摘要： 多目标跟踪是计算机视觉领域的一个重要研究内容。JDE（joint detection and embedding）多目标跟踪算法推理速度和精度较高，但是当目标重叠或尺度较小时，该算法的跟踪效果较差。针对以上问题，提出了Attention-JDE，该模型结合了注意力机制、多尺度融合等思想，利用特征金字塔（feature pyramid）和空间金字塔池化（spatial pyramid pooling）提升模型对于小尺度目标的检测和跟踪能力，结合空间域注意力机制和通道域注意力机制改进模型在目标发生重叠时的跟踪效果。此外，还引入了Mish激活函数有效地降低跟踪时的ID切换次数。在MOT16数据集进行验证，结果表明，与原JDE方法以及其他主流方法相比，Attention-JDE具有更高的跟踪精度（MOTA），同时速度能够达到19.5?FPS，实时性较高。

关键词: 多目标跟踪, 注意力机制, 多尺度融合, 特征增强, JDE算法

YAN Kang, ZENG Fengcai, HE Ning, HE Yuzhe, ZHANG Ren. JDE Multi-Object Tracking Method with Attention Mechanism[J]. Computer Engineering and Applications, 2022, 58(21): 189-196.

晏康, 曾凤彩, 何宁, 贺宇哲, 张人. 引入注意力机制的JDE多目标跟踪方法[J]. 计算机工程与应用, 2022, 58(21): 189-196.

References

[1] CIAPARRONE G，SáNCHEZ F L，TABIK S，et al.Deep learning in video multi-object tracking：a survey[J].Neurocomputing，2020，381：61-88.
[2] MAHMOUDI N，AHADI S M，RAHMATI M.Multi-target tracking using CNN-based features：CNNMTT[J].Multimedia Tools and Applications，2019，78（6）：7077-7096.
[3] YU F，LI W，LI Q，et al.Poi：multiple object tracking with high performance detection and appearance feature[C]//European Conference on Computer Vision.Cham：Springer，2016：36-42.
[4] ZHOU Z，XING J，ZHANG M，et al.Online multi-target tracking with tensor-based high-order graph matching[C]//2018 24th International Conference on Pattern Recognition（ICPR），2018：1809-1814.
[5] ZHANG L，GRAY H，YE X，et al.Automatic individual pig detection and tracking in pig farms[J].Sensors，2019，19（5）：1188.
[6] WANG Z，ZHENG L，LIU Y，et al.Towards real-time multi-object tracking[J].arXiv：1909.12605，2019.
[7] ZHAN Y，WANG C，WANG X，et al.A simple baseline for multi-object tracking[J].arXiv：2004.01888，2020.
[8] MILAN A，LEAL-TAIXé L，REID I，et al.MOT16：a benchmark for multi-object tracking[J].arXiv：1603.00831，2016.
[9] VOIGTLAENDER P，KRAUSE M，OSEP A，et al.Mots：multi-object tracking and segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：7942-7951.
[10] HE K，GKIOXARI G，DOLLáR P，et al.Mask R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：2961-2969.
[11] REDMON J，FARHADI A.Yolov3：an incremental improvement[J].arXiv：1804.02767，2018.
[12] LIN T Y，DOLLáR P，GIRSHICK R，et al.Feature pyramid networks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：2117-2125.
[13] CHAUDHARI S，POLATKAN G，RAMANATH R，et al.An attentive survey of attention models[J].arXiv：1904. 02874，2019.
[14] WANG F，TAX D M J.Survey on the attention based RNN model and its applications in computer vision[J].arXiv：1601.06823，2016.
[15] HU J，SHEN L，SUN G.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：7132-7141.
[16] WOO S，PARK J，LEE J Y，et al.Cbam：convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：3-19.
[17] WANG Q，WU B，ZHU P，et al.ECA-Net：efficient channel attention for deep convolutional neural networks[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），2020.
[18] JADERBERG M，SIMONYAN K，ZISSERMAN A，et al.Spatial transformer networks[J].arXiv：1506.02025，2015.
[19] WANG F，JIANG M，QIAN C，et al.Residual attention network for image classification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：3156-3164.
[20] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[21] DENG J，DONG W，SOCHER R，et al.Imagenet：a large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition，2009：248-255.
[22] REDMON J，FARHADI A.YOLO9000：better，faster，stronger[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：7263-7271.
[23] MISRA D.Mish：a self regularized non-monotonic neural activation function[J].arXiv：1908.08681，2019.
[24] KALMAN R E.A new approach to linear filtering and prediction problems[J].Transactions of the ASME-Journal of Basic Engineering，1960，82（1）：35-45.
[25] KUHN H W.The hungarian method for the assignment problem[J].Naval Research Logistics Quarterly，1955，2（1/2）：83-97.
[26] ESS A，LEIBE B，SCHINDLER K，et al.A mobile vision system for robust multi-person tracking[C]//2008 IEEE Conference on Computer Vision and Pattern Recognition，2008：1-8.
[27] ZHANG S，BENENSON R，SCHIELE B.Citypersons：a diverse dataset for pedestrian detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：3213-3221.
[28] DOLLáR P，WOJEK C，SCHIELE B，et al.Pedestrian detection：a benchmark[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition，2009：304-311.
[29] XIAO T，LI S，WANG B，et al.Joint detection and identification feature learning for person search[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：3415-3424.
[30] ZHENG L，ZHANG H，SUN S，et al.Person re-identification in the wild[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：1367-1376.
[31] PANG B，LI Y，ZHANG Y，et al.TubeTK：adopting tubes to track multi-object in a one-step training model[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：6308-6318.
[32] FANG K，XIANG Y，LI X，et al.Recurrent autoregressive networks for online multi-object tracking[C]//2018 IEEE Winter Conference on Applications of Computer Vision（WACV），2018：466-475.
[33] WAN X，WANG J，KONG Z，et al.Multi-object tracking using online metric learning with long short-term memory[C]//2018 25th IEEE International Conference on Image Processing（ICIP），2018：788-792.
[34] CHOI W.Near-online multi-target tracking with aggregated local flow descriptor[C]//Proceedings of the IEEE International Conference on Computer Vision，2015：3029-3037.
[35] WOJKE N，BEWLEY A，PAULUS D.Simple online and realtime tracking with a deep association metric[C]//2017 IEEE International Conference on Image Processing（ICIP），2017：3645-3649.