关键词: 多目标跟踪, Transformer, 特征融合

Abstract: In convolutional neural network (CNN), convolution can efficiently extract local features of the object, but it is difficult to capture global representation; in the visual Transformer, the attention mechanism can capture long-distance feature dependency, but will ignore local feature details. To solve the above problems, a multi-object tracking algorithm CTMOT (CNN transformer multi-object tracking) based on CNN-Transformer hybrid network for feature extraction and fusion is proposed. Firstly, the backbone network is adopted based on CNN and Transformer to extract the local and global features of the image respectively. Secondly, two way bridge module (TBM) is used to fully integrate two features. Then, the fused features are input to two parallel decoders for processing. Finally, the detection box and the tracking box outputted by the decoder are matched to obtain final tracking result and complete the multi-target tracking task. Evaluated on MOT17, MOT20, KITTI and UA-DETRAC multi-object tracking datasets, the MOTA indicators of CTMOT algorithm have reached 76.4%, 66.3%, 92.36% and 88.57% respectively. It is equivalent to the SOTA method on the MOT dataset, and achieves the SOTA effect on the KITTI dataset. At the same time, the MOTP and IDs indicators have reached the SOTA effect on all datasets. In addition, since the object detection and correlation are completed at the same time, the object tracking can be carried out end-to-end, and the tracking speed can reach 35 FPS, which shows that CTMOT algorithm achieves a good balance in the real-time and accuracy of tracking, and has great potential.

Key words: multi-object tracking, Transformer, feature fusion