Transformer Real-Time Target Tracking Algorithm Combining Multiple Attention Mechanisms

doi:10.3778/j.issn.1002-8331.2307-0343

Abstract

Abstract: In recent years, target tracking has great potential in academic research and practical application, and has received more and more attention and become a hot research direction of computer vision. Aiming at the problem that the existing target tracking algorithms have low tracking accuracy under complex background conditions of illumination change, occlusion and fast motion, this paper proposes a Transformer tracking algorithm combined with multi-attention mechanism-TrKYS. Firstly, the multi-attention module is introduced to capture target features in spatial and channel dimensions, and the modeling of context dependence for target feature is realized. Then, the feature map is sampled by multiple parallel hole convolution with different void rates to obtain multi-scale features of the image and enhance local feature expression ability. Finally, the Exemplar Transformer module is constructed, and the appearance tracking model is established by using templated spatial information and target features. In order to adapt to the appearance change, the tracker will update the tracking template feature vector, motion parameters and spatial information in real time when tracking the target object in consecutive frames, improving the accuracy of tracking and positioning. Experimental results on LaSOT, VOT2018, NFS, OTB-2015, TrackingNet and GOT-10k datasets show that in comparison with other mainstream target tracking algorithms, the proposed algorithm has better tracking performance, in particular, the tracking algorithm in this paper (TrKYS) on TrackingNet improves the success rate by 3 percentage points relative to the benchmark tracking algorithm (KYS).

Key words: Exemplar Transformer, target tracking, feature enhancement, cosmetic changes

摘要： 近年来目标跟踪在学术研究和实际应用中具有巨大潜力，受到越来越多的关注并成为计算机视觉的热点研究方向。针对现有的目标跟踪算法在光照变化、遮挡和快速运动复杂背景条件下跟踪精度较低这一问题，提出一种结合多注意力机制的Transformer跟踪算法——TrKYS。引入多注意力模块捕捉目标在空间和通道维度中的特征，实现对目标特征上下文依赖关系的建模；通过多个不同空洞率的平行空洞卷积对特征图进行采样，以获得图像的多尺度特征，增强局部特征表达能力；构建Exemplar Transformer模块，利用模板化的空间信息和目标特征建立了外观跟踪模型。为了适应外观变化，该跟踪算法在连续帧中跟踪目标物体时会实时更新跟踪模板特征向量、运动参数和空间信息，提高跟踪定位的精度。在LaSOT、VOT2018、NFS、OTB-2015、TrackingNet和GOT-10k数据集上实验结果表明，与其他主流目标跟踪算法相比，所提算法具有更好的跟踪性能，特别是在TrackingNet上的跟踪算法（TrKYS）相对于基准跟踪算法（KYS）成功率提高了3个百分点。

关键词: Exemplar Transformer, 目标跟踪, 特征增强, 外观变化

HUANG Yiqian, XU Yang, ZHANG Yongdan, XIAO Ci, FEMG Mingwen. Transformer Real-Time Target Tracking Algorithm Combining Multiple Attention Mechanisms[J]. Computer Engineering and Applications, 2024, 60(23): 187-197.

黄易仟, 徐杨, 张永丹, 肖慈, 冯明文. 结合多注意力机制的Transformer实时目标跟踪算法[J]. 计算机工程与应用, 2024, 60(23): 187-197.

References

[1] 江英杰, 宋晓宁. 基于视觉Transformer的双流目标跟踪算法[J]. 计算机工程与应用, 2022, 58(12): 183-190.
JIANG Y J, SONG X N. Dual-Stream object tracking algorithm based on vision transformer[J]. Computer Engineering and Applications, 2022, 58(12): 183-190.
[2] 厍向阳, 罗佳琪, 任海青, 等. 基于稀疏表示的相关滤波目标跟踪算法[J]. 计算机工程与应用, 2023, 59(11): 71-79.
SHE X Y, LUO J Q, REN H Q, et al. Correlation filter for object tracking method based on spare representation[J]. Computer Engineering and Applications, 2023, 59(11): 71-79.
[3] BHAT G, DANELLJAN M, GOOL L V, et al. Learning discriminative model prediction for tracking[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019: 6182-6191.
[4] HENRIQUES F J, RUI C, PEDRO M, et al. High-speed tracking with kernelized correlation filters[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3): 583-596.
[5] LUKEZIC A, VOJIR T, ZAJC L C, et al. Discriminative correlation filter with channel and spatial reliability[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 6309-6318.
[6] TOUIL E D, TERKI N, MEDOUAKH S. Hierarchical convolutional features for visual tracking via two combined color spaces with SVM classifier[J]. Signal, Image and Video Processing, 2019, 13(2): 359-368.
[7] WANG N, XIE Z, HUANG N. Improved context-aware correlation filter tracking[C]//Proceedings of the 2018 International Conference on Image and Video Processing, and Artificial Intelligence, 2018, 10836: 144-152.
[8] DANELLJAN M, ROBINSON A, SHAHBAZ K F, et al. Beyond correlation filters: learning continuous convolution operators for visual tracking[C]//Proceedings of the 14th European Conference on Computer Vision, 2016: 472-488.
[9] DANELLJAN M, BHAT G, SHAHBAZ K F, et al. Eco: efficient convolution operators for tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 6638-6646.
[10] YAN B, PENG H, FU J, et al. Learning spatio-temporal transformer for visual tracking[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 10448-10457.
[11] CHEN X, YAN B, ZHU J, et al. Transformer tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 8126-8135.
[12] CUI Y, JIANG C, WANG L, et al. End-to-end tracking with iterative mixed attention[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 18-24.
[13] 李昊轩. 基于深度学习的音频事件分类研究[D]. 北京: 北京邮电大学, 2020.
LI H X. Research on audio event classification based on deep learning[D]. Beijing: Beijing University of Posts and Telecommunications, 2020.
[14] HUANG L, ZHAO X, HUANG K. Got-10k: a large high diversity benchmark for generic object tracking in the wild[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 43(5): 1562-1577.
[15] FAN H, LIN L, YANG F, et al. LaSOT: a high-quality benchmark for large-scale single object tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 5374-5383.
[16] KRISTAN M, LEONARDIS A, MATAS J, et al. The sixth visual object tracking vot2018 challenge results[C]//Proceedings of the European Conference on Computer Vision Workshops, 2018: 3-53.
[17] WU Y, LIM J w, YANG M H. Object tracking benchmark[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1834-1848.
[18] KIANI G H，FAGG A，HUANG C，et al. Need for speed：a benchmark for higher frame rate object tracking[C]//Proceedings of the IEEE International Conference on Computer Vision, 2017: 1125-1134.
[19] SONG Y, MA C, GONG L, et al. CREST: convolutional residual learning for visual tracking[C]//Proceedings of the IEEE International Conference on Computer Vision, 2017: 2555-2564.
[20] LU X, MA C, NI B, et al. Deep regression tracking with shrinkage loss[C]//Proceedings of the European Conference on Computer Vision, 2018: 353-369.
[21] DANELLJAN M, BHAT G, KHAN F S, et al. ATOM: accurate tracking by overlap maximization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 4660-4669.
[22] BHAT G, DANELLJAN M, GOOL V L, et al. Know your surroundings: exploiting scene information for object tracking[C]//Proceedings of the 16th European Conference on Computer Vision, 2020: 205-221.
[23] CARION N, MASSA F, SYNNAEVE G, et al. End-to-end object detection with transformers[C]//Proceedings of the European Conference on Computer Vision, 2020: 213-229.
[24] REN S, HE K, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(6): 1137-1149.
[25] VOIGTLAENDER P, LUITEN J, TORR P H S, et al. Siam R-CNN: visual tracking by re-detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 6578-6588.
[26] WANG X, GIRSHICK R, GUPTA A, et al. Non-local neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7794-7803.
[27] FU J, LIU J, TIAN H, et al. Dual attention network for scene segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 3146-3154.
[28] HOWARD A G, ZHU M, CHEN B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications[J]. arXiv:1704.04861, 2017.
[29] IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]//Proceedings of the International Conference on Machine Learning, 2015: 448-456.
[30] BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional siamese networks for object tracking[C]//Proceedings of the European Conference on Computer Vision, 2016: 850-865.
[31] LI B, WU W, WANG Q, et al. Siamrpn++: evolution of siamese visual tracking with very deep networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 4282-4291.
[32] ZHENG L, TANG M, CHEN Y, et al. Learning feature embeddings for discriminant model based tracking[C]//Proceedings of the European Conference on Computer Vision, 2020: 759-775.
[33] DANELLJAN M, GOOL L V, TIMOFTE R. Probabilistic regression for visual tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 7183-7192.
[34] WANG G, LUO C, SUN X, et al. Tracking by instance detection: a meta-learning approach[C]//Proceedings of theIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 6288-6297.
[35] LUKEZIC A, MATAS J, KRISTAN M. D3S-A discriminative single shot segmentation tracker[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 7133-7142.
[36] WANG N, ZHOU W, WANG J, et al. Transformer meets tracker: exploiting temporal context for robust visual tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 1571-1580.