Tracking Algorithm Combining Multi-Level Feature Fusion and Efficient Attention

doi:10.3778/j.issn.1002-8331.2204-0454

Abstract

Abstract: In complex scenes such as illumination change, size change, and motion blur, the existing target tracking algorithms are easy to fail. To solve these problems, some tracking algorithms try to introduce global attention to enhance feature expression for improving accuracy, but it needs to consume a lot of computing resources. This paper proposes an object tracking algorithm SiamEff（Siamese efficient network） that combines multi-level feature fusion and efficient attention, which enables global attention to improve accuracy while only using a small number of computing resources. Firstly, an anchor-free mechanism is employed to avoid the sensitivity of anchor-based tracking algorithms to a large number of hyperparameters. Then, an attention mechanism is introduced, which is more efficient than conventional global attention and greatly reduces memory and computational requirements, low-level and high-level multi-level features are fused, the feature information at different levels is used to improve tracking performance. Finally, centrality constraints are introduced to reduce the interference of low-quality prediction boxes. Tested on the public test platforms OTB100 and VOT2018, the results show that SiamEff tracking performance is better than mainstream tracking algorithms, more robust and accurate in various complex scenarios, and the tracking speed reaches 150 FPS.

Key words: object tracking, anchor-free, attention mechanism, center-ness, multi-level feature

摘要： 在光照变化、形状变化、运动模糊等复杂场景下，现有的目标跟踪算法很容易跟踪失败，为了解决这些问题，一些跟踪算法尝试引入全局注意力来增强特征表达以提高准确率，但这需要消耗大量的计算资源。提出一种结合多级特征融合和高效注意力的目标跟踪算法SiamEff（Siamese efficient network），使全局注意力能在耗费少量计算资源的前提下提高准确率。使用无锚框机制以避免基于锚框的跟踪算法对大量超参的敏感问题；引入一种比传统全局注意力更加高效的注意力机制，大大降低内存和运算量的需求，并融合了低层和高层的多级特征，充分利用不同层次的特征信息提高跟踪性能；引入了中心度约束，以减少低质量预测框的干扰。在公开测试平台OTB100和VOT2018上进行了测试，结果表明SiamEff的跟踪性能优于主流跟踪算法，在各种复杂场景下更加鲁棒和准确，且跟踪速度达到150?FPS。

关键词: 目标跟踪, 无锚框, 注意力机制, 中心度约束, 多级特征

YAO Zhuangze, ZENG Bi, LIN Zhentao, JIANG Chunling, DENG Bin. Tracking Algorithm Combining Multi-Level Feature Fusion and Efficient Attention[J]. Computer Engineering and Applications, 2023, 59(16): 177-186.

姚壮泽, 曾碧, 林镇涛, 江春灵, 邓斌. 结合多级特征融合和高效注意力的跟踪算法[J]. 计算机工程与应用, 2023, 59(16): 177-186.

References

[1] BERTINETTO L，VALMADRE J，HENRIQUES J F，et al.Fully-convolutional siamese networks for object tracking[C]//Proceedings of the 14th European Conference on Computer Vision.Cham：Springer，2016：850-865.
[2] LI B，YAN J，WU W，et al.High performance visual tracking with siamese region proposal network[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition，2018：8971-8980.
[3] REN S，HE K，GIRSHICK R，et al.Faster R-CNN：towards real-time object detection with region proposal networks[C]//Advances in Neural Information Processing Systems 28，2015.
[4] ZHU Z，WANG Q，LI B，et al.Distractor-aware siamese networks for visual object tracking[C]//Proceedings of the 15th European Conference on Computer Vision，2018：101-117.
[5] LI B，WU W，WANG Q，et al.SiamRPN++：evolution of siamese visual tracking with very deep networks[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：4282-4291.
[6] CHEN Z，ZHONG B，LI G，et al.Siamese box adaptive network for visual tracking[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：6668-6677.
[7] XU Y，WANG Z，LI Z，et al.SiamFC++：towards robust and accurate visual tracking with target estimation guidelines[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence，2020：12549-12556.
[8] ZHANG Z，PENG H，FU J，et al.Ocean：object-aware anchor-free tracking[C]//Proceedings of the 16th European Conference on Computer Vision.Cham：Springer，2020：771-787.
[9] 王玲，周磊，王鹏，等.基于注意力机制和孪生网络的跟踪算法研究[J].计算机工程与应用，2022，58（23）：161-168.
WANG L，ZHOU L，WANG P，et al.Research on tracking algorithm based on attention mechanism and siamese network[J].Computer Engineering and Applications，2022，58（23）：161-168.
[10] YU Y，XIONG Y，HUANG W，et al.Deformable siamese attention networks for visual object tracking[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：6728-6737.
[11] TAN H，ZHANG X，ZHANG Z，et al.Nocal-Siam：refining visual features and response with advanced non-local blocks for real-time siamese tracking[J].IEEE Transactions on Image Processing，2021，30：2656-2668.
[12] WU Y，LIM J，YANG M H.Object tracking benchmark[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，37（9）：1834-1848.
[13] KRISTAN M，LEONARDIS A，MATAS J，et al.The sixth visual object tracking VOT2018 challenge results[C]//Proceedings of the 15th European Conference on Computer Vision Workshops，2018.
[14] KRIZHEVSKY A，SUTSKEVER I，HINTON G E.Image-
Net classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems 25，2012.
[15] TIAN Z，SHEN C，CHEN H，et al.FCOS：fully convolutional one-stage object detection[C]//Proceedings of 2019 the IEEE/CVF International Conference on Computer Vision，2019：9627-9636.
[16] SHEN Z，ZHANG M，ZHAO H，et al.Efficient attention：attention with linear complexities[C]//Proceedings of the 2021 IEEE/CVF Winter Conference on Applications of Computer Vision，2021：3531-3539.
[17] WANG X，GIRSHICK R，GUPTA A，et al.Non-local neural networks[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition，2018：7794-7803.
[18] WOO S，PARK J，LEE J Y，et al.CBAM：convolutional block attention module[C]//Proceedings of the 15th European Conference on Computer Vision，2018：3-19.
[19] RUSSAKOVSKY O，DENG J，SU H，et al.ImageNet large scale visual recognition challenge[J].International Journal of Computer Vision，2015，115（3）：211-252.
[20] REAL E，SHLENS J，MAZZOCCHI S，et al.YouTube-boundingboxes：a large high-precision human-annotated data set for object detection in video[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition，2017：5296-5305.
[21] LIN T Y，MAIRE M，BELONGIE S，et al.Microsoft COCO：common objects in context[C]//Proceedings of the 13th European Conference on Computer Vision.Cham：Springer，2014：740-755.
[22] HUANG L，ZHAO X，HUANG K.GOT-10k：a large high-diversity benchmark for generic object tracking in the wild[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2019，43（5）：1562-1577.
[23] FAN H，LIN L，YANG F，et al.LaSOT：a high-quality benchmark for large-scale single object tracking[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：5374-5383.
[24] DANELLJAN M，HAGER G，SHAHBAZKHAN F，et al.Learning spatially regularized correlation filters for visual tracking[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision，2015：4310-4318.
[25] DANELLJAN M，HAGER G，KHAN F S，et al.Convolutional features for correlation filter based visual tracking[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision Workshops，2015：58-66.
[26] DANELLJAN M，H?GER G，KHAN F S，et al.Discriminative scale space tracking[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2016，39（8）：1561-1575.
[27] BERTINETTO L，VALMADRE J，GOLODETZ S，et al.Staple：complementary learners for real-time tracking[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition，2016：1401-1409.
[28] LI P，CHEN B，OUYANG W，et al.GradNet：gradient-guided network for visual object tracking[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision，2019：6162-6171.
[29] VALMADRE J，BERTINETTO L，HENRIQUES J，et al.End-to-end representation learning for correlation filter based tracking[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition，2017：2805-2813.
[30] ZHANG Z，PENG H.Deeper and wider siamese networks for real-time visual tracking[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：4591-4600.
[31] LI Y，ZHANG X.SiamVGG：visual tracking using deeper siamese networks[J].arXiv：1902.02804，2019.
[32] GUO Q，FENG W，ZHOU C，et al.Learning dynamic siamese network for visual object tracking[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision，2017：1763-1771.
[33] WANG Q，GAO J，XING J，et al.Discriminant correlation filters network for visual tracking[J].arXiv：1704.04057，2017.
[34] HU J，SHEN L，SUN G.Squeeze-and-excitation networks[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition，2018：7132-7141.
[35] WANG Q，WU B，ZHU P，et al.ECA-Net：efficient channel attention for deep convolutional neural networks[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition，Seattle，2020：11531-11539.
[36] 陈志浩，杨小军.基于孪生网络多级注意特征融合目标跟踪算法[J].计算机技术与发展，2021，31（11）：58-63.
CHEN Z H，YANG X J.Multistage attention feature fusion tracking algorithm based on siamese netwotk[J].Computer Technology and Development，2021，31（11）：58-63.
[37] 刘洋，杨小军.基于孪生网络特征融合与阈值更新的跟踪算法[J].计算机技术与发展，2022，32（3）：65-70.
LIU Y，YANG X J.Tracking algorithm based on twin networks feature fusion and threshold update[J].Computer Technology and Development，2022，32（3）：65-70.
[38] WANG P，FU H，LI X，et al.Multi-feature fusion tracking algorithm based on generative compression network[J].Future Generation Computer Systems，2021，124：206-214.