融合像素互相关的Transformer跟踪算法

doi:10.3778/j.issn.1002-8331.2403-0298

摘要/Abstract

摘要： Siamese网络互相关操作的局部匹配性无法有效获得全局上下文信息，而Transformer网络依赖全局关系获得语义信息，但需要更多的局部边缘信息来区分目标和背景。因此，提出了一种结合像素互相关（pixel-wise cross-correlation，PW-Corr）和Transformer的目标跟踪算法。构建并行编码器并采用非线性重加权注意力（non-linear reweighting attention，NRA）提高Transformer获取全局上下文的能力；设计解码器并融合像素互相关从空间和通道两方面的交互提高特征融合的精确度，过滤多余背景干扰。分类回归任务使用一个基于多层感知器（multi-layer perceptron，MLP）的分类头和具有全局上下文感知模块（global context awareness module，GCAM）的回归头，捕捉全局信息同时提取目标局部信息，促进算法对跟踪目标的准确定位。实验结果表明，改进后的算法在OTB100数据集上成功率和准确率分别可达70.6%、92.1%，提高了跟踪的成功率和准确率。

关键词: Transformer网络, 像素互相关, 注意力机制, 全局上下文感知

Abstract: The local matching of the cross-correlation operation of the Siamese network cannot effectively obtain global context information, while the Transformer network relies on global relationships to obtain semantic information, but requires more local edge information to distinguish the target and the background. Therefore, a target tracking algorithm combining pixel-wise cross-correlation (PW-Corr) and Transformer is proposed. This paper constructs a parallel encoder and uses non-linear reweighting attention (NRA) to improve the ability of Transformer to obtain global context. A decoder and integrate pixel-wise cross-correlation are designed from the interaction between space and channel to improve the accuracy of feature fusion and filters out redundant background interference. The classification and regression task uses a classification head based on multi-layer perceptron (MLP) and a regression head with global context awareness module (GCAM) to capture global information while extracting local information of the target, promoting the algorithm to accurate positioning of tracking targets. Experimental results show that the success rate and accuracy rate of the improved algorithm on the OTB100 dataset are 70.6% and 92.1% respectively, which improves the tracking success rate and accuracy.

Key words: Transformer network, pixel-wise cross-correlation, attention mechanism, global context awareness

薛紫涵, 葛海波, 杨雨迪, 田攀帅. 融合像素互相关的Transformer跟踪算法[J]. 计算机工程与应用, 2025, 61(12): 279-290.

XUE Zihan, GE Haibo, YANG Yudi, TIAN Panshuai. Transformer Tracking Algorithm Integrating Pixel-Wise Cross-Correlation[J]. Computer Engineering and Applications, 2025, 61(12): 279-290.

参考文献

[1] 刘艺, 李蒙蒙, 郑奇斌, 等. 视频目标跟踪算法综述[J]. 计算机科学与探索, 2022, 16(7): 1504-1515.
LIU Y, LI M M, ZHENG Q B, et al. Survey on video object tracking algorithms[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1504-1515.
[2] LI X, ZHA Y F, ZHANG T Z, et al. Survey of visual object tracking algorithms based on deep learning[J].Journal of Image and Graphics, 2019, 24(12): 2057-2080.
[3] QIU S M, GU Y Z, CHEN M H, et al. A dynamic adjust-head siamese network for object tracking[J]. IET Computer Vision, 2023, 17(2): 203-210.
[4] BOLME D S, BEVERIDGE J R, DRAPER B A, et al. Visual object tracking using adaptive correlation filters[C]//Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2010: 2544-2550.
[5] 陈云芳, 吴懿, 张伟. 基于孪生网络结构的目标跟踪算法综述[J]. 计算机工程与应用, 2020, 56(6): 10-18.
CHEN Y F, WU Y, ZHANG W. Survey of target tracking algorithm based on Siamese network structure[J]. Computer Engineering and Applications, 2020, 56(6): 10-18.
[6] ZHANG L, VARADARAJAN J, SUGANTHAN P N, et al. Robust visual tracking using oblique random forests[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 5825-5834.
[7] BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional Siamese networks for object tracking[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer International Publishing, 2016: 850-865.
[8] LI Y J, CAI J T, ZHOU Q, et al. Joint semantic-instance segmentation method for intelligent transportation system[J]. IEEE Transactions on Intelligent Transportation Systems, 2023, 24(12): 15540-15547.
[9] LI B, WU W, WANG Q, et al. SiamRPN++: evolution of Siamese visual tracking with very deep networks[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 4277-4286.
[10] YAN B, ZHANG X Y, WANG D, et al. Alpha-refine: boosting tracking performance by precise bounding box estimation[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 5285-5294.
[11] CHEN Z D, ZHONG B N, LI G R, et al. Siamese box adaptive network for visual tracking[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 6667-6676.
[12] XU Y D, WANG Z Y, LI Z X, et al. SiamFC++: towards robust and accurate visual tracking with target estimation guidelines[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2020: 12549-12556.
[13] LI B, YAN J J, WU W, et al. High performance visual tracking with Siamese Region proposal network[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 8971-8980.
[14] GUO D Y, SHAO Y Y, CUI Y, et al. Graph attention tracking[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 9538-9547.
[15] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770-778.
[16] GUO D Y, WANG J, CUI Y, et al. SiamCAR: Siamese fully convolutional classification and regression for visual tracking[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 6268-6276.
[17] WANG N, ZHOU W G, WANG J, et al. Transformer meets tracker: exploiting temporal context for robust visual tracking[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 1571-1580.
[18] CARION N, MASSA F, SYNNAEVE G, et al. End-to-end object detection with transformers[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer International Publishing, 2020: 213-229.
[19] CHEN X, YAN B, ZHU J, et al. Transformer tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 8126-8135.
[20] YAN B, PENG H W, FU J L, et al. Learning spatio-temporal transformer for visual tracking[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2021: 10428-10437.
[21] GUO J Y, HAN K, WU H, et al. CMT: convolutional neural networks meet vision transformers[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 12165-12175.
[22] PENG Z L, HUANG W, GU S Z, et al. Conformer: local features coupling global representations for visual recognition[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2021: 357-366.
[23] ZHANG J M, SUN J, WANG J, et al. An object tracking framework with recapture based on correlation filters and Siamese networks[J]. Computers & Electrical Engineering, 2022, 98: 107730.
[24] ZHANG Z P, PENG H W, FU J L, et al. Ocean: object-aware anchor-free tracking[C]//Proceedings of the 16th European Conference on Computer Vision. Cham: Springer International Publishing, 2020: 771-787.
[25] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems, 2017.
[26] YANG S L, ZHOU D M, CAO J D, et al. LightingNet: an integrated learning method for low-light image enhancement[J]. IEEE Transactions on Computational Imaging, 2023, 9: 29-42.
[27] SONG Z K, YU J Q, CHEN Y P, et al. Transformer tracking with cyclic shifting window attention[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 8781-8790.
[28] FU Zhihong, FU Zehua, LIU Q J, et al. SparseTT: visual tracking with sparse transformers[J]. arXiv:2205.03776, 2022.
[29] CUI Y T, JIANG C, WANG L M, et al. MixFormer: end-to-end tracking with iterative mixed attention[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 13598-13608.
[30] WU Y, CHEN Y P, YUAN L, et al. Rethinking classification and localization for object detection[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 10183-10192.
[31] HUANG L H, ZHAO X, HUANG K Q. GOT-10k: a large high-diversity benchmark for generic object tracking in the wild[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(5): 1562-1577.
[32] LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft COCO: common objects in context[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer, 2014: 740-755.
[33] RUSSAKOVSKY O, DENG J, SU H, et al. ImageNet large scale visual recognition challenge[J]. International Journal of Computer Vision, 2015, 115(3): 211-252.
[34] GLOROTl X, BENGIO Y.Understanding the difficulty of training deep feedforward neural networks[C]//Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, 2010: 249-256.
[35] LOSHCHILOV I, HUTTER F. Decoupled weight decay regularization[J]. arXiv:1711.05101, 2017.