多尺度特征响应融合的复杂场景红外目标跟踪

doi:10.3778/j.issn.1002-8331.2402-0076

摘要/Abstract

摘要： 针对红外目标跟踪算法在复杂场景下性能退化的问题，提出了多尺度特征响应融合的红外目标跟踪算法。该算法基于Siamese网络框架，构建了一个自适应模版更新的目标模板库，以提高模版匹配准确性。采用ResNet-50构建了多尺度特征提取与融合网络，通过多分支结构捕获不同信息的特征，并引入分组卷积以及增加每组的基数来提取多样化的深层特征。通过自适应权值分配策略，将不同尺度的特征进行融合；提出了全局感知与快速响应模块，通过可微相关滤波器层实现对整帧图像的全局感知，同时动态生成适应性滤波器以捕捉目标特征。采用核估计概率直方图建立红外目标的多尺度特征模型，在每个前向传播步骤中与候选区域进行比较，提高算法对于目标变化的响应速度和感知能力；提出了空间-通道-帧间交互自注意力模块，使模型能够更好地聚焦于全局空间特征和高响应通道，并利用帧间交互注意力增强了前、后帧信息的互补性。在LSOTB-TIR和PTB-TIR数据集上进行了实验。实验结果表明，在多种复杂场景下，提出的算法显著增强了目标辨别、感知和抗干扰能力。算法的成功率和精准率，在LSOTB-TIR数据集上分别达到了67.3%和80.0%，在PTB-TIR数据集上分别达到了64.5%和83.1%，综合优于对比跟踪算法。

关键词: 红外目标跟踪, 计算机视觉, 孪生网络, 相关滤波

Abstract: To address the issue of performance degradation of infrared target tracking algorithms in complex scenes, a multi-scale feature response fusion infrared target tracking algorithm is proposed. This algorithm is based on the siamese network framework, constructing an adaptive template updating target template library to improve template matching accuracy. A ResNet-50 is utilized to build a multi-scale feature extraction and fusion network, capturing diverse features through a multi-branch structure, and introducing grouped convolutions with increased cardinality per group to extract diversified deep features. Additionally, through an adaptive weight allocation strategy, features of different scales are fused. Moreover, a global perception and rapid response module is proposed, achieving global perception of the entire frame image through differentiable correlation filter layers. Meanwhile, dynamically generating adaptive filters to capture target features. Furthermore, a multi-scale feature model of infrared targets is established using kernel estimation probability histograms. This model is compared with candidate regions at each forward propagation step to enhance the algorithm’s responsiveness and perception to target changes. Finally, a spatial-channel-inter-frame interaction self-attention module is proposed, allowing the model to better focus on global spatial features and highly responsive channels. Additionally, utilizing frame interaction attention to enhance the complementary nature of information between preceding and subsequent frames. Experiments are conducted on the LSOTB-TIR and PTB-TIR datasets. Experimental results demonstrate that proposed algorithm significantly enhances target discrimination, perception, and anti-interference capabilities across various complex scenarios. The success and precision rate of the algorithm reaches 67.3% and 80.0% respectively on the LSOTB-TIR dataset, and 64.5% and 83.1% respectively on the PTB-TIR dataset, outperforming the compared tracking algorithms comprehensively.

Key words: infrared target tracking, computer vision, siamese network, correlation filter

熊偌炎, 张上, 张岳. 多尺度特征响应融合的复杂场景红外目标跟踪[J]. 计算机工程与应用, 2025, 61(12): 243-257.

XIONG Ruoyan, ZHANG Shang, ZHANG Yue. Multi-Scale Feature Response Fusion for Complex Scene Infrared Target Tracking[J]. Computer Engineering and Applications, 2025, 61(12): 243-257.

参考文献

[1] OSEP A, MEHNER W, MATHIAS M, et al. Combined image-and world-space tracking in traffic scenes[C]//Proceedings of the 2017 IEEE International Conference on Robotics and Automation. Piscataway: IEEE, 2017: 1988-1995.
[2] DING M, CHEN W H, CAO Y F. Thermal infrared single-pedestrian tracking for advanced driver assistance system[J]. IEEE Transactions on Intelligent Vehicles, 2023, 8(1): 814-824.
[3] FANG H Z, DING L, WANG L M, et al. Infrared small UAV target detection based on depthwise separable residual dense network and multiscale feature fusion[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 5019120.
[4] GADE R, MOESLUND T B. Thermal cameras and applications: a survey[J]. Machine Vision and Applications, 2014, 25(1): 245-262.
[5] DANELLJAN M, BHAT G, KHAN F S, et al. ECO: efficient convolution operators for tracking[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 6931-6939.
[6] ZHANG L C, GONZALEZ-GARCIA A, VAN DE WEIJER J, et al. Synthetic data generation for end-to-end thermal infrared tracking[J]. IEEE Transactions on Image Processing, 2019, 28(4): 1837-1850.
[7] GAO P, MA Y P, SONG K, et al. Large margin structured convolution operator for thermal infrared object tracking[C]//Proceedings of the 2018 24th International Conference on Pattern Recognition. Piscataway: IEEE, 2018: 2380-2385.
[8] ZHANG H, YIN Z Y, ZHANG H L. Thermal infrared object tracking using correlation filters improved by level set[J]. Signal, Image and Video Processing, 2023, 17(3): 791-797.
[9] BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional Siamese networks for object tracking[C]//Proceedings of the 2018 25th IEEE International Conference on Image Processing. Cham: Springer International Publishing, 2016: 850-865.
[10] DONG X P, SHEN J B. Triplet loss in Siamese network for object tracking[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer, 2018: 472-488.
[11] LI B, YAN J J, WU W, et al. High performance visual tracking with Siamese region proposal network[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 8971-8980.
[12] LI B, WU W, WANG Q, et al. SiamRPN++: evolution of Siamese visual tracking with very deep networks[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 4277-4286.
[13] WANG Q, ZHANG L, BERTINETTO L, et al. Fast online object tracking and segmentation: a unifying approach[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 1328-1338.
[14] CHEN R M, LIU S J, MIAO Z, et al. GFSNet: generalization-friendly Siamese network for thermal infrared object tracking[J]. Infrared Physics & Technology, 2022, 123: 104190.
[15] YAO T T, HU J C, ZHANG B, et al. Scale and appearance variation enhanced Siamese network for thermal infrared target tracking[J]. Infrared Physics & Technology, 2021, 117: 103825.
[16] HUANG Y P, HE Y J, LU R T, et al. Thermal infrared object tracking via unsupervised deep correlation filters[J]. Digital Signal Processing, 2022, 123: 103432.
[17] YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[J]. arXiv:1511.07122, 2015.
[18] 成浩维, 资文杰, 彭双, 等. 基于半监督学习的三维Mesh建筑物立面提取与语义分割方法[J]. 郑州大学学报 (理学版), 2023, 55(4): 8-15.
CHENG H W, ZI W J, PENG S, et al. Semi-supervised learning based 3D mesh building facade extraction and semantic segmentation method[J]. Journal of Zhengzhou University (Natural Science Edition), 2023, 55(4): 8-15.
[19] 王浩桐, 郭中华. 锚框策略匹配的SSD飞机遥感图像目标检测[J]. 计算机科学与探索, 2022, 16(11): 2596-2608.
WANG H T, GUO Z H. Target detection of SSD aircraft remote sensing images based on anchor frame strategy matching[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(11): 2596-2608.
[20] 云飞, 殷雁君, 张文轩, 等. 融合注意力机制的对抗式半监督语义分割[J]. 计算机工程与应用, 2023, 59(8): 254-262.
YUN F, YIN Y J, ZHANG W X, et al. Adversarial semi-supervised semantic segmentation with attention mechanism[J]. Computer Engineering and Applications, 2023, 59(8): 254-262.
[21] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
[22] XIE S N, GIRSHICK R, DOLLáR P, et al. Aggregated residual transformations for deep neural networks[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 5987-5995.
[23] XIAO B, XU B C, BI X L, et al. Global-feature encoding U-Net (GEU-Net) for multi-focus image fusion[J]. IEEE Transactions on Image Processing, 2021, 30: 163-175.
[24] DING X Y, LIN W S, CHEN Z Z, et al. Point cloud saliency detection by local and global feature fusion[J]. IEEE Transactions on Image Processing, 2019, 28(11): 5379-5393.
[25] LI C L, CHENG H, HU S Y, et al. Learning collaborative sparse representation for grayscale-thermal tracking[J]. IEEE Transactions on Image Processing, 2016, 25(12): 5743-5756.
[26] ZHANG W H, JIAO L C, LI Y X, et al. Sparse learning-based correlation filter for robust tracking[J]. IEEE Transactions on Image Processing, 2021, 30: 878-891.
[27] PENG C, ZHANG X Y, YU G, et al. Large kernel matters: improve semantic segmentation by global convolutional network[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 1743-1751.
[28] FU J, LIU J, TIAN H J, et al. Dual attention network for scene segmentation[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 3141-3149.
[29] 王燕, 吕艳萍. 混合深度CNN联合注意力的高光谱图像分类[J]. 计算机科学与探索, 2023, 17(2): 385-395.
WANG Y, LYU Y P. Hybrid deep CNN-attention for hyperspectral image classification[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(2): 385-395.
[30] 梁义涛, 韩永波, 李磊. 深度长时目标跟踪算法综述[J]. 计算机工程与应用, 2023, 59(4): 1-17.
LIANG Y T, HAN Y B, LI L. Survey on deep-learning-based long-term object tracking algorithms[J]. Computer Engineering and Applications, 2023, 59(4): 1-17.
[31] ZHANG L C, GONZALEZ-GARCIA A, VAN DE WEIJER J, et al. Learning the model update for Siamese trackers[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2019: 4009-4018.
[32] LIU Q, LI X, HE Z Y, et al. LSOTB-TIR: a large-scale high-diversity thermal infrared object tracking benchmark[C]//Proceedings of the 28th ACM International Conference on Multimedia. New York: ACM, 2020: 3847-3856.
[33] LIU Q, HE Z Y, LI X, et al. PTB-TIR: a thermal infrared pedestrian tracking benchmark[J]. IEEE Transactions on Multimedia, 2020, 22(3): 666-675.
[34] DANELLJAN M, H?GER G, KHAN F S, et al. Learning spatially regularized correlation filters for visual tracking[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2015: 4310-4318.
[35] DANELLJAN M, H?GER G, KHAN F, et al. Accurate scale estimation for robust visual tracking [C]//Proceedings of the British Machine Vision Conference. Nottingham: Bmva Press, 2014.
[36] LI F, TIAN C, ZUO W, et al. Learning spatial-temporal regularized correlation filters for visual tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 4904-4913.
[37] NAM H, HAN B. Learning multi-domain convolutional neural networks for visual tracking [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 4293-4302.
[38] DANELLJAN M, BHAT G, KHAN F S, et al. ATOM: accurate tracking by overlap maximization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 4660-4669.
[39] SONG Y, MA C, WU X, et al. Vital: visual tracking via adversarial learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 8990-8999.
[40] BHAT G, DANELLJAN M, GOOL L V, et al. Learning discriminative model prediction for tracking[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019: 6182-6191.