Target Tracking Based on Conditional Confrontation Network and Hierarchical Feature Fusion

doi:10.3778/j.issn.1002-8331.2106-0470

Abstract

Abstract: In order to solve the problem of poor tracking effect caused by motion blur and low resolution in the process of target tracking, this paper proposes target tracking algorithm based on conditional confrontation network and hierarchical feature fusion. Firstly, the input low-resolution video frames are deblurred by using the conditional confrontation generation network model（DeblurGAN-v2）. Then, VGG-19 network is used to extract Conv2, Conv4 and Conv6 features of the target candidate region, and the low-level structural features, middle-level features and high-level semantic features extracted by siamese network are fused to improve the characterization ability of features. The experimental results on the target tracking evaluation datasets OTB2015 and VOT2018 show that compared with other algorithms such as SiamFC and SiamDW, the proposed algorithm has higher accuracy, and can adapt to complex situations such as motion blur, appearance change and background interference. Compared with SiamFC, the improved algorithm improves the success rate by 5.5?percentage points on OTB2015 datasets and EAO by 16.4?percentage points on VOT2018 datasets.

Key words: target tracking, conditional confrontation network, siamese network, feature fusion

摘要： 为了解决目标跟踪过程中因运动模糊和低分辨率导致跟踪效果变差的问题，提出一种基于条件对抗网和层次特征融合的目标跟踪算法。使用条件对抗生成网络模型（DeblurGAN-v2），对输入的低分辨率视频帧去模糊；使用改进型VGG-19网络提取目标候选区域的Conv2、Conv4、Conv6三层特征，将孪生网络提取到的低层结构特征、中层特征与高层语义特征进行融合，以提高特征的表征能力。在目标跟踪评估数据集OTB2015与VOT2018上的实验结果表明，与SiamFC、SiamDW等其他算法相比，该算法具有更高的准确性，能够适应目标遮挡运动模糊、外观变化及背景干扰等复杂情况。相比于SiamFC，改进算法在OTB2015数据集上成功率提升5.5个百分点，在VOT2018数据集上EAO提升16.4个百分点。

关键词: 目标跟踪, 条件对抗网络, 孪生网络, 特征融合

ZHANG Lei, SHAN Yugang, YUAN Jie. Target Tracking Based on Conditional Confrontation Network and Hierarchical Feature Fusion[J]. Computer Engineering and Applications, 2022, 58(23): 221-229.

张磊, 单玉刚, 袁杰. 基于条件对抗网和层次特征融合的目标跟踪[J]. 计算机工程与应用, 2022, 58(23): 221-229.

References

[1] 单玉刚，胡卫国.尺度方向自适应视觉目标跟踪方法综述[J].计算机工程与应用，2020，56（9）：13-23.
SHAN Y G，HU W G.Review of visual object tracking algorithms of adaptive direction and scale[J].Computer Engineering and Applications，2020，56（9）：13-23.
[2] DANELLJAN M，BHAT G，SHAHBAZ K F，et al.ECO：efficient convolution operators for tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：6638-6646.
[3] DANELLJAN M，HAGER G，SHAHBAZ K F，et al.Convolutional features for correlation filter based visual tracking[C]//Proceedings of the IEEE International Conference on Computer Vision Workshops，2015：58-66.
[4] WANG N，YUNG D Y.Learning a deep compact image representation for visual tracking[C]//Advances in Neural Information Processing Systems，2013：809-817.
[5] BERTINETTO L，VALMADRE J，HENRIQUES J F，et al.Fully-convolutional siamese networks for object tracking[C]//European Conference on Computer Vision，2016：850-865.
[6] LI B，YAN J，WU W，et al.High performance visual tracking with siamese region proposal network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：8971-8980.
[7] WANG Q，ZHANG L，BERTINETTO L，et al.Fast online object tracking and segmentation：a unifying approach[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2019：1328-1338.
[8] HE A，LUO C，TIAN X，et al.A twofold siamese network for real-time object tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：4834-4843.
[9] LI B，WU W，WANG Q，et al.Evolution of siamese visual tracking with very deep networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2019：16-20.
[10] ZHANG Z，PENG H.Deeper and wider siamese networks for real-time visual tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2019：4591-4600.
[11] CHEN X，YAN B，ZHU J，et al.Transformer tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2021.
[12] KRIZHECSKY A，SUTSKEVER I，HINTON G E.Image-Net classification with deep convolutional neural networks[C]//Advances in Neural Information Processing System，2012：1097-1105.
[13] KUPYN O，MARTYNIUK T，WU J，et al.DeblurGAN-v2：deblurrng faster and better[C]//International Conference on Computer Vision，2019：8878-8887.
[14] SIMONYAN K，ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition，2014.
[15] GOODFELLOW I，POUGET A J，MIRZA M，et al.Generative adversarial nets[C]//Advances in Neural Information Processing Systems，2014：2672-2680.
[16] 陈富健，谢维信.引入抗遮挡机制的SiamVGG网络目标跟踪算法[J].信号处理，2020，36（4）：562-571.
CHEN F J，XIE W X.SiamVGG network target tracking algorithm with anti-occlusion mechanism[J].Journal of Signal Processing，2020，36（4）：562-571.
[17] STERGION A，KALLIATAKIS G.Refining activation down sampling with SoftPool[EB/OL].[2021-02-02].https：//arxiv.org/abs/2101.040440.
[18] 张晶，黄浩淼.结合重检测机制的多卷积层特征响应跟踪算法[J].计算机科学与探索，2021，15（3）：533-544.
ZHANG J，HUANG H M.Multi-convolutional layer feature response tracking algorithm combined with re-detection mechanism[J].Journal of Frontiers of Computer Science and Technology，2021，15（3）：533-544.
[19] 李睿，连继荣.改进的Siamese自适应网络和多特征融合跟踪算法[J].计算机科学与探索，2022，16（11）：2587-2595.
LI R，LIAN J R.Improved Siamese adaptive network and multi-feature fusion tracking algorithm[J].Journal of Frontiers of Computer Science and Technology，2022，16（11）：2587-2595.
[20] WU Y，LI J，YANG M H.Objectr tracking benchmark[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，37（9）：1834-1848.
[21] KRISTAN M，LEONARDIS A，MATAS J，et al.The sixth visual object tracking VOT2018 challenge results[C]//European Conference on Computer Vision，2018：3-53.
[22] DANELLJIAN M，HAGER G，KHAN F S，et al.Learning spatially regularized correlation filters for visual tracking[C]//Proceedings of the IEEE International Conference on Computer Vision，2015：4310-4318.
[23] DANELLJIAN M，HAGER G，SHAHBAZ K F，et al.Convolutional features for correlation filter based visual tracking[C]//Proceeding of the IEEE International Conference on Computer Vision Workshops，2015：621-629.
[24] DANELLJIAN M，HAGER G，KHAN F S，et al.Discrimi-native scale space tracking[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，39（8）：1561-1575.
[25] VALMADRE J，BERTINETTO L，HENRIQUES J，et al.End-to-end representation learning for correlation filter based tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：2805-2813.
[26] BERTINETTO L，CALMADRE J，GOLODETZ，et al.Staple：complementary learners for real-time tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：1401-1409.
[27] HENRIQUES J F，CASEIRO R，MARTINS P，et al.High-speed tracking with kernelized correlation filters[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2014，37（3）：583-596.