Double Adjust Head Siamese Network for Object Tracking

doi:10.3778/j.issn.1002-8331.2105-0377

Abstract

Abstract:

Siamese network based trackers formulate tracking as a similarity matching problem between a target template and a search region. Virtually all popular Siamese trackers use cross-correlation to measure the similarity between the feature of template and search image. The emphasis for feature extraction in different regions（inside and contours） are the same. Besides, the global matching also seriously neglects the part-level information and the deformation of targets during tracking. In this paper, a simple but effective Double Adjust Head Siamese Network is proposed to extract features from an object inside and object contours respectively. A Pixelwise Cross-correlation model（PWC） is designed to solve the problem caused by the fixed template structure in conventional correlation operations. Compared with baseline algorithm, the AO, SR0.5, SR0.75 of the proposed algorithm on GOT10k dataset are increased by 3.4%, 7.0% and 2.3%. Running at over 90 frames per second on RTX 2080Ti GPU.

Key words: visual tracking, siamese network, double adjust-head, pixelwise cross-correlation

摘要：

基于孪生网络的目标跟踪算法将跟踪问题建模为目标特征和搜索区特征之间的匹配问题。匹配程度通常是根据二者特征之间的相关响应来衡量。目前该衡量方式仍存在以下局限：一方面，对目标的不同区域使用的是相同的特征提取器，没有考虑到目标内部和轮廓处的区别；另一方面，在特征之间相关性的求解过程中，模板空间结构是固定的，无法很好地应对目标形变时的情况，鲁棒性较差。为解决上述问题，提出了一种双分支孪生网络目标跟踪算法SiamDAH（Double Adjust Head Siamese Network for Object Tracking），其中双分支结构旨在考虑目标内部区域和轮廓处的表征需求差异。此外，提出了一种改进的逐像素相关模块，有效降低了传统相关操作时模板结构固定带来的问题。在GOT-10k数据集上的实验结果表明，提出的算法在AO、SR0.5、SR0.75指标上较基准算法分别实现了3.4%、7.0%、2.3%的提升。在NVIDIA RTX 2080Ti上速度可达90 frame/s。

关键词: 目标跟踪, 孪生网络, 双分支, 逐像素相关

QIU Shoumeng, GU Yuzhang, YUAN Zeqiang. Double Adjust Head Siamese Network for Object Tracking[J]. Computer Engineering and Applications, 2021, 57(24): 135-143.

邱守猛，谷宇章，袁泽强. 基于双分支孪生网络的目标跟踪[J]. 计算机工程与应用, 2021, 57(24): 135-143.

References

[1] HENRIQUES J F，CASEIRO R，MARTINS P，et al.High-speed tracking with kernelized correlation filters[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2014，37（3）：583-596.
[2] DANELLJAN M，HAGER G，SHAHBAZ KHAN F，et al.Learning spatially regularized correlation filters for visual tracking[C]//Proceedings of the IEEE International Conference on Computer Vision，2015：4310-4318.
[3] 李彪，孙瑾，李星达，等.自适应特征融合的相关滤波跟踪算法[J/OL].计算机工程与应用：1-14[2021-06-02].http：//kns.cnki.net/kcms/detail/11.2127.TP.20210326.1645.006.html.
LI B，SUN J，LI X D，et al.Correlation filter target tracking based on adaptive multi-feature fusion[J].Computer Engineering and Applications：1-14[2021-06-02].http：//kns.cnki.net/kcms/detail/11.2127.TP.20210326.1645.006.html.
[4] DANELLJAN M，ROBINSON A，KHAN F S，et al.Beyond correlation filters：learning continuous convolution operators for visual tracking[C]//European Conference on Computer Vision，2016：472-488.
[5] 张晶，黄浩淼.结合重检测机制的多卷积层特征响应跟踪算法[J].计算机科学与探索，2021，15（3）：533-544.
ZHANG J，HUANG H M.Multi-convolutional layer feature response tracking algorithm combined with re-detection mechanism[J].Journal of Frontiers of Computer Science and Technology，2021，15（3）：533-544.
[6] BERTINETTO L，VALMADRE J，HENRIQUES J，et al.Fully-convolutional siamese networks for object tracking[C]//European Conference on Computer Vision，2016：850-865.
[7] 陈云芳，吴懿，张伟.基于孪生网络结构的目标跟踪算法综述[J].计算机工程与应用，2020，56（6）：10-18.
CHEN Y F，WU Y，ZHANG W.Survey of target tracking algorithm based on siamese network structure[J].Computer Engineering and Applications，2020，56（6）：10-18.
[8] 戴凤智，魏宝昌，欧阳育星，等.基于深度学习的视频跟踪研究进展综述[J].计算机工程与应用，2019，55（10）：16-29.
DAI F Z，WEI B C，OUYANG Y X，et al.Survey of research progress of video tracking based on deep learning[J].Computer Engineering and Applications，2019，55（10）：16-29.
[9] LI B，YAN J，WU W，et al.High performance visual tracking with siamese region proposal network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：8971-8980.
[10] LI B，WU W，WANG Q，et al.Evolution of siamese visual tracking with very deep networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2019：16-20.
[11] 陈富健，谢维信.引入抗遮挡机制的SiamVGG网络目标跟踪算法[J].信号处理，2020，248（4）：92-101.
CHEN F J，XIE W X.SiamVGG network target tracking algorithm with anti-occlusion mechanism[J].Journal of Signal Processing，2020，248（4）：92-101.
[12] XU Y，WANG Z，LI Z，et al.Siamfc++：towards robust and accurate visual tracking with target estimation guidelines[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2020：12549-12556.
[13] GUO D，WANG J，CUI Y，et al.SiamCAR：siamese fully convolutional classification and regression for visual tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：6269-6277.
[14] DU F，LIU P，ZHAO W，et al.Correlation-guided attention for corner detection based visual tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：6836-6845.
[15] LUO W，LI Y，URTASUN R，et al.Understanding the effective receptive field in deep convolutional neural networks[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems，2016：4905-4913.
[16] WU Y，LIM J，YANG M H.Online object tracking：a benchmark[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2013：2411-2418.
[17] HUANG L，ZHAO X，HUANG K.Got-10k：a large high-diversity benchmark for generic object tracking in the wild[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2021，43（5）：1562-1577.
[18] SZEGEDY C，LIU W，JIA Y，et al.Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2015：1-9.
[19] MULLER M，BIBI A，GIANCOLA S，et al.Trackingnet：a large-scale dataset and benchmark for object tracking in the wild[C]//Proceedings of the European Conference on Computer Vision（ECCV），2016：300-317.
[20] DANELLJAN M，BHAT G，SHAHBAZ KHAN F，et al.Eco：efficient convolution operators for tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：6638-6646.
[21] ZHU Z，WANG Q，LI B，et al.Distractor-aware siamese networks for visual object tracking[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：101-117.
[22] DANELLJAN M，BHAT G，KHAN F S，et al.Atom：Accurate tracking by overlap maximization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：4660-4669.
[23] BHAT G，DANELLJAN M，GOOL L，et al.Learning discriminative model prediction for tracking[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：6182-6191.
[24] ZHANG Z，PENG H，FU J，et al.Ocean：object-aware anchor-free tracking[J].arXiv：2006.10721，2020.
[25] NAM H，HAN B.Learning multi-domain convolutional neural networks for visual tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：4293-4302.
[26] LUKEZIC A，MATAS J，KRISTAN M.D3S-A discriminative single shot segmentation tracker[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：7133-7142.