旋转区域提议网络的孪生神经网络跟踪算法

doi:10.3778/j.issn.1002-8331.2204-0182

摘要/Abstract

摘要： 孪生区域提议网络跟踪算法是一种高效的目标跟踪算法，通过锚框规避了图像金字塔对跟踪性能带来的影响，但这种跟踪方法受制于区域提议网络本身的局限性，在目标旋转时，跟踪精度将受到较大损失。而其他对旋转鲁棒性较高的方法则因为使用了复杂的旋转结构，导致算法的跟踪速度大幅下降。为了解决旋转目标对区域提议网络跟踪精度的影响，提出了旋转区域提议网络的孪生神经网络跟踪算法，通过AO-RPN（arbitrary-oriented region proposal network）结构将旋转与区域提议网络相统一，引入角度预测分支，在目标跟踪的过程中，直接对旋转的目标进行搜索，并得到最小外接矩形。该方法在保持较高跟踪速度的同时，精度超过了对目标进行旋转采样或使用局部特征进行跟踪的算法。通过在数据集OTB2015、VOT2016和VOT2018上进行的大量实验。结果表明，该算法在遮挡、形变、光照等多种复杂情况下表现出了较强的鲁棒性和适应性。

关键词: 目标跟踪, 特征融合, 旋转锚框, 区域提议网络, 孪生神经网络

Abstract: Siamese region proposal network tracing algorithm is an efficient target tracing algorithm, where anchor box is used to avoid the influence of image pyramid to tracing performance. However, this tracing method is subject to limitation of region proposal network itself. While the target is rotating, the tracing accuracy is greatly reduced. As to other algorithms which demonstrate high rotation robustness, the adoption of complex rotation structure results in substantial decline of tracing speed. In order to address the influence of rotating target to the region proposal network tracing accuracy, a siamese network tracing algorithm of rotating region proposal network is proposed. Through unification of rotation and region proposal network via arbitrary-oriented region proposal network（AO-RPN） and by introduction of angle prediction branch, the rotating target is searched directly during target tracing, obtaining the minimum bounding rectangle. This method maintains high tracing speed and achieves higher accuracy than an algorithm where target is subject to rotating sampling or traced using local characteristics. The experimental results of OTB2015, VOT2016 and VOT2018 data sets suggest that the proposed algorithm demonstrates strong robustness and adaptability to multiple complex conditions such as blocking, deformation and lighting.

Key words: object tracing, feature fusion, rotating anchor box, region proposal network, siamese network

姜文涛, 崔江磊. 旋转区域提议网络的孪生神经网络跟踪算法[J]. 计算机工程与应用, 2022, 58(24): 247-255.

JIANG Wentao, CUI Jianglei. Siamese Network Tracing Algorithm of Rotating Region Proposal Network[J]. Computer Engineering and Applications, 2022, 58(24): 247-255.

参考文献

[1] 朱向雷，王海弛，尤翰墨，等.自动驾驶智能系统测试研究综述[J].软件学报，2021，32（7）：2056-2077.
ZHU X L，WANG H C，YOU H M，et al.Survey on testing of intelligent systems in autonomous vehicles[J].Journal of Software，2021，32（7）：2056-2077.
[2] 李玺，查宇飞，张天柱，等.深度学习的目标跟踪算法综述[J].中国图象图形学报，2019，24（12）：2057-2080.
LI X，ZHA Y F，ZHANG T Z，et al.Survey of visual object tracking algorithms based on deep learning[J].Journal of Image and Graphics，2019，24（12）：2057-2080.
[3] 杨哲，孙力帆，付主木，等.基于孪生网络融合多模板的目标跟踪算法[J].计算机工程与应用，2022，58（4）：212-220.
YANG Z，SUN L F，FU Z M，et al.Object tracking algorithm based on siamese network with multiple templates[J].Computer Engineering and Applications，2022，58（4）：212-220.
[4] 李瀚，刘坤华，刘嘉杰，等.实时视觉目标跟踪与视频对象分割多任务框架[J].中国图象图形学报，2021，26（1）：101-112.
LI H，LIU K H，LIU J J，et al.Multitask framework for video object tracking and segmentation combined with multi-scale interframe information[J].Journal of Image and Graphics，2021，26（1）：101-112.
[5] 张焕龙，胡士强，杨国胜，基于外观模型学习的视频目标跟踪方法综述[J].计算机研究与发展，2015，52（1）：177-190.
ZHANG H L，HU S Q，YANG G S.Video object tracking based on appearance models learning[J].Computer Research and Development，2015，52（1）：177-190.
[6] 李东洁，李东阁，杨柳.卷积神经网络在动态手势跟踪中的应用[J].计算机科学与探索，2020，14（5）：841-847.
LI D J，LI D G，YANG L.Application of convolutional neural network in dynamic gesture tracking[J].Frontiers of Computer Science and Technology，2020，14（5）：841-847.
[7] 武随烁，杨金福，单义，等.使用孪生注意力机制的生成对抗网络的研究[J].计算机科学与探索，2020，14（5）：833-840.
WU S S，YANG J F，SHAN Y，et al.Research on generative adversarial networks using twins attention mechanism[J].Journal of Frontiers of Computer Science and Technology，2020，14（5）：833-840.
[8] 柳培忠，汪鸿翔，骆炎民，等.一种结合时空上下文的在线卷积网络跟踪算法[J].计算机研究与发展，2018，55（12）：2785-2793.
LIU P Z，WANG H X，LUO Y M，et al.Online convolutional network tracking via spatio-temporal context[J].Computer Research and Development，2018，55（12）：2785-2793.
[9] 谭建豪，张思远.基于自适应空间正则化的视觉目标跟踪算法[J].计算机研究与发展，2021，58（2）：427-435.
TAN J H，ZHANG S Y.Visual tracking algorithm based on adaptive spatial regularization[J].Journal of Computer Research and Development，2021，58（2）：427-435.
[10] 韩明，王景芹，王敬涛，等.级联特征融合孪生网络目标跟踪算法研究[J].计算机工程与应用，2022，58（6）：208-218.
HAN M，WANG J Q，WANG J T，et al.Research on object tracking algorithm based on cascading feature fusion of siamese network[J].Computer Engineering and Applications，2022，58（6）：208-218.
[11] BERTINETTO L，VALMADRE J，HENRIQUES J F，et al.Fully convolutional siamese networks for object tracking[C]//European Conference on Computer Vision，2016：850-865.
[12] REN S，HE K，GIRSHICK R，et al.Faster R-CNN：towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，39（6）：1137-1149.
[13] LI B，YAN J，WU W，et al.High performance visual tracking with siamese region proposal network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：8971-8980.
[14] LI B，WU W，WANG Q，et al.Siamrpn++：evolution of siamese visual tracking with very deep networks[C]//IEEE Conference on Computer Vision and Pattern Recognition，2018：4282-4291.
[15] ZHANG L，GONZALEZ-GARCIA A，WEIJER J，et al.Learning the model update for siamese trackers[C]//IEEE International Conference on Computer Vision，2019：4009-4018.
[16] WANG Q，ZHANG L，BERTINETTO L，et al.Fast online object tracking and segmentation：a unifying approach[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：1328-1338.
[17] WANG Z Y，LI Z X，YU G，et al.SiamFC++：towards robust and accurate visual tracking with target estimation guidelines[C]//IEEE Conference on Computer Vision and Pattern Recognition，2017：6931-6939.
[18] GUPTA D K，ARYA D，GAVVES E.Rotation equivariant siamese networks for tracking[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：12357-12366.
[19] QIN R，LIU Q，GAO G，et al.MRDet：a multihead network for accurate rotated object detection in aerial images[J].Advances in Neural Information Processing Systems，2012，1（2）：4310-4318.
[20] ZHU Z，WANG Q，LI B，et al.Distractor-aware siamese networks for visual object tracking[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：101-117.
[21] GUO D，WANG J，CUI Y，et al.SiamCAR：siamese fully convolutional classification and regression for visual tracking[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），2020：8972-8982.
[22] SOSNOVIK I，MOSKALEV A，SMEULDERS A W M.Scale equivariance improves siamese tracking[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision，2021：2765-2774.