Object Tracking Algorithm Based on Siamese Network with Multiple Templates

doi:10.3778/j.issn.1002-8331.2009-0184

Abstract

Abstract: The object tracking algorithms based on fully-convolutional siamese network use single template in the tracking phase. It causes the tracking drift when the appearance of moving target changes, which decreases tracking accuracy. In view of this, an object tracking algorithm based on siamese network with multiple templates is proposed, which builds the template library at the feature level. Here, the average peak-to correlation energy and template similarity are used to ensure the effectiveness of each template in the template library. Multiple response maps are fused to improve the tracking accuracy. The experiment results on OTB2015 and VOT2016 datasets demonstrate that the proposed algorithm not only gets fast tracking speed, but also achieves better tracking performance in comparison with other mainstream algorithms in the complex environment where the appearance changes.

Key words: object tracking, siamese network, template update, translation equivariance, template library

摘要： 基于全卷积孪生网络的视频目标跟踪算法由于在跟踪过程中使用单一模板，在运动目标外观发生变化时容易出现跟踪漂移并导致精度下降。因此，提出了一种基于孪生网络融合多模板的目标跟踪算法。该算法可在特征级上建立模板库，并使用平均峰值相关能量和模板相似度来保证模板库中各个模板的有效性，从而对多个响应图进行融合以获得更高的跟踪精度。OTB2015和VOT2016数据集上的测试结果表明，在运动目标外观发生变化的复杂环境下，所提算法不但具有较快的跟踪速度，而且相比现有的其他算法能取得更为优异的跟踪性能。

关键词: 目标跟踪, 孪生网络, 模板更新, 平移等变性, 模板库

YANG Zhe, SUN Lifan, FU Zhumu, ZHANG Jinjin, CHANG Jiashun. Object Tracking Algorithm Based on Siamese Network with Multiple Templates[J]. Computer Engineering and Applications, 2022, 58(4): 212-220.

杨哲, 孙力帆, 付主木, 张金锦, 常家顺. 基于孪生网络融合多模板的目标跟踪算法[J]. 计算机工程与应用, 2022, 58(4): 212-220.

References

[1] 卢湖川，李佩霞，王栋.目标跟踪算法综述[J].模式识别与人工智能，2018，31（1）：61-76.
LU H C，LI P X，WANG D.Visual object tracking：a survey[J].Pattern Recognition and Artificial Intelligence，2018，31（1）：61-76.
[2] HENRIQUES J F，CASEIRO R，MARTINS P，et al.High-speed tracking with kernelized correlation filters[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2014，37（3）：583-596.
[3] DANELLJAN M，BHAT G，SHAHBAZ K F，et al.ECO：efficient convolution operators for tracking[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition，2017：6638-6646.
[4] DANELLJAN M，HAGER G，SHAHBAZ K F，et al.Convolutional features for correlation filter based visual tracking[C]//2015 IEEE International Conference on Computer Vision Workshops，2015：58-66.
[5] TAO R，GAVVES E，SMEULDERS A W M.Siamese instance search for tracking[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition，Las Vegas，2016：1420-1429.
[6] BERTINETTO L，VALMADRE J，HENRIQUES J F，et al.Fully-convolutional siamese networks for object tracking[C]//14th European Conference on Computer Vision.Cham：Springer，2016：850-865.
[7] LI B，YAN J，WU W，et al.High performance visual tracking with siamese region proposal network[C]//2018 IEEE Conference on Computer Vision and Pattern Recognition，2018：8971-8980.
[8] VALMADRE J，BERTINETTO L，HENRIQUES J，et al.End-to-end representation learning for correlation filter based tracking[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，2017：5000-5008.
[9] GUO Q，FENG W，ZHOU C，et al.Learning dynamic siamese network for visual object tracking[C]//2017 IEEE International Conference on Computer Vision，Venice，2017：1781-1789.
[10] GUO J，XU T F，JIANG S W，et al.Generating reliable online adaptive templates for visual tracking[C]//2018 25th IEEE International Conference on Image Processing，2018：226-230.
[11] WANG M M，LIU Y，HUANG Z Y.Large margin object tracking with circulant feature maps[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，2017：4800-4808.
[12] WU Y，LIM J，YANG M H.Object tracking benchmark[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，37（9）：1834-1848.
[13] KRISTAN M，LEONARDIS A，MATAS J，et al.The visual object tracking VOT2016 challenge results[C]//14th European Conference on Computer Vision.Cham：Springer，2016：777-823.
[14] 伊恩·古德费洛，约书亚·本吉奥，亚伦·库维尔.深度学习[M].赵申剑，黎或君，符天凡，等译.北京：人民邮电出版社，2017：286.
GOODFELLOW I，BENGIO Y，COURVILLE A.Deep learning[M].ZHAO S J，LI H J，FU T F，et al.Beijing：People’s Posts and Telecommunications Press，2017：286.
[15] KRIZHEVSKY A，SUTSKEVER I，HINTON G E.ImageNet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems，2012：1097-1105.
[16] DANELLJAN M，H?GER G，KHAN F S，et al.Learning spatially regularized correlation filters for visual tracking[C]//2015 IEEE International Conference on Computer Vision，Santiago，2015：4310-4318.
[17] BERTINETTO L，VALMADRE J，GOLODETZ S，et al.Staple：complementary learners for real-time tracking[C]//2016 IEEE conference on Computer Vision and Pattern Recognition，Las Vegas，2016：1401-1409.
[18] DANELLJAN M，H?GER G，KHAN F S，et al.Discriminative scale space tracking[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2016，39（8）：1561-1575.
[19] HARE S，SAFFARI A，TORR P H S.Struck：structured output tracking with kernels[C]//2011 IEEE International Conference on Computer Vision，Barcelona，2011.