Siamese Network Tracking with Short-Term Memory

doi:10.3778/j.issn.1002-8331.2009-0121

Abstract

Abstract: In order to improve the robustness of the target tracking algorithm in complex scenes and adapt the algorithm to long-term tracking scenarios, a tracking method combining multi-layer feature fusion and short-term memory mechanism is proposed to improve the robustness of target tracking. Firstly, the multi-layer features of the convolutional neural network are fused to improve the feature extraction capability of network. Then, in the tracking stage, a short-term memory module is introduced, and the features of the search area are matched with the initial template features and the dynamic features of the short-term memory, and then the response map is fused to improve the robustness of target tracking. The discriminability of the tracking target is enhanced through local information of video. Experiments are carried out on the OTB2015 and GOT-10K target tracking standard data sets. The accuracy and success rate of OTB2015 reach 0.808 and 0.593 respectively. Experimental results show that test effect of the proposed algorithm is significantly improved incomparison with several mainstream tracking algorithms, and the real-time tracking speed reaches 27 frame/s, which proves the effectiveness of the proposed method.

Key words: siamese network, long-term tracking, feature fusion, template update

摘要： 为提升目标跟踪算法在复杂场景下的鲁棒性，使算法适应长时跟踪场景，提出结合多层特征融合和短时记忆机制的跟踪方法，提升目标跟踪的鲁棒性。融合卷积神经网络多层特征，提升网络的特征提取能力。在跟踪阶段，引入了短时记忆模块，搜索区域特征分别与初始的基准模板特征和短时记忆的动态特征进行匹配，对得到的响应图进行融合，提升目标跟踪的鲁棒性，通过视频局部信息增强算法对跟踪目标的判别性。在OTB2015和GOT-10K目标跟踪标准数据集上进行了实验，在OTB2015上的精确度和成功率分别达到了0.808和0.593。实验结果表明，所提算法的测试效果与几种主流跟踪算法相比有了显著的提升，并且达到了27 帧/s的实时跟踪速度，证明了所提方法的有效性。

关键词: 孪生网络, 长时跟踪, 特征融合, 模板更新

WANG Xipeng, LI Yong, LI Zhi, LIANG Qiming. Siamese Network Tracking with Short-Term Memory[J]. Computer Engineering and Applications, 2022, 58(3): 235-241.

王希鹏, 李永, 李智, 梁起明. 引入短时记忆的Siamese网络目标跟踪算法[J]. 计算机工程与应用, 2022, 58(3): 235-241.

References

[1] HENRIQUES J F，CASEIRO R，MARTINS P，et al.High-speed tracking with kernelized correlation filters[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2014，37（3）：583-596.
[2] HENRIQUES J F，CASEIRO R，MARTINS P，et al.Exploiting the circulant structure of tracking-by-detection with kernels[C]//European Conference on Computer Vision，2012：702-715.
[3] KRIZHEVSKY A，SUTSKEVER I，HINTON G E.Imagenet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems，2012：1097-1105.
[4] TAO R，GAVVES E，SMEULDERS A W.Siamese instance search for tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：1420-1429.
[5] BERTINETTO L，VALMADRE J，HENRIQUES J F，et al.Fully-convolutional siamese networks for object tracking[C]//European Conference on Computer Vision，2016：850-865.
[6] VALMADRE J，BERTINETTO L，HENRIQUES J，et al.End-to-end representation learning for correlation filter based tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：2805-2813.
[7] GUO Q，FENG W，ZHOU C，et al.Learning dynamic siamese network for visual object tracking[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：1763-1771.
[8] WANG X，LI C，LUO B，et al.Sint++：Robust visual tracking via adversarial positive instance generation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：4864-4873.
[9] HE A，LUO C，TIAN X，et al.A twofold siamese network for real-time object tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：4834-4843.
[10] WANG Q，TENG Z，XING J，et al.Learning attentions：residual attentional siamese network for high performance online visual tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：4854-4863.
[11] FAN H，LIN L，YANG F，et al.Lasot：a high-quality benchmark for large-scale single object tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2019：5374-5383.
[12] MULLER M，BIBI A，GIANCOLA S，et al.Trackingnet：a large-scale dataset and benchmark for object tracking in the wild[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：300-317.
[13] WANG M，LIU Y，HUANG Z.Large margin object tracking with circulant feature maps[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：4021-4029.
[14] 沈玉玲，伍忠东，赵汝进，等.基于模型更新与快速重检测的长时目标跟踪[J].光学学报，2020，40（3）：115-124.
SHEN Y L，WU Z D，ZHAO R J，et al.Long-term object tracking based on model updating and fast re-detection[J].Acta Optica Sinica，2020，40（3）：115-124.
[15] 梁浩，刘克俭，刘康，等.引入再检测机制的孪生神经网络目标跟踪[J].光学精密工程，2019，27（7）：1621-1631.
LIANG H，LIU K J，LIU K，et al.Siamese network tracking with redetection mechanism[J].Optics and Precision Engineering，2019，27（7）：1621-1631.
[16] WANG N，ZHOU W，LI H.Reliable re-detection for long-term tracking[J].IEEE Transactions on Circuits and Systems for Video Technology，2019，29（3）：730-743.
[17] ZHANG W，KANG B，ZHANG S.Spatial-temporal aware long-term object tracking[J].IEEE Access，2020，8：71662-71684.
[18] 仇祝令，查宇飞，朱鹏，等.基于孪生神经网络在线判别特征的视觉跟踪算法[J].光学学报，2019，39（9）：253-261.
QIU Z L，ZHA Y F，ZHU P，et al.Visual tracking algorithm based on online feature discrimination with siamese network[J].Acta Optica Sinica，2019，39（9）：253-261.
[19] 苗军，李凯，许少武.基于卷积神经网络多层特征融合的目标跟踪[J].现代电子技术，2018，41（24）：114-116.
MIAO J，LI K，XU S W.Target tracking based on multi-layer feature fusion of convolutional neural network[J].Modern Electronics Technique，2018，41（24）：114-116.
[20] HUANG G，LIU Z，VAN DER MAATEN L，et al.Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：4700-4708.
[21] HU J，SHEN L，SUN G.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：7132-7141.
[22] CHEN Y，CAO Y，HU H，et al.Memory enhanced global-local aggregation for video object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：10337-10346.
[23] HUANG L，ZHAO X，HUANG K.Got-10k：a large high-diversity benchmark for generic object tracking in the wild[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2019，43（5）：1562-1577.
[24] WU Y，LIM J，YANG M H.Object tracking benchmark[J].IEEE Transactions on Pattern Analysis Machihe Intelligence，2015，37（9）：1834-1848.
[25] DANELLJAN M，HAGER G，SHAHBAZ KHAN F，et al.Learning spatially regularized correlation filters for visual tracking[C]//Proceedings of the IEEE International Conference on Computer Vision，2015：4310-4318.
[26] BERTINETTO L，VALMADRE J，GOLODETZ S，et al.Staple：complementary learners for real-time tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：1401-1409.
[27] DANELLJAN M，H?GER G，KHAN F，et al.Accurate scale estimation for robust visual tracking[C]//British Machine Vision Conference，Nottingham，September 1-5，2014.