Siamese Neural Network Target Tracking Algorithm with Lightweight Attention

doi:10.3778/j.issn.1002-8331.2110-0200

Abstract

Abstract: Aiming at the problem that it is difficult to balance the accuracy and real-time performance of target tracking tasks across various scenarios, a novel siamese neural network target tracking algorithm, namely, SiamNL is proposed. The proposed SiamNL algorithm leverages cross-correlation operations based on depth-wise convolution, and thereby reducing the amount of network parameters and operations, and improving the real-time performance of the algorithm within the unified deep learning framework. Additionally, SiamNL borrows the merits of the non-local attention module to encode template graph features and search graph features, therefore self-attention and cross-attention operations are performed on the features, so as to effectively improve the accuracy of the algorithm. The experimental results demonstrate that the proposed SiamNL lifts the performance of the state-of-the-art approaches with a large margin crossing three popular datasets, e.g., VOT2016, VOT2018, and OTB100, while the solid experimental discussion verifies the contribution of the proposed SiamNL in balancing the accuracy and real-time performance.

Key words: target tracking, siamese neural network, lightweight attention, real-time

摘要： 针对目标跟踪算法在各种场景下很难做到准确率和实时性平衡的问题，提出了一种引入轻量注意力的孪生神经网络（siamese neural network）目标跟踪算法，称为SiamNL。SiamNL算法使用基于深度级卷积（depth-wise convolution）的交叉相关运算，降低了网络的参数量和运算量，提升了算法的实时性。同时，SiamNL使用Non-Local注意力网络编码模板图特征和搜索图特征，对特征进行了自注意力和互注意力的运算，有效提升了算法的准确率。在VOT2016、VOT2018、OTB100等公开数据集上的测试结果表明，SiamNL算法优于主流的目标跟踪算法，更有效地平衡了准确率和实时性。

关键词: 目标跟踪, 孪生神经网络, 轻量注意力, 实时性

HONG Peiqin, LUO Lingkun, LIU Bing, FANG Yuan, HU Shiqiang. Siamese Neural Network Target Tracking Algorithm with Lightweight Attention[J]. Computer Engineering and Applications, 2022, 58(12): 112-121.

洪培钦, 罗灵鲲, 刘冰, 方元, 胡士强. 引入轻量注意力的孪生神经网络目标跟踪算法[J]. 计算机工程与应用, 2022, 58(12): 112-121.

References

[1] BOUGUET J Y.Pyramidal implementation of the affine lucaskanade feature tracker description of the algorithm[J].Intel Corporation，2001，5（1/10）：4.
[2] FUKUNAGA K，HOSTETLER L.The estimation of the gradient of a density function，with applications in pattern recognition[J].IEEE Transactions on Information Theory，1975，21（1）：32-40.
[3] BOLME D S，BEVERIDGE J R，DRAPER B A，et al.Visual object tracking using adaptive correlation filters[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition，2010：2544-2550.
[4] HENRIQUES J F，CASEIRO R，MARTINS P，et al.Exploiting the circulant structure of tracking-by-detection with kernels[C]//European Conference on Computer Vision.Berlin，Heidelberg：Springer，2012：702-715.
[5] HENRIQUES J F，CASEIRO R，MARTINS P，et al.High-speed tracking with kernelized correlation filters[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2014，37（3）：583-596.
[6] DANELLJAN M，H?GER G，KHAN F，et al.Accurate scale estimation for robust visual tracking[C]//British Machine Vision Conference，Nottingham，September 1-5，2014.[S.l.]：Bmva Press，2014.
[7] QIN C，JIN Y，TAO J，et al.DTCNNMI：a deep twin convolutional neural networks with multi-domain inputs for strongly noisy diesel engine misfire detection[J].Measurement，2021，180：109548.
[8] FATIMA N，AREEB Q M，KHAN I M，et al.Siamese network-based computer vision approach to detect papaya seed adulteration in black peppercorns[J].Journal of Food Processing and Preservation，2021.
[9] CHEN J，YUAN Z，PENG J，et al.DASNet：dual attentive fully convolutional siamese networks for change detection of high resolution satellite images[J].arXiv：2003.03608，2020.
[10] TAO R，GAVVES E，SMEULDERS A W M.Siamese instance search for tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：1420-1429.
[11] BERTINETTO L，VALMADRE J，HENRIQUES J F，et al.Fully-convolutional siamese networks for object tracking[C]//European Conference on Computer Vision.Cham：Springer，2016：850-865.
[12] GUO Q，FENG W，ZHOU C，et al.Learning dynamic siamese network for visual object tracking[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：1763-1771.
[13] WANG Q，TENG Z，XING J，et al.Learning attentions：residual attentional siamese network for high performance online visual tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：4854-4863.
[14] HE A，LUO C，TIAN X，et al.A twofold siamese network for real-time object tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：4834-4843.
[15] LI B，YAN J，WU W，et al.High performance visual tracking with siamese region proposal network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：8971-8980.
[16] WANG Q，ZHANG L，BERTINETTO L，et al.Fast online object tracking and segmentation：a unifying approach[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：1328-1338.
[17] LI B，WU W，WANG Q，et al.Siamrpn++：evolution of siamese visual tracking with very deep networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：4282-4291.
[18] YU Y，XIONG Y，HUANG W，et al.Deformable siamese attention networks for visual object tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：6728-6737.
[19] WANG N，ZHOU W，WANG J，et al.Transformer meets tracker：exploiting temporal context for robust visual tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：1571-1580.
[20] CHEN X，YAN B，ZHU J，et al.Transformer tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：8126-8135.
[21] 王玲，周磊，王鹏，等.基于注意力机制和孪生网络的跟踪算法研究[J/OL].计算机工程与应用：1-11[2021-11-13].http：//kns.cnki.net/kcms/detail/11.2127.TP.20210905.1414.002.html.
WANG L，ZHOU L，WANG P，et al.Research on tracking algorithm based on attention mechanism and twin network[J/OL].Computer Engineering and Applications：1-11[2021-11-13].http：//kns.cnki.net/kcms/detail/11.2127.TP.20210905.1414.002.html.
[22] FIAZ M，MAHMOOD A，JUNG S K.Learning soft mask based feature fusion with channel and spatial attention for robust visual object tracking[J].Sensors，2020，20（14）：4021.
[23] REN S，HE K，GIRSHICK R，et al.Faster R-CNN：towards real-time object detection with region proposal networks[J].Advances in Neural Information Processing Systems，2015，28：91-99.
[24] WANG X，GIRSHICK R，GUPTA A，et al.Non-local neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：7794-7803.
[25] RUSSAKOVSKY O，DENG J，SU H，et al.Imagenet large scale visual recognition challenge[J].International Journal of Computer Vision，2015，115（3）：211-252.
[26] LIN T Y，MAIRE M，BELONGIE S，et al.Microsoft coco：common objects in context[C]//European Conference on Computer Vision.Cham：Springer，2014：740-755.
[27] LEBEDA K，HADFIELD S J，BOWDEN R.The thermal infrared visual object tracking VOT-TIR2016 challenge result[C]//Proceedings European Conference on Computer Vision（ECCV），2016.
[28] KRISTAN M，LEONARDIS A，MATAS J，et al.The sixth visual object tracking vot2018 challenge results[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018.
[29] WU Y，LIM J，YANG M H.Object tracking benchmark[J].IEEE Transactions on Pattern Analysis & Machine Intelligence，2015，37（9）：1834-1848.
[30] ZHU P，WEN L，BIAN X，et al.Vision meets drones：a challenge[J].arXiv：1804.07437，2018.
[31] BERTINETTO L，VALMADRE J，GOLODETZ S，et al.Staple：complementary learners for real-time tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：1401-1409.
[32] DANELLJAN M，BHAT G，SHAHBAZ KHAN F，et al.ECO：efficient convolution operators for tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：6638-6646.
[33] DANELLJAN M，ROBINSON A，KHAN F S，et al.Beyond correlation filters：learning continuous convolution operators for visual tracking[C]//European Conference on Computer Vision，2016：472-488.
[34] NAM H，HAN B.Learning multi-domain convolutional neural networks for visual tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：4293-4302.
[35] FAN H，LING H.Siamese cascaded region proposal networks for real-time visual tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：7952-7961.
[36] HUANG B，XU T，JIANG S，et al.Robust visual tracking via constrained multi-kernel correlation filters[J].IEEE Transactions on Multimedia，2020，22（11）：2820-2832.
[37] LI P，CHEN B，OUYANG W，et al.GradNet：gradient-guided network for visual object tracking[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：6162-6171.
[38] ABDELPAKEY M H，SHEHATA M S，MOHAMED M M.Denssiam：end-to-end densely-siamese network with self-attention model for object tracking[C]//International Symposium on Visual Computing.Cham：Springer，2018：463-473.
[39] GUO Q，FENG W，ZHOU C，et al.Learning dynamic siamese network for visual object tracking[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：1763-1771.
[40] VALMADRE J，BERTINETTO L，HENRIQUES J，et al.End-to-end representation learning for correlation filter based tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：2805-2813.
[41] ZHANG Y，WANG L，QI J，et al.Structured siamese network for real-time visual tracking[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：351-366.