抗遮挡目标跟踪的模型学习综述

doi:10.3778/j.issn.1002-8331.2103-0178

摘要/Abstract

摘要： 视觉目标跟踪任务中的遮挡问题是最具挑战的场景属性之一，研究有效的抗遮挡模型学习方案，对构建适应复杂场景的长期鲁棒跟踪模型具有重要意义。剖析了遮挡影响跟踪性能的本质原因，以抗遮挡性能较好的先进跟踪算法为研究对象，系统分析了模型学习中有效抗遮挡机制，并对其改善长短期遮挡问题的有效性进行比较分析，包括以硬负样本挖掘、有效样本管理、类遮挡硬正样本生成的训练样本提质策略，提供模型充足判别信息；以时间一致性学习、自适应外观学习的被动稳定学习方式和基于多域属性、目标感知、干扰感知、特征融合等适用跟踪任务的主动学习策略，构建可抵抗场景干扰、目标形变等因素可适用跟踪的鲁棒模型；以手工置信度评估、自适应决策、时序记忆库、自适应估计模板的更新策略，平衡模型在线跟踪状态变化目标的适应性与稳定性。通过对代表跟踪算法在遮挡及背景杂乱、出视野、平面内外旋转、形变场景下的性能比较，详尽分析了各策略抗遮挡有效性，指出相比更新策略，数据处理、学习策略设计更有利于提高抗遮挡性能；同时分析了各策略对长期遮挡、背景杂乱、出视野等属性的适用性及适用多类复杂场景的策略。总结了有效抗遮挡策略，提出骨干网替换及迁移场景理解、运动规律等先验信息至跟踪任务的研究方向。

关键词: 目标跟踪, 遮挡, 高质训练样本集, 时间一致性, 置信度

Abstract: The occlusion problem in the visual object tracking task is one of the most challenging scene attributes. The study of effective anti-occlusion model learning schemes is of great significance for building long-term robust tracking models that adapt to complex scenes. First, it analyzes essentially why occlusion affects the tracking performance, and then advanced tracking algorithms with better anti-occlusion performance are taken as the research object, it systematically analyzes the effective anti-occlusion mechanism in model learning, and compares its effectiveness in improving long-term and short-term occlusion problems. The analysis includes training sample quality improvement strategies to provide sufficient discriminative information for the model that contains hard negative sample mining, effective sample management, and occlusion of hard positive samples generation; Passive and stable learning methods based on time consistency learning, adaptive appearance learning, and multiple active learning strategies suitable for tracking tasks builds robust models that can resist scene interference, target deformation and other factors, such as domain attributes, target perception, interference perception, feature fusion, etc.; The update strategy balances the adaptability and stability of the model tracking object with change state online such as manual confidence evaluation, adaptive decision-making, and time series memory storage, adaptive estimation template. Then, by comparing the performance of the representative tracking algorithm in occlusion and background clutter, out of view, in and out of the plane rotation, and deformation, the effectiveness of each strategy against occlusion is analyzed in detail, and it is pointed out that compared to the update strategy, the data processing and learning strategy design is more helpful to improve the anti-occlusion performance; Meanwhile, the applicability of each strategy to long-term occlusion, background clutter, out-of-view, and other attributes is analyzed, as well as the strategies applicable to multiple types of complex scenes. Finally, the effective anti-occlusion strategies are summarized, and research direction of backbone network replacement and migration scene understanding, motion law, and other prior information to the tracking task is proposed.

Key words: object tracking, occlusion, high-quality training sample set, time consistency, confidence

谢郭蓉, 曲毅, 蒋镕圻. 抗遮挡目标跟踪的模型学习综述[J]. 计算机工程与应用, 2022, 58(2): 43-56.

XIE Guorong, QU Yi, JIANG Rongqi. Survey of Model Learning for Anti-occlusion Object Tracking[J]. Computer Engineering and Applications, 2022, 58(2): 43-56.

参考文献

[1] 李玺，查宇飞，张天柱，等.深度学习的目标跟踪算法综述[J].中国图象图形学报，2019，24（12）：2057-2080.
LI X，CHA Y F，ZHANG T Z，et al.Survey of visual object tracking algorithms based on deep learning[J].Journal of Image and Graphics，2019，24（12）：2057-2080.
[2] SONG Y，MA C，WU X，et al.Vital：visual tracking via adversarial learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：8990-8999.
[3] LU X，MA C，NI B，et al.Deep regression tracking with shrinkage loss[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：353-369.
[4] BHAT G，DANELLJAN M，GOOL L V，et al.Learning discriminative model prediction for tracking[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：6182-6191.
[5] ZHU G，PORIKLI F，LI H.Robust visual tracking with deep convolutional neural network based object proposals on pets[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops，2016：26-33.
[6] NAM H，HAN B.Learning multi-domain convolutional neural networks for visual tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：4293-4302.
[7] FAN H，LING H.Sanet：structure-aware network for visual tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops，2017：42-49.
[8] JUNG I，SON J，BAEK M，et al.Real-time MDNet[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：83-98.
[9] KIANI GALOOGAHI H，FAGG A，LUCEY S.Learning background-aware correlation filters for visual tracking[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：1135-1143.
[10] ZHU Z，WANG Q，LI B，et al.Distractor-aware Siamese networks for visual object tracking[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：101-117.
[11] KIANI GALOOGAHI H，SIM T，LUCEY S.Correlation filters with limited boundaries[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2015：4630-4638.
[12] BHAT G，JOHNANDER J，DANELLJAN M，et al.Unveiling the power of deep tracking[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：483-498.
[13] ZHANG J，MA S，SCLAROFF S.MEEM：robust tracking via multiple experts using entropy minimization[C]//European Conference on Computer Vision.Cham：Springer，2014：188-203.
[14] BOLME D S，BEVERIDGE J R，DRAPER B A，et al.Visual object tracking using adaptive correlation filters[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition，2010：2544-2550.
[15] SUPANCIC J S，RAMANAN D.Self-paced learning for long-term tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2013：2379-2386.
[16] DANELLJAN M，HAGER G，SHAHBAZ KHAN F，et al.Adaptive decontamination of the training set：a unified formulation for discriminative visual tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：1430-1438.
[17] CHOI J，CHANG H J，FISCHER T，et al.Context-aware deep feature compression for high-speed visual tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：479-488.
[18] DANELLJAN M，BHAT G，SHAHBAZ KHAN F，et al.Eco：efficient convolution operators for tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：6638-6646.
[19] WANG X，LI C，LUO B，et al.Sint++：robust visual tracking via adversarial positive instance generation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：4864-4873.
[20] HAN Y，ZHANG P，HUANG W，et al.Robust visual tracking based on adversarial unlabeled instance generation with label smoothing loss regularization[J].Pattern Recognition，2020，97：107027.
[21] ZHANG X，WANG M，WEI J.Robust visual tracking based on adversarial fusion networks[C]//2018 37th Chinese Control Conference（CCC），2018：9142-9147.
[22] LI F，TIAN C，ZUO W，et al.Learning spatial-temporal regularized correlation filters for visual tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：4904-4913.
[23] YUAN D，SHU X，HE Z.TRBACF：learning temporal regularized correlation filters for high performance online visual object tracking[J].Journal of Visual Communication and Image Representation，2020，72：102882.
[24] FAN J，SONG H，ZHANG K，et al.Complementary tracking via dual color clustering and spatio-temporal regularized correlation learning[J].IEEE Access，2018，6：56526-56538.
[25] XU T，FENG Z H，WU X J，et al.Learning adaptive discriminative correlation filters via temporal consistency preserving spatial feature selection for robust visual object tracking[J].IEEE Transactions on Image Processing，2019，28（11）：5596-5609.
[26] ZHANG Y，LIU G，ZHANG H，et al.Robust visual tracker combining temporal consistent constraint and adaptive spatial regularization[J].Neural Computing and Applications，2021：1-20.
[27] DAI K，WANG D，LU H，et al.Visual tracking via adaptive spatially-regularized correlation filters[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：4670-4679.
[28] NAM H，BAEK M，HAN B.Modeling and propagating cnns in a tree structure for visual tracking[J].arXiv：1608.07242，2016.
[29] WANG N，ZHOU W，TIAN Q，et al.Multi-cue correlation filters for robust visual tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：4844-4853.
[30] YANG T，CHAN A B.Learning dynamic memory networks for object tracking[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：152-167.
[31] DANELLJAN M，HAGER G，SHAHBAZ KHAN F，et al.Learning spatially regularized correlation filters for visual tracking[C]//Proceedings of the IEEE International Conference on Computer Vision，2015：4310-4318.
[32] BERTINETTO L，VALMADRE J，GOLODETZ S，et al.Staple：complementary learners for real-time tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：1401-1409.
[33] LI B，WU W，WANG Q，et al.SiamRPN++：evolution of Siamese visual tracking with very deep networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：4282-4291.
[34] BAI S，HE Z，DONG Y，et al.Multi-hierarchical independent correlation filters for visual tracking[C]//2020 IEEE International Conference on Multimedia and Expo（ICME），2020：1-6.
[35] CHOI J，KWON J，LEE K M.Deep meta learning for real-time target-aware visual tracking[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：911-920.
[36] QI Y，ZHANG S，ZHANG W，et al.Learning attribute-specific representations for visual tracking[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2019，33（1）：8835-8842.
[37] LI X，MA C，WU B，et al.Target-aware deep tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：1369-1378.
[38] MA C，YANG X，ZHANG C，et al.Long-term correlation tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2015：5388-5396.
[39] WANG M，LIU Y，HUANG Z.Large margin object tracking with circulant feature maps[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：4021-4029.
[40] LIU T，WANG G，YANG Q.Real-time part-based visual tracking via adaptive correlation filters[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2015：4902-4912.
[41] LUKE?I? A，ZAJC L ?，VOJí? T，et al.Fucolot—a fully-correlational long-term tracker[C]//Asian Conference on Computer Vision.Cham：Springer，2018：595-611.
[42] LUKEZIC A，VOJIR T，ˇCEHOVIN ZAJC L，et al.Discriminative correlation filter with channel and spatial reliability[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：6309-6318.
[43] ZHU Z，HUANG G，ZOU W，et al.UCT：learning unified convolutional networks for real-time visual tracking[C]//Proceedings of the IEEE International Conference on Computer Vision Workshops，2017：1973-1982.
[44] ZHU G，WANG J，WU Y，et al.MC-HOG correlation tracking with saliency proposal[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2016.
[45] SUPANCIC J，RAMANAN D.Tracking as online decision-making：learning a policy from streaming videos with reinforcement learning[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：322-331.
[46] DAI K，ZHANG Y，WANG D，et al.High-performance long-term tracking with meta-updater[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：6298-6307.
[47] HELD D，THRUN S，SAVARESE S.Learning to track at 100 fps with deep regression networks[C]//European Conference on Computer Vision.Cham：Springer，2016：749-765.
[48] BERTINETTO L，VALMADRE J，HENRIQUES J F，et al.Fully-convolutional Siamese networks for object tracking[C]//European Conference on Computer Vision.Cham：Springer，2016：850-865.
[49] VALMADRE J，BERTINETTO L，HENRIQUES J，et al.End-to-end representation learning for correlation filter based tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：2805-2813.
[50] LI B，YAN J，WU W，et al.High performance visual tracking with Siamese region proposal network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：8971-8980.
[51] HENRIQUES J F，CASEIRO R，MARTINS P，et al.High-speed tracking with kernelized correlation filters[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2014，37（3）：583-596.
[52] DANELLJAN M，ROBINSON A，KHAN F S，et al.Beyond correlation filters：learning continuous convolution operators for visual tracking[C]//European Conference on Computer Vision.Cham：Springer，2016：472-488.
[53] CHOI J，KWON J，LEE K M.Visual tracking by reinforced decision making[J].Computer Vision & Image Understanding，2017：10-19.
[54] ZHU Z，WU W，ZOU W，et al.End-to-end flow correlation tracking with spatial-temporal attention[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：548-557.
[55] DANELLJAN M，BHAT G，KHAN F S，et al.Atom：accurate tracking by overlap maximization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：4660-4669.
[56] LIANG Y，WU Q，LIU Y，et al.Robust correlation filter tracking with shepherded instance-aware proposals[C]//Proceedings of the 26th ACM International Conference on Multimedia，2018：420-428.
[57] CHENG X，ZHANG Y，ZHOU L，et al.Visual tracking via auto-encoder pair correlation filter[J].IEEE Transactions on Industrial Electronics，2019，67（4）：3288-3297.
[58] HONG Z，CHEN Z，WANG C，et al.Multi-store tracker（muster）：a cognitive psychology inspired approach to object tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2015：749-758.
[59] LEE H，CHOI S，KIM C.A memory model based on the Siamese network for long-term tracking[C]//Proceedings of the European Conference on Computer Vision（ECCV） Workshops，2018.
[60] XIE Y，XIAO J，HUANG K，et al.Correlation filter selection for visual tracking using reinforcement learning[J].IEEE Transactions on Circuits and Systems for Video Technology，2018，30（1）：192-204.
[61] YANG T，CHAN A B.Recurrent filter learning for visual tracking[C]//Proceedings of the IEEE International Conference on Computer Vision Workshops，2017：2010-2019.
[62] ZHANG L，GONZALEZ-GARCIA A，WEIJER J，et al.Learning the model update for siamese trackers[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：4010-4019.