DiMP Based Transfer Learning and Model Recomposion for RGBD Tracking

doi:10.3778/j.issn.1002-8331.2105-0125

Abstract

Abstract: The depth map provides information about the three-dimensional spatial structure of the moving objects, so it can be used to improve tracking performance. However, the current lack of RGBD-based target tracking datasets makes it impossible to directly train deep learning trackers based on RGBD input. In this regard, a model transfer learning algorithm based on knowledge alignment is proposed. Models trained on other RGBD tasks can be easily transferred to the DiMP-based object tracking algorithm, and there is no need to recalculate the transfer parameters for different tracking objects. In addition, a smoothing algorithm is proposed to solve the problem of instability of depth information. The experimental results on the VOTRGBD dataset show that the transferred and fused features can significantly improve the discrimination between the target and the background, and effectively improve the performance of the tracker.

Key words: transfer learning, depth information, information fusion, object tracking

摘要： 深度图可以提供运动目标所处的三维空间结构信息，因此可以用来提升跟踪性能。但目前缺少基于RGBD的目标跟踪数据集，无法直接训练RGBD输入下的深度学习跟踪器。对此，提出了一种基于知识对齐的模型迁移重组算法，可以方便地将在其他RGBD任务上训练得到的模型迁移到基于DiMP的跟踪算法上来，并且对于不同的跟踪对象不需要重新计算迁移参数。另外，针对深度图信息不稳定的问题，提出了一种高效的平滑稳定算法。在VOTRGBD数据集上的实验结果表明，迁移融合后的特征可以显著提升目标和背景之间的判别性，有效提升跟踪器的性能。

关键词: 模型迁移, 深度信息, 信息融合, 目标跟踪

QIU Shoumeng, GU Yuzhang, YUAN Zeqiang. DiMP Based Transfer Learning and Model Recomposion for RGBD Tracking[J]. Computer Engineering and Applications, 2022, 58(20): 181-187.

邱守猛, 谷宇章, 袁泽强. 基于DiMP模型迁移重组的RGBD目标跟踪[J]. 计算机工程与应用, 2022, 58(20): 181-187.

References

[1] WANG Q，FANG J，YUAN Y.Multi-cue based tracking[J].Neurocomputing，2014，131：227-236.
[2] SHI H，GAO C，SANG N.Using consistency of depth gradient to improve visual tracking in RGB-D sequences[C]//2015 Chinese Automation Congress（CAC），2015：518-522.
[3] XIAO J，STOLKIN R，GAO Y，et al.Robust fusion of color and depth data for RGB-D target tracking using adaptive range-invariant depth models and spatio-temporal consistency constraints[J].IEEE Transactions on Cybernetics，2017，48（8）：2485-2499.
[4] ZHANG H，CAI M，LI J.A real-time RGB-D tracker based on KCF[C]//2018 Chinese Control and Decision Conference（CCDC），2018：4856-4861.
[5] HENRIQUES J F，CASEIRO R，MARTINS P，et al.High-speed tracking with kernelized correlation filters[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，37（3）：583-596.
[6] ZHANG W，YANG K，XIN Y，et al.An occlusion-aware RGB-D visual object tracking method based on siamese network[C]/2020 15th IEEE International Conference on Signal Processing（ICSP），2020：327-332.
[7] ZHAO P，LIU Q，WANG W，et al.TSDM：tracking by SiamRPN++ with a depth-refiner and a mask-generator[J].arXiv：2005.04063，2020.
[8] LI B，WU W，WANG Q，et al.SiamRPN++：evolution of siamese visual tracking with very deep networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），2019：4282-4291.
[9] KRIZHEVSKY A，SUTSKEVER I，HINTON G E.Imagenet classification with deep convolutional neural networks[J].Communications of the ACM，2017，60（6）：84-90.
[10] BHAT G，DANELLJAN M，GOOL L，et al.Learning discriminative model prediction for tracking[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：6182-6191.
[11] JIANG B，LUO R，MAO J，et al.Acquisition of localization confidence for accurate object detection[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：784-799.
[12] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2016：770-778.
[13] FERNANDO B，HABRARD A，SEBBAN M，et al.Unsupervised visual domain adaptation using subspace alignment[C]//Proceedings of the IEEE International Conference on Computer Vision，2013：2960-2967.
[14] KRISTAN M，MATAS J，LEONARDIS A，et al.The seventh visual object tracking vot2019 challenge results[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops，2019.
[15] JIANG J，ZHENG L，LUO F，et al.RedNet：residual encoder-decoder network for indoor RGB-D semantic segmentation[J].arXiv：1806.01054，2018.
[16] HU Xinxin，YANG Kailun，LEI Fei，et al.ACNET：attention based network to exploit complementary features for RGBD semantic segmentation[C]//2019 IEEE International Conference on Image Processing（ICIP），2019：1440-1444.
[17] DANELLJAN M，BHAT G，SHAHBAZ KHAN F，et al.ECO：efficient convolution operators for tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：6638-6646.
[18] BERTINETTO L，VALMADRE J，HENRIQUES J，et al.Fully-convolutional siamese networks for object tracking[C]//European Conference on Computer Vision，2016：850-865.
[19] NAM H，HAN B.Learning multi-domain convolutional neural networks for visual tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：4293-4302.
[20] YAN B，ZHANG X，WANG D，et al.Alpha-refine：boosting tracking performance by precise bounding box estimation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：5289-5298.
[21] ZHANG Z，LI B，HU W，et al.Towards accurate pixel-wise object tracking by attention retrieval[J].IEEE Transactions on Image Processing，2021：8553-8566.
[22] MA Z，WANG L，ZHANG H，et al.RPT：learning point set representation for siamese visual tracking[C]//European Conference on Computer Vision.Switzerland：Springer，2020：653-665.