结合移动对象发现和对比学习的无监督跟踪

doi:10.3778/j.issn.1002-8331.2309-0453

摘要/Abstract

摘要： 常规的深度学习跟踪方法需要使用大量人工注释的视频标签来完成对目标跟踪任务的监督学习，而无监督跟踪方法可以在未标注视频中进行模型训练，这有利于部署到实际场景中。传统的无监督跟踪方法严重依赖训练样本的空间信息探索，难以持续跟踪具有强烈运动变化的目标对象。提出一种基于循环一致性理论的无监督跟踪方法，采用无监督光流估计与动态规划发现移动目标，通过循环记忆学习方案构建内存队列来利用丰富的时间信息，并提出基于对比学习思想的方法完成对模板的更新。实验结果表明，提出的跟踪方法在VOT2016以及VOT2018数据集上的EAO分别达到了0.402和0.344，主要性能指标与一些主流监督学习方法的跟踪效果相当。

关键词: 目标跟踪, 无监督学习, 循环一致性, 移动对象发现, 对比学习

Abstract: The conventional deep learning tracking methods require numerous manually annotated video labels to complete supervised learning for object tracking tasks, while unsupervised tracking methods can train models by using unlabeled videos, which is beneficial for deployment in real-world scenarios. Traditional unsupervised tracking methods heavily rely on exploring the spatial information of training samples, so it is difficult to track the target objects which have intense movement changes over a long period of time. This paper proposes an unsupervised tracking method based on the theory of cyclic consistency, which adopts unsupervised optical flow estimation and dynamic programming to discover the moving targets. In addition, for the purpose of utilizing the rich temporal information, a cyclic memory learning scheme is used to construct a memory queue, and a method based on contrastive learning is proposed to update the template. The experimental results show that the proposed tracking method achieves EAO of 0.402 and 0.344 on the VOT2016 and VOT2018 datasets respectively. The main performance indicators are comparable to some mainstream supervised learning methods in terms of tracking performance.

Key words: object tracking, unsupervised learning, cycle consistency, moving object discovery, contrastive learning

段苛苛, 郑俊蓉, 晏泽. 结合移动对象发现和对比学习的无监督跟踪[J]. 计算机工程与应用, 2025, 61(4): 141-149.

DUAN Keke, ZHENG Junrong, YAN Ze. Unsupervised Tracking Combining Moving Object Discovery and Contrastive Learning[J]. Computer Engineering and Applications, 2025, 61(4): 141-149.

参考文献

[1] 孟琭, 杨旭. 目标跟踪算法综述[J]. 自动化学报, 2019, 45(7): 1244-1260.
MENG L, YANG X. A survey of object tracking algorithms[J]. Acta Automatica Sinica, 2019, 45(7): 1244-1260.
[2] 纪守领, 杜天宇, 邓水光, 等. 深度学习模型鲁棒性研究综述[J]. 计算机学报, 2022, 45(1): 190-206.
JI S L, DU T Y, DENG S G, et al. Robustness certification research on deep learning models: a survey[J]. Chinese Journal of Computers, 2022, 45(1): 190-206.
[3] 陈云芳, 吴懿, 张伟. 基于孪生网络结构的目标跟踪算法综述[J]. 计算机工程与应用, 2020, 56(6): 10-18.
CHEN Y F, WU Y, ZHANG W. Survey of target tracking alogorithnm based on siamese network structure[J]. Computer Engineering and Application, 2020, 56(6): 10-18.
[4] MARVASTI-ZADEH S M, CHEN G L, GHANEI-YAKHDAN H, et al. Deep learning for visual tracking: a comprehensive survey[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 23(5): 3943-3968.
[5] 韩旭. 基于前后向跟踪验证的无监督目标跟踪算法研究[D]. 南京: 东南大学, 2021.
HAN X. Research of unsupervised object tracking algorithm based on forward-backward tracking verification[D]. Nanjing: Southeast University, 2021.
[6] WANG N, SONG Y B, MA C, et al. Unsupervised deep tracking[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 1308-1317.
[7] YUANG WH, WANG M Y, LIU Q F, et al. Self-supercised object tracking with cycle-consistent siamese network[C]//Proceedings of the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2020: 10351-10358.
[8] LI B, YAN J J, WU W. et al. High performance visual tracking with siamese region proposal network[C]//Proceedings of the 2018 IEEE/CVF conference on Computer Vision and Pattern Recognition, 2018: 8971-8980.
[9] SIO C H, MA Y J, SHUAI H H, et al. S2SiamFC: self-supervised fully convolution siamese network for visual tracking[C]//Proceedings of the 28th ACM International Conference on Multimedia, 2020: 1948-1957.
[10] ZHENG J L, MA C, PENG H W, et al. Learning to track objects from unlabled videos[C]//Proceedings of the 2021 IEEE International Conference on Computer, 2021: 13546-13555.
[11] SHEN Q H, QIAO L, GUO J Y, et al. Unsupervised learning of accurate siamese tracking[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 8101-8110.
[12] ZHOU Z R, FU H Y, YOU S Y, et al. GUSOT: green and unsupervised single object tracking for long video sequence[C]//Proceedings of the 2022 IEEE Conference on Computer Vision and Pattern Recognition, 2022.
[13] KARAMATH E, KIRBIZ S. MixCycle: unsupervised speech separation via cyclic mixture permutation invariant training[J]. IEEE Signal Processing Letters, 2022, 29: 2637-2641.
[14] CHEN T, KORNBLITH S, NOROUZI M. A simple framework for contrastive learning of visual representations[C]//Proceedings of the 37th International Conference on Machine Learning, 2020: 1597-1607.
[15] TIAN Y L, SUN C, POOLE B, et al. What makes for good views for contrastive Learning[C]//Proceedings of the 34th Conference on Neural Information Processing Systems, 2020.
[16] HE K M, FAN H Q, WU Y X, et al. Momentum contrast for unsupervised visual representation learning[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 9729-9738.
[17] WU Q Q, WAN J, CHEN A B, et al. Progressive unsupervised learning for visual object tracking[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 2993-3002.
[18] GRILL J B, STRUB F, ALTCHE F, et al. Bootstrap your own latent: a new approach to self-supervised learning[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 1137-1151.
[19] CHEN X L, HE K M. Exploring simple siamese representation learning[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 15750-15758.
[20] SHEN Y H, JI Z W, HONG X P, et al. Noise-aware fully webly supervised object detection[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 11326-11335.
[21] YANG L R, MENG F M, LI H L, et al. Learning with noisy class labels for instance segmentation[C]//Proceedings of the 16th European Conference on Computer Vision, 2020.
[22] LIU L, ZHANG J N, HE R F, et al. Learning by analogy: reliable supervision from transformations for unsupervised optical flow estimation[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 6489-6498.
[23] CHEN M B, JUNG C. Fully-convolutional siamese networks for object tracking[C]//Proceedings of the 25th IEEE International Conference on Image Processing, 2018.
[24] HE K M, ZHANG X Y, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the 25th IEEE International Conference on Image Processing, 2018.
[25] JIANG B R, LUO R X, MAO J Y, et al. Acquisition of localization confidence for accurate object detection[C]//Proceedings of the 15th European Conference on Computer Vision, 2018: 816-832.
[26] BERTINETTO L, VALMADRE J, GOLODETZ S, et al. Staple: complementary learners for real-time tracking[C]//Proceedings of the 2016 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2016: 1401-1409.
[27] SHEN Z L, DAI Y C, RAO Z B, et al. CFNet: cascade and fused cost volume for robust stereo[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 13906-13915.
[28] DANELLJAN M, BHAT G, KHAN F S, et al. Accurate scale estimation for robust visual tracking[C]//Proceedings of the 2014 British Machine Vision Conference, 2014.
[29] DANELLJAN M, BHAT G, SHAHBAZ K F, et al. ATOM: accurate tracking by overlap maximization[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, 2018: 4660-4669.
[30] ZHU Z, WANG Q, LI B, et al. Distractor-aware siamese networks for visual object tracking[C]//Proceedings of the 15th European Conference on Computer Vision, 2018: 103-119.
[31] HENRIQUES F J, CASEIRO R, MARTINS P, et al. High-speed tracking with kernelized correlation filters[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3): 583-596.
[32] DANELLJAN M, BHAT G, KHAN F S, et al. ECO: efficient convolution operators for tracking[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 6638-6646.