Review of Cross-Domain Object Detection Algorithms Based on Depth Domain Adaptation

doi:10.3778/j.issn.1002-8331.2210-0063

Abstract

Abstract: In recent years, the object detection algorithm based on deep learning has attracted wide attention due to its high detection performance. It has been successfully applied in many fields such as automatic driving and human-computer interaction and has achieved certain achievements. However, traditional deep learning methods are based on the assumption that the training set （source domain） and the test set （target domain） follow the same distribution, but this assumption is not realistic, which severely reduces the generalization performance of the model. How to align the distribution of the source domain and the target domain so as to improve the generalization of the object detection model has become a research hotspot in the past two years. This article reviews cross-domain object detection algorithms. First, it introduces the preliminary knowledge of cross-domain object detection：depth domain adaptation and object detection. The cross-domain object detection is decomposed into two small areas for an overview, in order to understand its development from the bottom logic. In turn, this article introduces the latest developments in cross-domain object detection algorithms, from the perspectives of differences, confrontation, reconstruction, hybrid and other five categories, and sorts out the research context of each category. Finally, this article summarizes and looks forward to the development trend of cross-domain object detection algorithms.

Key words: deep learning, object detection, deep domain adaptation, cross-domain object detection

摘要： 近年来，基于深度学习的目标检测算法在自动驾驶、人机交互等众多域上有着成功的应用，且因其检测性能较高引起学者的广泛关注。传统的深度学习方法一般基于源域与目标域服从同一分布的假设，但该假设不具备现实性，严重地降低了模型的泛化性能。如何对齐源域与目标域的分布，提高目标检测模型的泛化性成为近两年的研究热点。对跨域目标检测算法进行了综述，介绍了跨域目标检测的预备知识：深度域适应和目标检测，将跨域目标检测分解为两个子问题进行了概述，从底层逻辑理解其发展进程；给出了跨域目标检测算法的最新进展，从差异、对抗、重构、混合和其他等几个分类角度切入，梳理了每个类别的研究脉络并对比了在不同数据集上的性能；通过对目前跨域目标检测算法的梳理和总结，就其未来的研究方向进行展望。

关键词: 深度学习, 目标检测, 深度域适应, 跨域目标检测

LIU Hualing, PI Changpeng, ZHAO Chenyu, QIAO Liang. Review of Cross-Domain Object Detection Algorithms Based on Depth Domain Adaptation[J]. Computer Engineering and Applications, 2023, 59(8): 1-12.

刘华玲, 皮常鹏, 赵晨宇, 乔梁. 基于深度域适应的跨域目标检测算法综述[J]. 计算机工程与应用, 2023, 59(8): 1-12.

References

[1] 范苍宁，刘鹏，肖婷，等.深度域适应综述：一般情况与复杂情况[J].自动化学报，2021，47（3）：515-548.
FAN C N，LIU P，XIAO T，et al.A review of deep domain adaptation：general situation and complex situation[J].Acta Automatica Sinica，2021，47（3）：515-548.
[2] LONG M，CAO Y，WANG J，et al.Learning transferable features with deep adaptation networks[C]//International Conference on Machine Learning，2015：97-105.
[3] PEI Z，CAO Z，LONG M，et al.Multi-adversarial domain adaptation[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2018.
[4] BOUSMALIS K，TRIGEORGIS G，SILBERMAN N，et al.Domain separation networks[J].arXiv：1608.06019，2016.
[5] ZHU J Y，PARK T，ISOLA P，et al.Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：2223-2232.
[6] GHIFARY M，KLEIJN W B，ZHANG M.Domain adaptive neural networks for object recognition[C]//Pacific Rim International Conference on Artificial Intelligence.Cham：Springer，2014：898-904.
[7] TZENG E，HOFFMAN J，ZHANG N，et al.Deep domain confusion：maximizing for domain invariance[J].arXiv：1412.3474，2014.
[8] SUN B，FENG J，SAENKO K.Return of frustratingly easy domain adaptation[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2016.
[9] SUN B，SAENKO K.Deep coral：correlation alignment for deep domain adaptation[C]//European Conference on Computer Vision.Cham：Springer，2016：443-450.
[10] CHEN C，CHEN Z，JIANG B，et al.Joint domain alignment and discriminative feature learning for unsupervised deep domain adaptation[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2019：3296-3303.
[11] ZELLINGER W，GRUBINGER T，LUGHOFER E，et al.Central moment discrepancy （cmd） for domain-invariant representation learning[J].arXiv：1702.08811，2017.
[12] DAMODARAN B B，KELLENBERGER B，FLAMARY R，et al.Deepjdot：deep joint distribution optimal transport for unsupervised domain adaptation[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：447-463.
[13] SHEN J，QU Y，ZHANG W，et al.Wasserstein distance guided representation learning for domain adaptation[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2018.
[14] ROZANTSEV A，SALZMANN M，FUA P.Beyond sharing weights for deep domain adaptation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2018，41（4）：801-814.
[15] SHU X，QI G J，TANG J，et al.Weakly-shared deep transfer networks for heterogeneous-domain knowledge propagation[C]//Proceedings of the 23rd ACM International Conference on Multimedia，2015：35-44.
[16] XU S H，MU X D，CHAI D，et al.Domain adaption algorithm with ELM parameter transfer[J].Acta Autom Sin，2018，44：311-317.
[17] IOFFE S，SZEGEDY C.Batch normalization：accelerating deep network training by reducing internal covariate shift[C]//International Conference on Machine Learning，2015：448-456.
[18] CHANG W G，YOU T，SEO S，et al.Domain-specific batch normalization for unsupervised domain adaptation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：7354-7362.
[19] ROY S，SIAROHIN A，SANGINETO E，et al.Unsupervised domain adaptation using feature-whitening and consensus loss[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：9471-9480.
[20] LI Y，WANG N，SHI J，et al.Revisiting batch normalization for practical domain adaptation[J].arXiv：1603.04779，2016.
[21] ULYANOV D，VEDALDI A，LEMPITSKY V.Improved texture networks：maximizing quality and diversity in feed-forward stylization and texture synthesis[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：6924-6932.
[22] GOPALAN R，LI R，CHELLAPPA R.Domain adaptation for object recognition：an unsupervised approach[C]//2011 International Conference on Computer Vision，2011：999-1006.
[23] GONG B，SHI Y，SHA F，et al.Geodesic flow kernel for unsupervised domain adaptation[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition，2012：2066-2073.
[24] CHOPRA S，BALAKRISHNAN S，GOPALAN R.Dlid：deep learning for domain adaptation by interpolating between domains[C]//ICML Workshop on Challenges in Representation Learning，2013.
[25] GONG R，LI W，CHEN Y，et al.Dlow：domain flow for adaptation and generalization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：2477-2486.
[26] XU X，ZHOU X，VENKATESAN R，et al.d-SNE：domain adaptation using stochastic neighborhood embedding[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：2497-2506.
[27] YANG B，YUEN P C.Cross-domain visual representations via unsupervised graph alignment[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2019：5613-5620.
[28] MA X，ZHANG T，XU C.Gcan：graph convolutional adversarial network for unsupervised domain adaptation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：8266-8276.
[29] YANG Z，ZHAO J J，DHINGRA B，et al.Glomo：unsupervised learning of transferable relational graphs[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems，2018：8964-8975.
[30] GANIN Y，LEMPITSKY V.Unsupervised domain adaptation by backpropagation[C]//International Conference on Machine Learning，2015：1180-1189.
[31] CHEN X，WANG S，LONG M，et al.Transferability vs.discriminability：batch spectral penalization for adversarial domain adaptation[C]//International Conference on Machine Learning，2019：1081-1090.
[32] TZENG E，HOFFMAN J，SAENKO K，et al.Adversarial discriminative domain adaptation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：7167-7176.
[33] VOLPI R，MORERIO P，SAVARESE S，et al.Adversarial feature augmentation for unsupervised domain adaptation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：5495-5504.
[34] LONG M，CAO Z，WANG J，et al.Conditional adversarial domain adaptation[J].arXiv：1705.10667，2017.
[35] TZENG E，HOFFMAN J，DARRELL T，et al.Simultaneous deep transfer across domains and tasks[C]//Proceedings of the IEEE International Conference on Computer Vision，2015：4068-4076.
[36] BENGIO Y.Learning deep architectures for AI[M].[S.l.]：Now Publishers Inc，2009.
[37] GLOROT X，BORDES A，BENGIO Y.Domain adaptation for large-scale sentiment classification：a deep learning approach[C]//International Conference on Machine Learning，2011.
[38] CHEN M，XU Z，WEINBERGER K，et al.Marginalized denoising autoencoders for domain adaptation[J].arXiv：1206.4683，2012.
[39] GHIFARY M，KLEIJN W B，ZHANG M，et al.Deep reconstruction-classification networks for unsupervised domain adaptation[C]//European Conference on Computer Vision.Cham：Springer，2016：597-613.
[40] ZHUANG F，CHENG X，LUO P，et al.Supervised representation learning：transfer learning with deep autoencoders[C]//Twenty-Fourth International Joint Conference on Artificial Intelligence，2015.
[41] SUN R，ZHU X，WU C，et al.Not all areas are equal：transfer learning for semantic segmentation via hierarchical region selection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：4360-4369.
[42] TSAI J C，CHIEN J T.Adversarial domain separation and adaptation[C]//2017 IEEE 27th International Workshop on Machine Learning for Signal Processing（MLSP），2017：1-6.
[43] ZHAO A，DING M，GUAN J，et al.Domain-invariant projection learning for zero-shot recognition[J].arXiv：1810.08326，2018.
[44] LIU M Y，TUZEL O.Coupled generative adversarial networks[J].arXiv：1606.07536，2016.
[45] XIA Y，HE D，QIN T，et al.Dual learning for machine translation[J].arXiv：1611.00179，2016.
[46] YI Z，ZHANG H，TAN P，et al.Dualgan：unsupervised dual learning for image-to-image translation[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：2849-2857.
[47] KIM T，CHA M，KIM H，et al.Learning to discover cross-domain relations with generative adversarial networks[C]//International Conference on Machine Learning，2017：1857-1865.
[48] SANKARANARAYANAN S，BALAJI Y，CASTILLO C D，et al.Generate to adapt：aligning domains using generative adversarial networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：8503-8512.
[49] DOLLáR P，WOJEK C，SCHIELE B，et al.Pedestrian detection：a benchmark[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition，2009：304-311.
[50] UIJLINGS J R R，VAN DE SANDE K E A，GEVERS T，et al.Selective search for object recognition[J].International Journal of Computer Vision，2013，104（2）：154-171.
[51] VEDALDI A，GULSHAN V，VARMA M，et al.Multiple kernels for object detection[C]//2009 IEEE 12th International Conference on Computer Vision，2009：606-613.
[52] YU Y，ZHANG J，HUANG Y，et al.Object detection by context and boosted HOG-LBP[C]//ECCV Workshop on PASCAL VOC，2010.
[53] ZITNICK C L，DOLLáR P.Edge boxes：locating object proposals from edges[C]//European Conference on Computer Vision.Cham：Springer，2014：391-405.
[54] SERMANET P，EIGEN D，ZHANG X，et al.Overfeat：integrated recognition，localization and detection using convolutional networks[J].arXiv：1312.6229，2013.
[55] GIRSHICK R，DONAHUE J，DARRELL T，et al.Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2014：580-587.
[56] HE K，ZHANG X，REN S，et al.Spatial pyramid pooling in deep convolutional networks for visual recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，37（9）：1904-1916.
[57] GIRSHICK R.Fast R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision，2015：1440-1448.
[58] REN S，HE K，GIRSHICK R，et al.Faster R-CNN：towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2016，39（6）：1137-1149.
[59] DAI J，LI Y，HE K，et al.R-FCN：object detection via region-based fully convolutional networks[J].arXiv：1605.
06409，2016.
[60] HE K，GKIOXARI G，DOLLáR P，et al.Mask R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：2961-2969.
[61] REDMON J，DIVVALA S，GIRSHICK R，et al.You only look once：unified，real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：779-788.
[62] REDMON J，FARHADI A.YOLO9000：better，faster，stronger[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：7263-7271.
[63] REDMON J，FARHADI A.Yolov3：an incremental improvement[J].arXiv：1804.02767，2018.
[64] BOCHKOVSKIY A，WANG C Y，LIAO H Y M.Yolov4：optimal speed and accuracy of object detection[J].arXiv：2004.10934，2020.
[65] JOCHER G，NISHIMURA K，MINEEVA T，et al.Yolov5[EB/OL].[2022-08-10].https：//github.com/ultralytics/yolov5.
[66] LIU W，ANGUELOV D，ERHAN D，et al.SSD：single shot multibox detector[C]//European Conference on Computer Vision.Cham：Springer，2016：21-37.
[67] FU C Y，LIU W，RANGA A，et al.DSSD：deconvolutional single shot detector[J].arXiv：1701.06659，2017.
[68] JEONG J，PARK H，KWAK N.Enhancement of SSD by concatenating feature maps for object detection[J].arXiv：1705.09587，2017.
[69] LI Z，ZHOU F.FSSD：feature fusion single shot multibox detector[J].arXiv：1712.00960，2017.
[70] SHEN Z，LIU Z，LI J，et al.DSOD：learning deeply supervised object detectors from scratch[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：1919-1927.
[71] LIN T Y，GOYAL P，GIRSHICK R，et al.Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：2980-2988.
[72] KHODABANDEH M，VAHDAT A，RANJBAR M，et al.A robust learning approach to domain adaptive object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：480-490.
[73] CAI Q，PAN Y，NGO C W，et al.Exploring object relation in mean teacher for cross-domain detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：11457-11466.
[74] 冯佳伟，褚晶辉，吕卫.基于跨域学习的单样本目标检测方法[J].激光与光电子学进展：1-12[2022-08-10].http：//kns.cnki.net/kcms/detail/31.1690.TN.20220713.1451.440.html.
FENG J W，CHU J H，LV W.One-shot object detection based on cross-domain learning[J].Laser & Optoelectronics Progress：1-12[2022-08-10].http：//kns.cnki.net/kcms/detail/31.1690.TN.20220713.1451.440.html.
[75] 郭强，浦世亮，张世峰，等.适合跨域目标检测的雾霾图像增强[J].中国图象图形学报，2022，27（5）：1481-1492.
GUO Q，PU S L，ZHANG S F，et al.Cross-domain object detection based foggy image enhancement[J].Journal of Image and Craphics，2022，27（5）：1481-1492.
[76] CHEN Y，LI W，SAKARIDIS C，et al.Domain adaptive faster R-CNN for object detection in the wild[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：3339-3348.
[77] ZHU X，PANG J，YANG C，et al.Adapting object detectors via selective cross-domain alignment[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：687-696.
[78] 秦雨欣.遥感图像中跨域目标检测研究[D].北京：北方工业大学，2022.
QIN Y X.Research on cross-domain object detection in remote sensing images[D].Beijing：North China University of Technology，2022.
[79] LI Y Z，DAI X，MA C Y，et al.Cross-domain object detection via adaptive selftraining[J].arXiv：2111.13216，2021.
[80] FUJII K，KERA H，KAWAMOTO K.Adversarially trained object detector for unsupervised domain adaptation[J].arXiv：2109.05751，2021.
[81] ZHUANG C，HAN X，HUANG W，et al.iFAN：image-instance full alignment networks for adaptive object detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2020：13122-13129.
[82] CHEN C，ZHENG Z，DING X，et al.Harmonizing transferability and discriminability for adapting object detectors[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：8869-8878.
[83] 胡杰，徐博远，熊宗权，等.基于多尺度掩码分类域自适应网络的跨域目标检测算法[J].汽车工程，2022，44（9）：1327-1338.
HU J，XU B Y，XIONG Z Q，et al.Cross-domain object detection algorithm based on multi-scale mask classification domain adaptive network[J].Automotive Engineering，2022，44（9）：1327-1338.
[84] LIN C T.Cross domain adaptation for on-road object detection using multimodal structure-consistent image-to-image translation[C]//2019 IEEE International Conference on Image Processing（ICIP），2019：3029-3030.
[85] LIU S，JOHN V，BLASCH E，et al.IR2VI：enhanced night environmental perception by unsupervised thermal image translation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops，2018：1153-1160.
[86] ARRUDA V F，BERRIEL R F，PAIX?O T M，et al.Cross-domain object detection using unsupervised image translation[J].Expert Systems with Applications，2022，192：116334.
[87] 谢光达，李洋，曲洪权，等.基于改进Transformer的小目标车辆精确检测算法[J].激光与光电子学进展，2022，59（18）：364-371.
XIE G D，LI Y，QU H Q，et al.Small target accurate vehicle detection algorithm based on improved transformer[J].Laser and Optoelectronics Progress，2022，59（18）：364-371.
[88] 那峙雄，樊涛，孙涛，等.多损失融合的小样本光伏组件隐裂检测算法[J].计算机科学与探索，2022，16（2）：458-467.
NA Z X，FAN T，SUN T，et al.Micro-cracks detection of solar cells based on few shot samples with multi-loss[J].Journal of Frontiers of Computer Science and Technology，2022，16（2）：458-467.
[89] INOUE N，FURUTA R，YAMASAKI T，et al.Cross-domain weakly-supervised object detection through progressive domain adaptation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：5001-5009.
[90] YU F，WANG D，CHEN Y，et al.Unsupervised domain adaptation for object detection via cross-domain semi-supervised learning[J].arXiv：1911.07158，2019.
[91] ZHENG Y，HUANG D，LIU S，et al.Cross-domain object detection through coarse-to-fine feature adaptation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：13766-13775.
[92] KIM S，CHOI J，KIM T，et al.Self-training and adversarial background regularization for unsupervised domain adaptive one-stage object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：6092-6101.
[93] HSU H K，YAO C H，TSAI Y H，et al.Progressive domain adaptation for object detection[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision，2020：749-757.
[94] 范佳琦.基于生成对抗网络的智能汽车跨域检测算法研究[D].长春：吉林大学，2022.
FAN J Q.Research on intelligent vehicle cross-domain detection algorithm based on generative adversarial network[D].Changchun：Jilin University，2022.
[95] 吴泽远，朱明.基于图像风格迁移的端到端跨域目标检测[J].计算机系统应用，2021，30（1）：194-199.
WU Z Y，ZHU M.End-to-end cross-domain object detection based on image style transfer[J].Computer Systems & Applications，2021，30（1）：194-199.
[96] XU M，WANG H，NI B，et al.Cross-domain detection via graph-induced prototype alignment[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：12355-12364.
[97] XU C D，ZHAO X R，JIN X，et al.Exploring categorical regularization for domain adaptive object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：11724-11733.