Review of Unsupervised Domain Adaptation in Medical Image Segmentation

doi:10.3778/j.issn.1002-8331.2307-0421

Abstract

Abstract: Medical image segmentation has broad application prospects in the field of medical image processing, providing auxiliary information for diagnosis and treatment by locating and segmenting interested organs, tissues, or lesion areas. However, there is a domain offset problem between different modalities of medical images, which can lead to a significant decrease in the performance of the segmentation model during actual deployment. Domain adaptation technology is an effective way to solve this problem, especially unsupervised domain adaptation, which has become a research hotspot in the field of medical image processing because it does not require target domain label information. At present, there are relatively few review reports on unsupervised domain adaptation research in medical image segmentation. Therefore, this paper summarizes, analyzes, and prospects the future of unsupervised domain adaptation research in medical image segmentation in recent years, hoping to help relevant researchers quickly understand and familiarize themselves with the current research status and trends in this field.

Key words: medical image segmentation, domain shift, domain adaptation, unsupervised domain adaptation

摘要： 医学图像分割在医学图像处理领域中具有广泛的应用前景，通过定位和分割出感兴趣的器官、组织或病变区域，为诊断和治疗提供辅助信息。但不同模态医学图像之间存在域偏移问题，这会导致在实际部署时分割模型的性能大幅下降。域适应技术是解决该问题的有效途径，尤其是无监督域适应，因其不需要目标域标签信息而成为医学图像处理领域的研究热点。目前，针对医学图像分割的无监督域适应研究的综述报告相对较少，对近年医学图像分割的无监督域适应的相关研究进行了整理、分析和总结，并对未来进行了展望，希望帮助相关研究人员快速了解并熟悉该领域的研究现状及趋势。

关键词: 医学图像分割, 域偏移, 域适应, 无监督域适应

HU Wei, XU Qiaozhi, GE Xiangwei, YU Lei. Review of Unsupervised Domain Adaptation in Medical Image Segmentation[J]. Computer Engineering and Applications, 2024, 60(6): 10-26.

呼伟, 徐巧枝, 葛湘巍, 于磊. 医学图像分割的无监督域适应研究综述[J]. 计算机工程与应用, 2024, 60(6): 10-26.

References

[1] RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention(MICCAI 2015), Munich, October 5-9, 2015: 234-241.
[2] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
[3] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. arXiv:1409. 1556, 2014.
[4] 徐海, 谢洪涛, 张勇东.视觉域泛化技术及研究进展[J].广州大学学报 (自然科学版), 2022, 21(2):42-59.
XU H, XIE H T, ZHANG Y D. Review of domain generalization in vision[J]. Journal of Guangzhou University (Natural Science Edition), 2022, 21(2): 42-59
[5] WANG M, DENG W. Deep visual domain adaptation: a survey[J]. Neurocomputing, 2018, 312: 135-153.
[6] 范苍宁, 刘鹏, 肖婷, 等. 深度域适应综述: 一般情况与复杂情况[J]. 自动化学报, 2021, 47(3): 515-548.
FAN C N, LIU P, XIAO T, et al. A review of deep domain adaptation:general situation and complex situation[J]. Acta Automatica Sinica, 2021, 47(3):515-548.
[7] SUN Y, DAI D, XU S. Rethinking adversarial domain adaptation: orthogonal decomposition for unsupervised domain adaptation in medical image segmentation[J]. Medical Image Analysis, 2022, 82: 102623.
[8] GOLDBERGER J, GORDON S, GREENSPAN H. An efficient image similarity measure based on approximations of KL-divergence between two Gaussian mixtures[C]//Proceedings of the International Conference on Computer Vision, 2003: 487-493.
[9] ZHUANG F, CHENG X, LUO P, et al. Supervised representation learning: transfer learning with deep autoencoders[C]//Proceedings of the 24th International Joint Conference on Artificial Intelligence, 2015.
[10] PAN Y, YAO T, LI Y, et al. Transferrable prototypical networks for unsupervised domain adaptation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 2239-2247.
[11] 田青, 朱雅喃, 马闯.基于深度学习的域适应方法综述[J].数据采集与处理, 2022, 37(3):512-541.
TIAN Q, ZHU Y N, MA C. Review on domain adaptation methods based on deep learning[J]. Journal of Data Acquisition and Processing, 2022, 37(3): 512-541.
[12] FUGLEDE B, TOPSOE F. Jensen-Shannon divergence and Hilbert space embedding[C]//Proceedings of the 18th International Symposium on Information Theory, 2004.
[13] BORGWARDT K M, GRETTON A, RASCH M J, et al. Integrating structured biological data by kernel maximum mean discrepancy[J]. Bioinformatics, 2006, 22(14): e49-e57.
[14] SUN H. Mercer theorem for RKHS on noncompact sets[J]. Journal of Complexity, 2005, 21(3): 337-349.
[15] GRETTON A, BORGWARDT K M, RASCH M J, et al. A kernel two-sample test[J]. The Journal of Machine Learning Research, 2012, 13(1): 723-773.
[16] GHIFARY M, KLEIJN W B, ZHANG M. Domain adaptive neural networks for object recognition[C]//Proceedings of the 13th Pacific Rim International Conference on Artificial Intelligence, Gold Coast, December 1-5, 2014: 898-904.
[17] LONG M, CAO Y, WANG J, et al. Learning transferable features with deep adaptation networks[C]//Proceedings of the International Conference on Machine Learning, 2015:97-105.
[18] LONG M, ZHU H, WANG J, et al. Deep transfer learning with joint adaptation networks[C]//Proceedings of the International Conference on Machine Learning, 2017: 2208-2217.
[19] ZHANG X, YU F X, CHANG S F, et al. Deep transfer network: unsupervised domain adaptation[J]. arXiv:1503.00591, 2015.
[20] XU P, GURRAM P, WHIPPS G, et al. Wasserstein distance based domain adaptation for object detection[J]. arXiv:1909.08675, 2019.
[21] COURTY N, FLAMARY R, HABRARD A, et al. Joint distribution optimal transportation for domain adaptation[C]//Advances in Neural Information Processing Systems, 2017.
[22] LEE C Y, BATRA T, BAIG M H, et al. Sliced Wasserstein discrepancy for unsupervised domain adaptation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 10285-10295.
[23] DIAO S, CHEN X, XIANG D, et al. A two-stage unsupervised domain adaptation method for OCT image segmentation[C]//Medical Imaging 2023: Image Processing, 2023: 827-834.
[24] CHARTSIAS A, JOYCE T, PAPANASTASIOU G, et al. Disentangled representation learning in cardiac image analysis[J]. Medical Image Analysis, 2019, 58: 101535.
[25] ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-image translation with conditional adversarial networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 1125-1134.
[26] ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE International Conference on Computer Vision, 2017: 2223-2232.
[27] HUO Y, XU Z, MOON H, et al. SynSeg-Net: synthetic segmentation without target modality ground truth[J]. IEEE Transactions on Medical Imaging, 2018, 38(4): 1016-1025.
[28] ZENG G, LERCH T D, SCHMARANZER F, et al. Semantic consistent unsupervised domain adaptation for cross-modality medical image segmentation[C]//Proceedings of the 24th International Conference on Medical Image Computing and Computer Assisted Intervention, Strasbourg, September 27-October 1, 2021:201-210.
[29] IACONO P, KHAN N. Structure preserving cycle-GAN for unsupervised medical image domain adaptation[J]. arXiv:2304.09164, 2023.
[30] ZHANG X, ZHANG C, LIU D, et al. SynthMix: mixing up aligned synthesis for medical cross-modality domain adaptation[J]. arXiv:2305.04156, 2023.
[31] HU X, WANG C, SHI Y. Contrastive image synthesis and self-supervised feature adaptation for cross-modality biomedical image segmentation[J]. arXiv:2207.13240, 2022.
[32] LIU Q, CHEN C, QIN J, et al. Feddg: federated domain generalization on medical image segmentation via episodic learning in continuous frequency space[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 1013-1023.
[33] YANG C, GUO X, CHEN Z, et al. Source free domain adaptation for medical image segmentation with fourier style mining[J]. Medical Image Analysis, 2022, 79: 102457.
[34] ZHANG Z, LI Y, SHIN B S. C2‐GAN: content‐consistent generative adversarial networks for unsupervised domain adaptation in medical image segmentation[J]. Medical Physics, 2022, 49(10): 6491-6504.
[35] GANIN Y, USTINOVA E, AJAKAN H, et al. Domain-adversarial training of neural networks[J]. The Journal of Machine Learning Research, 2016, 17(1): 2096-2030.
[36] KAMNITSAS K, BAUMGARTNER C, LEDIG C, et al. Unsupervised domain adaptation in brain lesion segmentation with adversarial networks[C]//Proceedings of the 25th International Conference on Information Processing in Medical Imaging, Boone, NC, June 25-30, 2017: 597-609.
[37] DOU Q, OUYANG C, CHEN C, et al. Unsupervised cross-modality domain adaptation of convnets for biomedical image segmentations with adversarial loss[J]. arXiv:1804. 10916, 2018.
[38] DOU Q, OUYANG C, CHEN C, et al. Pnp-Adanet: plug-and-play adversarial domain adaptation network at unpaired cross-modality cardiac segmentation[J]. IEEE Access, 2019, 7: 99065-99076.
[39] CUI Z, LI C, DU Z, et al. Structure-driven unsupervised domain adaptation for cross-modality cardiac segmentation[J]. IEEE Transactions on Medical Imaging, 2021, 40(12): 3604-3616.
[40] DONG S, PAN Z, FU Y, et al. Partial unbalanced feature transport for cross-modality cardiac image segmentation[J]. IEEE Transactions on Medical Imaging, 2023, 42(6): 1758-1773.
[41] CUI H, YUWEN C, JIANG L, et al. Bidirectional cross-modality unsupervised domain adaptation using generative adversarial networks for cardiac image segmentation[J]. Computers in Biology and Medicine, 2021, 136: 104726.
[42] LIU B, PAN D, SHUAI Z, et al. ECSD-Net: a joint optic disc and cup segmentation and glaucoma classification network based on unsupervised domain adaptation[J]. Computer Methods and Programs in Biomedicine, 2022, 213: 106530.
[43] WANG S, FU Z, WANG B, et al. Fusing feature and output space for unsupervised domain adaptation on medical image segmentation[J]. International Journal of Imaging Systems and Technology, 2023, 33(5): 1672-1681.
[44] XIE Q, LI Y, HE N, et al. Unsupervised domain adaptation for medical image segmentation by disentanglement learning and self-training[J]. IEEE Transactions on Medical Imaging, 2024, 43: 4-14.
[45] WANG R, ZHENG G. CyCMIS: cycle-consistent cross-domain medical image segmentation via diverse image augmentation[J]. Medical Image Analysis, 2022, 76: 102328.
[46] YAO K, SU Z, HUANG K, et al. A novel 3D unsupervised domain adaptation framework for cross-modality medical image segmentation[J]. IEEE Journal of Biomedical and Health Informatics, 2022, 26(10): 4976-4986.
[47] WANG R, ZHOU Q, ZHENG G. EDRL: entropy-guided disentangled representation learning for unsupervised domain adaptation in semantic segmentation[J]. Computer Methods and Programs in Biomedicine, 2023, 240: 107729.
[48] CHARTSIAS A, PAPANASTASIOU G, WANG C, et al. Disentangle, align and fuse for multimodal and semi-supervised image segmentation[J]. IEEE Transactions on Medical Imaging, 2020, 40(3): 781-792.
[49] ZHONG Q, ZENG F, LIAO F, et al. Joint image and feature adaptative attention-aware networks for cross-modality semantic segmentation[J]. Neural Computing and Applications, 2023, 35(5): 3665-3676.
[50] HOFFMAN J, TZENG E, PARK T, et al. Cycada: cycle-consistent adversarial domain adaptation[C]//Proceedings of the International Conference on Machine Learning, 2018: 1989-1998.
[51] WANG Q, DU Y, FAN H, et al. Towards collaborative appearance and semantic adaptation for medical image segmentation[J]. Neurocomputing, 2022, 491: 633-643.
[52] CALISTO M G B, LAI-YUEN S K. C-MADA: unsupervised cross-modality adversarial domain adaptation framework for medical image segmentation[C]//Medical Imaging 2022: Image Processing, 2022: 971-978.
[53] CHEN C, DOU Q, CHEN H, et al. Synergistic image and feature adaptation: towards cross-modality domain adaptation for medical image segmentation[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2019: 865-872.
[54] LEI H, LIU W, XIE H, et al. Unsupervised domain adaptation based image synthesis and feature alignment for joint optic disc and cup segmentation[J]. IEEE Journal of Biomedical and Health Informatics, 2021, 26(1): 90-102.
[55] BALDEON CALISTO M, LAI-YUEN S K, PUENTE-MEJIA B. Stac-Da: structure aware cross-modality domain adaptation framework with image and feature level adaptation for medical image segmentation[J]. SSRN Electronic Journal, 2022. DOI:10.2139/ssrn.4075460.
[56] GHAMSARIAN N, TEJERO J G, NEILA P M, et al. Domain adaptation for medical image segmentation using transformation-invariant self-training[J]. arXiv:2307.16660, 2023.
[57] LEE J H, LEE G. Feature alignment by uncertainty and self-training for source-free unsupervised domain adaptation[J]. Neural Networks, 2023, 161: 682-692.
[58] ARASLANOV N, ROTH S. Self-supervised augmentation consistency for adapting semantic segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 15384-15394.
[59] PAN F, SHIN I, RAMEAU F, et al. Unsupervised intra-domain adaptation for semantic segmentation through self-supervision[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 3764-3773.
[60] HONG J, YU S C H, CHEN W. Unsupervised domain adaptation for cross-modality liver segmentation via joint adversarial learning and self-learning[J]. Applied Soft Computing, 2022, 121: 108729.
[61] SHIN H, KIM H, KIM S, et al. SDC-UDA: volumetric unsupervised domain adaptation framework for slice-direction continuous cross-modality medical image segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023: 7412-7421.
[62] LIU H, XU Z, GAO R, et al. COSST: multi-organ segmentation with partially labeled datasets using comprehensive supervisions and self-training[J]. arXiv:2304.14030, 2023.
[63] JIANG K, GONG T, QUAN L. A medical unsupervised domain adaptation framework based on Fourier transform image translation and multi-model ensemble self-training strategy[J]. International Journal of Computer Assisted Radiology and Surgery, 2023, 18(10):1885-1894.
[64] DU X, LIU Y. Constraint-based unsupervised domain adaptation network for multi-modality cardiac image segmentation[J]. IEEE Journal of Biomedical and Health Informatics, 2021, 26(1): 67-78.
[65] YANG F, LIANG F, LU L, et al. Dual attention-guided and learnable spatial transformation data augmentation multi-modal unsupervised medical image segmentation[J]. Biomedical Signal Processing and Control, 2022, 78: 103849.
[66] LI J, FENG C, LIN X, et al. Utilizing GCN and meta-learning strategy in unsupervised domain adaptation for pancreatic cancer segmentation[J]. IEEE Journal of Biomedical and Health Informatics, 2021, 26(1): 79-89.
[67] HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4700-4708.
[68] LIU J, LIU H, GONG S, et al. Automated cardiac segmentation of cross-modal medical images using unsupervised multi-domain adaptation and spatial neural attention structure[J]. Medical Image Analysis, 2021, 72: 102135.
[69] MA X, YUAN J, CHEN Y, et al. Attention-based cross-layer domain alignment for unsupervised domain adaptation[J]. Neurocomputing, 2022, 499: 1-10.
[70] CHEN X, KUANG T, DENG H, et al. Dual adversarial attention mechanism for unsupervised domain adaptive medical image segmentation[J]. IEEE Transactions on Medical Imaging, 2022, 41(11): 3445-3453.
[71] HINTON G, VINYALS O, DEAN J. Distilling the knowledge in a neural network[J]. arXiv:1503.02531, 2015.
[72] LIU P, TRAN C T, KONG B, et al. CADA: multi-scale collaborative adversarial domain adaptation for unsupervised optic disc and cup segmentation[J]. Neurocomputing, 2022, 469: 209-220.
[73] ZHAO Z, XU K, YEO H Z, et al. MS-MT: multi-scale mean teacher with contrastive unpaired translation for cross-modality vestibular schwannoma and cochlea segmentation[J]. arXiv:2303.15826, 2023.
[74] PARK T, EFROS A A, ZHANG R, et al. Contrastive learning for unpaired image-to-image translation[C]//Proceedings of the 16th European Conference on Computer Vision, Glasgow, August 23-28, 2020: 319-345.
[75] ZHAO Z, ZHOU F, XU K, et al. LE-UDA: label-efficient unsupervised domain adaptation for medical image segmentation[J]. arXiv:2212.02078, 2022.
[76] XU R, LUO Y, DU B. Unsupervised cross-domain pulmonary nodule detection without source data[J]. arXiv:2304. 01085, 2023.
[77] WEN Z, ZHANG X, YE C. Source-free domain adaptation for medical image segmentation via selectively updated mean teacher[C]//Proceedings of the International Conference on Information Processing in Medical Imaging. Cham: Springer Nature, 2023: 225-236.
[78] SHI A, FENG W. Triple consistency-based self-ensembling model for unsupervised domain adaption in medical image segmentation[C]//Proceedings of the International Joint Conference on Neural Networks (IJCNN), 2023: 1-7.
[79] QIN C, LI W, ZHENG B, et al. Dual adversarial models with cross-coordination consistency constraint for domain adaption in brain tumor segmentation[J]. Frontiers in Neuroscience, 2023, 17: 1043533.
[80] ZHAO Z, ZHOU F, ZENG Z, et al. Meta-hallucinator: towards few-shot cross-modality cardiac image segmentation[C]//Proceedings of the 25th International Conference on Medical Image Computing and Computer Assisted Intervention, September 18-22, 2022. Cham: Springer Nature, 2022: 128-139.
[81] WANG Y, CHENG J, CHEN Y, et al. FVP: Fourier visual prompting for source-free unsupervised domain adaptation of medical image segmentation[J]. arXiv:2304.13672, 2023.
[82] BIAN X, LUO X, WANG C, et al. DDA-Net: unsupervised cross-modality medical image segmentation via dual domain adaptation[J]. Computer Methods and Programs in Biomedicine, 2022, 213: 106531.
[83] BIAN C, YUAN C, MA K, et al. Domain adaptation meets zero-shot learning: an annotation-efficient approach to multi-modality medical image segmentation[J]. IEEE Transactions on Medical Imaging, 2021, 41(5):1043-1056.
[84] GU M, VESAL S, KOSTI R, et al. Few-shot unsupervised domain adaptation for multi-modal cardiac image segmentation[J].arXiv:2201.12386, 2022.
[85] LIU C, WANG L, LYU L, et al. DEJA VU: Continual model generalization for unseen domains[J]. arXiv:2301.10418, 2023.