计算机工程与应用 ›› 2024, Vol. 60 ›› Issue (11): 32-49.DOI: 10.3778/j.issn.1002-8331.2310-0335
崔珂,田启川,廉露
出版日期:
2024-06-01
发布日期:
2024-05-31
CUI Ke, TIAN Qichuan, LIAN Lu
Online:
2024-06-01
Published:
2024-05-31
摘要: U-Net简单高效的网络结构,被广泛应用于医学图像分割任务中,学者们针对U-Net结构进行了很多的研究和改进。基于U-Net网络结构的改进方法从以下方面进行归纳总结:总结了U-Net网络在医学图像分割领域的关键挑战;归纳了常用于U-Net网络的医学图像数据集格式及特点;重点总结U-Net和U-Net变体算法六大改进机制:跳跃连接机制、生成对抗网络、残差连接机制、3D-UNet、Transformer机制、密集连接机制。最后,探讨六大改进机制与常用医学数据之间的关系,并指出未来改进思路和方向,激发U-Net在医学图像分割的无限潜力。
崔珂, 田启川, 廉露. 基于U-Net变体的医学图像分割算法综述[J]. 计算机工程与应用, 2024, 60(11): 32-49.
CUI Ke, TIAN Qichuan, LIAN Lu. Review of Medical Image Segmentation Algorithms Based on U-Net Variants[J]. Computer Engineering and Applications, 2024, 60(11): 32-49.
[1] 呼伟, 徐巧枝, 葛湘巍, 等. 医学图像分割的无监督域适应研究综述[J]. 计算机工程与应用, 2024, 60(6): 10-26. HU W, XU Q Z, GE X W, et al. Review of unsupervised domain adaptation studies for medical image segmentation[J]. Computer Engineering and Applications, 2024, 60(6): 10-26. [2] JASWAL D, SOWMYA V, SOMAN K P . Image classification using convolutional neural networks[J]. International Journal of entific and Engineering Research, 2014, 5(6): 1661-1668. [3] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 2015. [4] LITJENS G, KOOI T, BEJNORDI B E, et al. A survey on deep learning in medical image analysis[J]. Medical Image Analysis, 2017, 42: 60-88. [5] 苏润. 基于U-Net框架的医学图像分割若干关键问题研究[D]. 合肥: 中国科学技术大学, 2021. SU R. Research on several key problems in medical image segmentation based on the U-Net framework[D]. Hefei: University of Science and Technology of China, 2021. [6] MILLETARI F, NAVAB N, AHMADI S A. V-Net: fully convolutional neural networks for volumetric medical image segmentation[C]//Proceedings of the Fourth International Conference on 3D Vision (3DV), Stanford, CA, USA, 2016. [7] WONG K C L, MORADI M, TANG H, et al. 3D segmentation with exponential logarithmic loss for highly unbalanced object sizes[C]//Proceedings of the International Conference on Medical Image Computing and Computer Assisted Intervention, 2018: 612-619. [8] JIANG Z, DING C, LIU M, et al. Two-stage cascaded U-Net: 1st place solution to BraTS challenge 2019 segmentation task[C]//Proceedings of the International MICCAI Brainlesion Workshop, 2020: 231-241. [9] DOLZ J, GOPINATH K, YUAN J, et al. HyperDense-Net: a hyper-densely connected cnn for multi-modal image segmen-tation[J]. IEEE Transactions on Medical Imaging, 2019, 38(5): 1116-1126. [10] BILIC P, CHRIST P, LI H B, et al. The liver tumor segmentation benchmark (LITS)[J]. Medical Image Analysis, 2023, 84: 102680. [11] BAKAS S, REYES M, BATTISTELLA E, et al. Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BraTS challenge[J]. arXiv:1811.02629, 2018. [12] AL-DHABYANI W, GOMAA M, KHALED H, et al. Dataset of breast ultrasound images[J]. Data in Brief, 2020, 28: 104863. [13] STAAL J, ABRAMOFF M D, NIEMEIJER M, et al. Ridge-based vessel segmentation in color images of the retina[J]. IEEE Transactions on Medical Imaging, 2004, 23(4): 501-509. [14] KUMAR N, VERMA R, SHARMA S, et al. A dataset and a technique for generalized nuclear segmentation for computational pathology[J]. IEEE Transactions on Medical Imaging, 2017, 36(7): 1550-1560. [15] MOREIRA I C, AMARAL I, DOMINGUES I, et al. INbreast: toward a full-field digital mammographic database[J]. Academic Radiology, 2012, 19(2): 236-248. [16] RONNEBERGER O, FISCHER P, BROX T. U-Net: convo- lutional networks for biomedical image segmentation[J]. arXiv:1505.04597, 2015. [17] DROZDZAL M, VORONTSOV E, CHARTRAND G, et al. The importance of skip connections in biomedical image segmentation[J]. arXiv:1608.04117, 2016. [18] HUANG H, LIN L, TONG R, et al. UNet3+: a full-scale connected unet for medical image segmentation[C]//Proceedings of the 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, 2020. [19] XIANG T, ZHANG C, LIU D, et al. BiO-Net: learning recurrent bi-directional connections for encoder-decoder architecture[C]//Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, 2020: 74-84. [20] LIU Y, WANG H, CHEN Z, et al. TransUNet+: redesigning the skip connection to enhance features in medical image segmentation[J]. Knowledge-Based Systems, 2022, 256: 109859. [21] IBTEHAZ N, RAHMAN M S. MultiResUNet: rethinking the U-Net architecture for multimodal biomedical image segme-ntation[J]. Neural Networks, 2020, 121: 74-87. [22] LI C, TAN Y, CHEN W, et al. Attention Unet++: a nested attention-aware U-Net for liver CT image segmentation[C]//Proceedings of the 2020 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates, 2020. [23] JIN Q G, MENG Z P, SUN C M, et al. RA-UNet: a hybrid deep attention-aware network to extract liver and tumor in CT scans[J]. arXiv:1811.01328, 2018. [24] NODIROV J, ABDUSALOMOV A B, WHANGBO T K. Attention 3D U-Net with multiple skip connections for segmentation of brain tumor images[J]. Sensors, 2022, 22(17): 6501. [25] CHEN Y, ZHANG J, ZONG X, et al. FSC-UNet: a lightweight medical image segmentation algorithm fused with skip connections[C]//Proceedings of the 14th International Conference on Digital Image Processing (ICDIP 2022), Wuhan, China, 2022. [26] LI H, FANG J, LIU S, et al. CR-Unet: a composite network for ovary and follicle segmentation in ultrasound images[J]. IEEE Journal of Biomedical and Health Informatics, 2020, 24(4): 974-983. [27] LACHINOV D, SEEB?CK P, MAI J, et al. Projective skip-connections for segmentation along a subset of dimensions in retinal OCT[C]//Proceedings of the 24th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2021), 2021: 431-441. [28] AZAD R, ASADI-AGHBOLAGHI M, FATHY M, et al. Bi-directional ConvLSTM U-Net with densley connected convolutions[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision Workshop, 2019. [29] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Advances in Neural Information Processing Systems, 2014: 2672-2680. [30] CHEN Y, JAKARY A, AVADIAPPAN S, et al. QSMGAN: improved quantitative susceptibility mapping using 3D generative adversarial networks with increased receptive field[J]. NeuroImage, 2020, 207: 116389. [31] WU C, ZOU Y, YANG Z. U-GAN: generative adversarial networks with U-Net for retinal vessel segmentation[C]//Proceedings of the 14th International Conference on Computer Science & Education (ICCSE), Toronto, ON, Canada, 2019. [32] SHEN T, GOU C, WANG F Y, et al. Learning from adversarial medical images for X-ray breast mass segmentation[J]. Computer Methods and Programs in Biomedicine, 2019, 180: 105012. [33] CAO X, YANG J, ZHANG J, et al. Deformable image registration based on similarity-steered CNN regression[C]//Proceedings of the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2017), 2017: 300-308. [34] LI G, ZHANG L, HU S, et al. Adversarial network with dual U-Net model and multiresolution loss computation for medical images registration[C]//Proceedings of the 12th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), Suzhou, China, 2019. [35] SUN Y, YUAN P, SUN Y. MM-GAN: 3D MRI data augmentation for medical image segmentation via generative adversarial networks[C]//Proceedings of the 2020 IEEE International Conference on Knowledge Graph (ICKG), Nanjing, China, 2020. [36] XU Z W, CAO Y K, JIN C, et al. GASNet: weakly-supervised framework for COVID-19 lesion segmentation[J]. arXiv:2010. 09456, 2020. [37] BEJI A, BLAIECH A G, SAID M, et al. An innovative medical image synthesis based on dual GAN deep neural networks for improved segmentation quality[J]. Applied Intelligence, 2023, 53(3): 3381-3397. [38] DAI W, LIU S, ENGSTROM CRAIG B, et al. Explainable semantic medical image segmentation with style[J]. arXiv:2303.0569, 2023. [39] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016. [40] DAS S, DEKA A, IWAHORI Y, et al. Contour-aware residual W-Net for nuclei segmentation[J]. Procedia Computer Science, 2019, 159: 1479-1488. [41] SHUVO Md B, AHOMMED R, REZA S, et al. CNL-UNet: a novel lightweight deep learning architecture for multimodal biomedical image segmentation with false output suppression[J]. Biomedical Signal Processing and Control, 2021, 70: 102959. [42] YUAN H, LIU Z, SHAO Y, et al. ResD-Unet research and application for pulmonary artery segmentation[J]. IEEE Access, 2021, 9: 67504-67511. [43] FANG Z, CHEN Y, NIE D, et al. RCA-U-Net: residual channel attention U-Net for fast tissue quantification in magnetic resonance fingerprinting[C]//Proceedings of the 22th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2019), 2019: 101-109. [44] RAHMAN A A, BISWAL B, PAPPU G P, et al. Robust segmentation of vascular network using deeply cascaded AReN-UNet[J]. Biomedical Signal Processing and Control, 2021, 69: 102953. [45] LIU Y, QI N, ZHU Q, et al. CR-U-Net: cascaded U-Net with residual mapping for liver segmentation in CT images*[C]//Proceedings of the 2019 IEEE Visual Communications and Image Processing (VCIP), Sydney, NSW, Australia, 2019. [46] WANG X, WANG Y. Composite attention residual U-Net for rib fracture detection[J]. Entropy, 2023, 25(3): 466. [47] BALDEON-CALISTO M, LAI-YUEN S K. AdaResU-Net: multiobjective adaptive convolutional neural network for medical image segmentation[J]. Neurocomputing, 2020, 392: 325-340. [48] ZULFIQAR M, STANUCH M, WODZINSKI M, et al. DRU-Net: pulmonary artery segmentation via dense residual U-Network with hybrid loss function[J]. Sensors, 2023, 23(12): 5427. [49] SHARMA A, MISHRA P K. DRI-UNet: dense residual-inception u-net for nuclei identification in microscopy cell images[J]. Neural Computing and Applications, 2023, 35: 19187-19220. [50] LI H, NAN Y, DEL SER J, YANG G. Large-kernel attention for 3D medical image segmentation[J]. arXiv:2207.11225, 2022. [51] FAN T, WANG G, LI Y, et al. MA-Net: a multi-scale attention network for liver and tumor segmentation[J]. IEEE Access, 2020, 8: 179656-179665. [52] SHAN T, YAN J, CUI X, et al. DSCA-Net: a depthwise separable convolutional neural network with attention mecha- nism for medical image segmentation[J]. Mathematical Biosciences and Engineering, 2023, 20: 365-382. [53] SHAKER A, MAAZ M, RASHEED H, et al. UNETR++: delving into efficient and accurate 3D medical image segmentation[J]. arXiv:2212.04497, 2022. [54] XIE Y, ZHANG J, LIAO Z, et al. PGL: prior-guided local self-supervised learning for 3D medical image segmentation[J]. arXiv:2011.12640, 2020. [55] SHIN H, KIM H, KIM S, et al. COSMOS: cross-modality unsupervised domain adaptation for 3D medical image segmentation based on target-aware domain translation and iterative self-training[J]. arXiv:2203.16557, 2022. [56] LIANG B, TANG C, ZHANG W, et al. N-Net: an UNet architecture with dual encoder for medical image segmentation[J]. Signal Image and Video Processing, 2023, 17(6): 3073-3081. [57] XU G, WU X, ZHANG X, et al. LeViT-UNet: make faster encoders with transformer for medical image segmentation[J]. arXiv:2107.08623, 2021. [58] HE Y, YANG D, ROTH H, et al. DiNTS: differentiable neural network topology search for 3D medical image segmentation[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 2021. [59] XIA T J, HUANG G H, PUN C M, et al. Multi-scale contextual semantic enhancement network for 3D medical image segmentation[J]. Physics in Medicine and Biology, 2022, 67(22): 225014. [60] GUO D, TERZOPOULOS D. A transformer-based network for anisotropic 3D medical image segmentation[C]//Proceedings of the 25th International Conference on Pattern Recognition (ICPR), Milan, Italy, 2021. [61] CHEN R, WANG X, JIN B, et al. CLD-Net: complement local detail for medical small-object segmentation[C]//Proceedings of the 2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Las Vegas, NV, USA, 2022. [62] 石磊, 籍庆余, 陈清威, 等. 视觉Transformer在医学图像分析中的应用研究综述[J]. 计算机工程与应用, 2023, 59(8): 41-55. SHI L, JI Q Y, CHEN Q W, et al. Review of research on application of vision Transformer in medical image analysis[J]. Computer Engineering and Applications, 2023, 59(8): 41-55. [63] CHEN J, LU Y, YU Q, et al. TransUNet: Transformers make strong encoders for medical image segmentation[J]. arXiv:2102.04306, 2021. [64] WANG W, CHEN C, DING M, et al. TransBTS: multimodal brain tumor segmentation using transformer[C]//Proceedings of the 24th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2021), 2021: 109-119. [65] LI X, FANG X, YANG G, et al. TransU2-Net: an effective medical image segmentation framework based on transformer and U2-Net[J]. IEEE Journal of Translational Engineering in Health and Medicine, 2023(1): 441-450. [66] WANG H, CAO P, WANG J, et al. UCTransNet: rethinking the skip connections in U-Net from a channel-wise perspective with transformer[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2022: 2441-2449. [67] LI Y, WANG S, WANG J, et al. GT U-Net: a U-Net like group transformer network for tooth root segmentation[C]// International Workshop on Machine Learning in Medical Imaging, 2021: 386-395. [68] XIE Y, ZHANG J, SHEN C, et al. CoTr: efficiently bridging CNN and transformer for 3D medical image segmentation[C]//Proceedings of the 24th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2021), 2021: 171-180. [69] HATAMIZADEH A, TANG Y, NATH V, et al. UNETR: Transformers for 3D medical image segmentation[C]//Proceedings of the 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, 2022. [70] CAO H, WANG Y, CHEN J, et al. Swin-UNet: UNet-like pure transformer for medical image segmentation[C]//Proceedings of the ECCV 2022 Workshops, 2022: 205-218. [71] HATAMIZADEH A, NATH V, TANG Y, et al. Swin UNETR: swin Transformers for semantic segmentation of brain tumors in MRI images[J]. arXiv:2201.01266, 2022. [72] WANG H, XIE S, LIN L, et al. Mixed transformer U-Net for medical image segmentation[C]//Proceedings of the 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022. [73] HUANG H, XIE1 S, LIN L, et al. ScaleFormer: revisiting the transformer-based backbones from a scale-wise perspective for medical image segmentation[C]//Proceedings of the 31st International Joint Conference on Artificial Intelligence, Vienna, Austria, 23-29 July, 2022: 964-971. [74] HATAMIZADEH A, XU Z, YANG D, et al. UNetFormer: a unified vision transformer model and pre-training framework for 3D medical image segmentation[J]. arXiv:2204.00631, 2022. [75] JIANG Y, ZHANG Y, LIN X, et al. SwinBTS: a method for 3D multimodal brain tumor segmentation using Swin Transformer[J]. Brain Sciences, 2022, 12(6): 797. [76] HEIDARI M, KAZEROUNI A, SOLTANY M, et al. HiFormer: hierarchical multi-scale representations using Transformers for medical image segmentation[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023: 6191-6201. [77] VALANARASU J M J, OZA P, HACIHALILOGLU I, et al. Medical transformer: gated axial-attention for medical image segmentation[C]//Proceedings of the 24th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2021), 2021: 36-46. [78] HUANG X, DENG Z, LI D, et al. MISSFormer: an effective medical image segmentation transformer[J]. arXiv:2109. 07162, 2021. [79] WU Y, LIAO K, CHEN J, et al. D-Former: a U-shaped dilated transformer for 3D medical image segmentation[J]. Neural Computing & Applications, 2023, 35(2): 1931-1944. [80] HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, 2017. [81] ZHANG Z, WU C, COLEMAN S, et al. DENSE-INception U-Net for medical image segmentation[J]. Computer Methods and Programs in Biomedicine, 2020, 192: 105395. [82] MENG C, SUN K, GUAN S, et al. Multiscale dense convolutional neural network for DSA cerebrovascular segmentation[J]. Neurocomputing, 2020, 373: 123-134. [83] QIANG Z, TU S, XU L. A K-Dense-UNet for biomedical image segmentation[C]//Proceedings of the IScIDE 2019, 2019: 552-562. [84] TRAN S T, CHENG C H, NGUYEN T T, et al. TMD-Unet: triple-unet with multi-scale input features and dense skip connection for medical image segmentation[J]. Healthcare, 2021 9(1): 54. [85] TRAN S T, NGUYEN T T, LE M H, et al. TDC-Unet: triple Unet with dilated convolution for medical image segmentation[J]. International Journal of Pharma Medicine and Biological Sciences, 2021, 11(1): 1-7. [86] MUBASHAR M, ALI H, GR?NLUND C, et al. R2U++: a multiscale recurrent residual U-Net with dense skip connections for medical image segmentation[J]. Neural Computing and Applications, 2022: 34(20): 17723-17739. [87] ZENG Z, FAN C, XIAO L, et al. DEA-UNet: a dense-edge-attention UNet architecture for medical image segmentation[J]. Journal of Electronic Imaging, 2022, 31(4): 043032. [88] CAI Z, XIN J, SHI P, et al. DSTUNet: UNet with efficient dense swin transformer pathway for medical image segmentation[C]//Proceedings of the 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), Kolkata, India, 2022. [89] YIN Y, XU W, CHEN L, et al. CoT-UNet++: a medical image segmentation method based on contextual transformer and dense connection[J]. Mathematical Biosciences and Engineering, 2023, 20(5): 8320-8336. [90] 杨鹤, 柏正尧. CoT-TransUNet: 轻量化的上下文Transformer医学图像分割网络[J]. 计算机工程与应用, 2023, 59(3): 218-225. YANG H, BAI Z Y. CoT-TransUNet: lightweight context transformer medical image segmentation network[J]. Computer Engineering and Applications, 2023, 59(3): 218-225. [91] ZHANG G, YANG Z, JIANG S. Automatic lung tumor segmentation from CT images using improved 3D densely connected UNet[J]. Medical & Biological Engineering & Computing, 2022, 60(11): 3311-3323. [92] WANG Z H, LIU Z, SONG Y Q, et al. Densely connected deep U-Net for abdominal multi-organ segmentation[C]//Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, China, 2019: 1415-1419. [93] KIRILLOV A, MINTUN E, RAVI N, et al. Segment anything[J]. arXiv:2304.02643, 2023. |
[1] | 车运龙, 袁亮, 孙丽慧. 基于强语义关键点采样的三维目标检测方法[J]. 计算机工程与应用, 2024, 60(9): 254-260. |
[2] | 邱云飞, 王宜帆. 双分支结构的多层级三维点云补全[J]. 计算机工程与应用, 2024, 60(9): 272-282. |
[3] | 叶彬, 朱兴帅, 姚康, 丁上上, 付威威. 面向桌面交互场景的双目深度测量方法[J]. 计算机工程与应用, 2024, 60(9): 283-291. |
[4] | 王彩玲, 闫晶晶, 张智栋. 基于多模态数据的人体行为识别方法研究综述[J]. 计算机工程与应用, 2024, 60(9): 1-18. |
[5] | 廉露, 田启川, 谭润, 张晓行. 基于神经网络的图像风格迁移研究进展[J]. 计算机工程与应用, 2024, 60(9): 30-47. |
[6] | 杨晨曦, 庄旭菲, 陈俊楠, 李衡. 基于深度学习的公交行驶轨迹预测研究综述[J]. 计算机工程与应用, 2024, 60(9): 65-78. |
[7] | 宋建平, 王毅, 孙开伟, 刘期烈. 结合双曲图注意力网络与标签信息的短文本分类方法[J]. 计算机工程与应用, 2024, 60(9): 188-195. |
[8] | 邓希泉, 陈刚. ConvUCaps:基于卷积胶囊网络的医学图像分割模型[J]. 计算机工程与应用, 2024, 60(8): 258-266. |
[9] | 周定威, 扈静, 张良锐, 段飞亚. 面向目标检测的数据集标签遗漏的协同修正技术[J]. 计算机工程与应用, 2024, 60(8): 267-273. |
[10] | 周伯俊, 陈峙宇. 基于深度元学习的小样本图像分类研究综述[J]. 计算机工程与应用, 2024, 60(8): 1-15. |
[11] | 孙石磊, 李明, 刘静, 马金刚, 陈天真. 深度学习在糖尿病视网膜病变分类领域的研究进展[J]. 计算机工程与应用, 2024, 60(8): 16-30. |
[12] | 汪维泰, 王晓强, 李雷孝, 陶乙豪, 林浩. 时空图神经网络在交通流预测研究中的构建与应用综述[J]. 计算机工程与应用, 2024, 60(8): 31-45. |
[13] | 谢威宇, 张强. 基于深度学习的图像中无人机与飞鸟检测研究综述[J]. 计算机工程与应用, 2024, 60(8): 46-55. |
[14] | 马冬梅, 李悦媛, 陈曦. 融合双重极化注意力的轻量化半监督语义分割[J]. 计算机工程与应用, 2024, 60(8): 225-233. |
[15] | 常禧龙, 梁琨, 李文涛. 深度学习优化器进展综述[J]. 计算机工程与应用, 2024, 60(7): 1-12. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||