Review of U-Net and Its Application in Segmentation of Liver and Liver Tumors

doi:10.3778/j.issn.1002-8331.2104-0408

Abstract

Abstract: It is of great clinical significance to realize automatic and precise segmentation of liver and liver tumor regions. With rapid developments of deep learning, deep neural networks have been gradually applied to medical fields, and computer-aided diagnosis has become a research hotspot. U-Net has been widely used in the fields of medical image segmentation because of its good performance in small sample datasets. Firstly, commonly used liver and liver tumor segmentation datasets and relevant evaluation criteria are introduced, the U-Net model and its improvements concerning encoder-decoder, skip-connection, and the overall structure are summarized. Then, the applications of U-Net in the segmentation of liver and liver tumors are analyzed from two aspects of single network structure and multi-network structure. Finally, the deficiencies of related works and the future development trends are discussed.

Key words: deep learning, U-Net, medical imaging processing, liver tumor segmentation

摘要： 医学上实现肝脏及肝脏肿瘤区域自动精准分割具有十分重要的临床意义，随着深度学习技术的迅速发展，深度神经网络逐步应用于医学领域，计算机辅助诊断成为研究热点。U-Net网络由于其在小样本数据集上的良好表现，在医学图像分割领域得到了广泛应用。基于此，介绍了肝脏和肝脏肿瘤分割中常用的数据集和评价指标，归纳了U-Net网络模型及围绕编解码器、跳跃连接和整体结构的改进。从单网络结构和多网络结构改进两个方面对U-Net模型在肝脏及肝脏肿瘤分割的相关应用加以论述。对相关研究工作的不足进行总结，并对未来发展予以展望。

关键词: 深度学习, U-Net, 医学图像处理, 肝脏肿瘤分割

ZHANG Huan, LIU Jing, FENG Yibo, QIU Dawei. Review of U-Net and Its Application in Segmentation of Liver and Liver Tumors[J]. Computer Engineering and Applications, 2022, 58(2): 1-14.

张欢, 刘静, 冯毅博, 仇大伟. U-Net及其在肝脏和肝脏肿瘤分割中的应用综述[J]. 计算机工程与应用, 2022, 58(2): 1-14.

References

[1] AUFFRAY C，BALLING R，BARROSO I，et al.Making sense of big data in health research：towards an EU action plan[J].Genome Medicine，2016，8（1）：1-13.
[2] AUSTIN C，KUSUMOTO F.The application of big data in medicine：current implications and future directions[J].Journal of Interventional Cardiac Electrophysiology，2016，47（1）：51-59.
[3] 施俊，汪琳琳，王珊珊，等.深度学习在医学影像中的应用综述[J].中国图象图形学报，2020，25（10）：1953-1981.
SHI J，WANG L L，WANG S S，et al.Applications of deep learning in medical imaging：a survey[J].Journal of Image and Graphics，2020，25（10）：1953-1981.
[4] 杨培伟，周余红，邢岗，等.卷积神经网络在生物医学图像上的应用进展[J].计算机工程与应用，2021，57（7）：44-58.
YANG P W，ZHOU Y H，XING G，et al.Applications of convolutional neural network in biomedical image[J].Computer Engineering and Applications，2021，57（7）：44-58.
[5] RONNEBERGER O，FISCHER P，BROX T.U-Net：convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer-assisted Intervention.Berlin：Springer，2015：234-241.
[6] BOSCH F X，RIBES J，DíAZ M，et al.Primary liver cancer：worldwide incidence and trends[J].Gastroenterology，2004，127（5）：5-16.
[7] BILIC P，CHRIST P F，VORONTSOV E，et al.The liver tumor segmentation benchmark（lits）[J].arXiv：1901.04056，2019.
[8] 乐美琰，魏千越，邓炜，等.基于电子计算机断层扫描图像的肝癌病灶自动分割方法研究进展[J].生物医学工程学杂志，2018，35（3）：481-487.
YUE M Y，WEI Q Y，DENG W，et al.A review of automatic liver tumor segmentation based on computed tomography[J].Journal of Biomedical Engineering，2018，35（3）：481-487.
[9] 马金林，邓媛媛，马自萍.肝脏肿瘤CT图像深度学习分割方法综述[J].中国图象图形学报，2020，25（10）：2024-2046.
MA J L，DENG Y Y，MA Z P.Review of deep learning segmentation methods for CT images of liver tumors[J].Journal of Image and Graphics，2020，25（10）：2024-2046.
[10] 郭雯，鞠忠建，吴青南，等.基于深度学习的器官自动分割研究进展[J].医疗卫生装备，2020，41（1）：85-94.
GUO W，JU Z J，WU Q N，et al.Research progress of automatic organ image segmentation based on deep learning[J].Chinese Medical Equipment Journal，2020，41（1）：85-94.
[11] XU P，CHEN C，WANG X，et al.ROI-based intraoperative MR-CT registration for image-guided multimode tumor ablation therapy in hepatic malignant tumors[J].IEEE Access，2020，8：13613-13619.
[12] SCHLEMPER J，OKTAY O，SCHAAP M，et al.Attention gated networks：learning to leverage salient regions in medical images[J].Medical Image Analysis，2019，53：197-207.
[13] HEIMANN T，VAN GINNEKEN B，STYNER M A，et al.Comparison and evaluation of methods for liver segmentation from CT datasets[J].IEEE Transactions on Medical Imaging，2009，28（8）：1251-1265.
[14] LONG J，SHELHAMER E，DARRELL T.Fully convolutional networks for semantic segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，39（4）：640-651.
[15] MILLETARI F，NAVAB N，AHMADI S A.V-net：fully convolutional neural networks for volumetric medical image segmentation[C]//2016 Fourth International Conference on 3D Vision（3DV）.Piscataway，NJ：IEEE，2016：565-571.
[16] IBTEHAZ N，RAHMAN M S.MultiResUNet：rethinking the U-Net architecture for multimodal biomedical image segmentation[J].Neural Networks，2020，121：74-87.
[17] ZHANG J，JIN Y，XU J，et al.MDU-Net：multi-scale densely connected U-Net for biomedical image segmentation[J].arXiv：1812.00352，2018.
[18] GUAN S，KHAN A A，SIKDAR S，et al.Fully dense UNet for 2-D sparse photoacoustic tomography artifact removal[J].IEEE Journal of Biomedical and Health Informatics，2019，24（2）：568-576.
[19] DOLZ J，AYED I B，DESROSIERS C.Dense multi-path U-Net for ischemic stroke lesion segmentation in multiple image modalities[C]//International MICCAI Brainlesion Workshop.Berlin：Springer，2018：271-282.
[20] OKTAY O，SCHLEMPER J，FOLGOC L L，et al.Attention U-Net：learning where to look for the pancreas[J].arXiv：1804.03999，2018.
[21] LI C，TAN Y，CHEN W，et al.ANU-Net：attention-based nested U-net to exploit full resolution features for medical image segmentation[J].Computers & Graphics，2020，90：11-20.
[22] 彭璟，罗浩宇，赵淦森，等.深度学习下的医学影像分割算法综述[J].计算机工程与应用，2021，57（3）：44-57.
PENG J，LUO H Y，ZHAO G S，et al.Survey of medical image segmentation algorithm in deep learning[J].Computer Engineering and Applications，2021，57（3）：44-57.
[23] SONG L，TSO G K F，HE K J.Bottleneck feature supervised U-Net for pixel-wise liver and tumor segmentation[J].Expert Systems with Applications，2020，145：113131.
[24] ZHANG Z，WU C D，COLEMAN S，et al.DENSE-Inception U-net for medical image segmentation[J].Computer Methods and Programs in Biomedicine，2020，192：105395.
[25] JIN Q，MENG Z，PHAM T D，et al.DUNet：a deformable network for retinal vessel segmentation[J].Knowledge-Based Systems，2019，178：149-162.
[26] CHEN C，LIU X，DING M，et al.3D dilated multi-fiber network for real-time brain tumor segmentation in MRI[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention.Berlin：Springer，2019：184-192.
[27] ALOM M Z，YAKOPCIC C，HASAN M，et al.Recurrent residual U-Net for medical image segmentation[J].Journal of Medical Imaging，2019，6（1）：014006.
[28] WANG W，CHEN J，ZHAO J，et al.Automated segmentation of pulmonary lobes using coordination-guided deep neural networks[C]//2019 IEEE 16th International Symposium on Biomedical Imaging（ISBI 2019）.Piscataway，NJ：IEEE，2019：1353-1357.
[29] CHEN W，ZHANG Y，HE J，et al.Prostate segmentation using 2d bridged U-net[C]//2019 International Joint Conference on Neural Networks（IJCNN）.Piscataway，NJ：IEEE，2019：1-7.
[30] ZHUANG J.Laddernet：multi-path networks based on U-net for medical image segmentation[J].arXiv：1810.07810，2018.
[31] ZHOU Z，SIDDIQUEE M M R，TAJBAKHSH N，et al.Unet++：a nested U-Net architecture for medical image segmentation[M]//Deep learning in medical image analysis and multimodal learning for clinical decision support.Berlin：Springer，2018：3-11.
[32] HUANG H，LIN L，TONG R，et al.Unet 3+：a full-scale connected unet for medical image segmentation[C]//ICASSP 2020-2020 IEEE International Conference on Acoustics，Speech and Signal Processing（ICASSP）.Piscataway，NJ：IEEE，2020：1055-1059.
[33] FU J，LIU J，TIAN H，et al.Dual attention network for scene segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：3146-3154.
[34] 谷鹏辉，肖志勇.改进的U-Net在视网膜血管分割上的应用[J/OL].计算机科学与探索：1-12[2021-07-20].http：//kns.cnki.net/kcms/detail/11.5602.TP.20210128.1041.008.html.
GU P H，XIAO Z Y.Application of improved U-Net in retinal vessel segmentation[J/OL].Journal of Frontiers of Computer Science and Technology：1-12[2021-07-20].http：//kns.cnki.net/kcms/detail/11.5602.TP.20210128.1041.
008.html.
[35] JHA D，RIEGLER M A，JOHANSEN D，et al.Doubleu-net：a deep convolutional neural network for medical image segmentation[C]//2020 IEEE 33rd International Symposium on Computer-Based Medical Systems（CBMS）.Piscataway，NJ：IEEE，2020：558-564.
[36] VALLOLI V K，MEHTA K.W-net：reinforced U-net for density map estimation[J].arXiv：1903.11249，2019.
[37] FU H，CHENG J，XU Y，et al.Joint optic disc and cup segmentation based on multi-label deep network and polar transformation[J].IEEE Transactions on Medical Imaging，2018，37（7）：1597-1605.
[38] ?I?EK ?，ABDULKADIR A，LIENKAMP S S，et al.3D U-Net：learning dense volumetric segmentation from sparse annotation[C]//International Conference on Medical Image Computing and Computer-assisted Intervention.Berlin：Springer，2016：424-432.
[39] 孙明建，徐军，马伟，等.基于新型深度全卷积网络的肝脏CT影像三维区域自动分割[J].中国生物医学工程学报，2018，37（4）：385-393.
SUN M J，XU J，MA W，et al.A new fully convolutional network for 3D liver region segmentation on CT images[J].Chinese Journal of Biomedical Engineering，2018，37（4）：385-393.
[40] XING Y，WANG J，CHEN X，et al.2.5 D convolution for RGB-D semantic segmentation[C]//2019 IEEE International Conference on Image Processing（ICIP）.Piscataway，NJ：IEEE，2019：1410-1414.
[41] HAN X.Automatic liver lesion segmentation using a deep convolutional neural network method[J].arXiv：1704.07239，2017.
[42] HAN L，CHEN Y，LI J，et al.Liver segmentation with 2.5 D perpendicular UNets[J].Computers & Electrical Engineering，2021，91：107118.
[43] JIN Q，MENG Z，SUN C，et al.RA-UNet：a hybrid deep attention-aware network to extract liver and tumor in CT scans[J].Frontiers in Bioengineering and Biotechnology，2020，8：1471.
[44] LEI T，ZHOU W，ZHANG Y，et al.Lightweight V-Net for liver segmentation[C]//ICASSP 2020-2020 IEEE International Conference on Acoustics，Speech and Signal Processing（ICASSP）.Piscataway，NJ：IEEE，2020：1379-1383.
[45] 李华，杨嘉能，刘凤，等.基于深度学习的乳腺癌病理图像分类研究综述[J].计算机工程与应用，2020，56（13）：1-11.
LI H，YANG J N，LIU F，et al.Survey of breast cancer histopathology image classification based on deep learning[J].Computer Engineering and Applications，2020，56（13）：1-11.
[46] CHRIST P F，ELSHAER M E A，ETTLINGER F，et al.Automatic liver and lesion segmentation in CT using cascaded fully convolutional neural networks and 3D conditional random fields[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention.Berlin：Springer，2016：415-423.
[47] VORONTSOV E，TANG A，PAL C，et al.Liver lesion segmentation informed by joint liver segmentation[C]//2018 IEEE 15th International Symposium on Biomedical Imaging（ISBI 2018）.Piscataway，NJ：IEEE，2018：1332-1335.
[48] ISENSEE F，PETERSEN J，KOHL S A A，et al.nnU-Net：breaking the spell on successful medical image segmentation[J].arXiv：1904.08128，2019.
[49] 刘云鹏，刘光品，王仁芳，等.深度学习结合影像组学的肝脏肿瘤CT分割[J].中国图象图形学报，2020，25（10）：2128-2141.
LIU Y P，LIU G P，WANG R F，et al.Accurate segmentation method of liver tumor CT based on the combination of deep learning and radiomics[J].Journal of Image and Graphics，2020，25（10）：2128-2141.
[50] XU C，HU D，ZHANG Y D，et al.Study on the segmentation method of multi-phase CT liver tumor based on dual channel U-Nets[J].Journal of Physics：Conference Series，2021，1828（1）：012043.
[51] 黄泳嘉，史再峰，王仲琦，等.基于混合损失函数的改进型U-Net肝部医学影像分割方法[J].激光与光电子学进展，2020，57（22）：66-75.
HUANG Y J，SHI Z F，WANG Z Q，et al.Improved U-Net based on mixed loss function for liver medical image segmentation[J].Laser & Optoelectronics Progress，2020，57（22）：66-75.
[52] LI X，CHEN H，QI X，et al.H-DenseUNet：hybrid densely connected UNet for liver and tumor segmentation from CT volumes[J].IEEE Transactions on Medical Imaging，2018，37（12）：2663-2674.
[53] ZHANG Y，JIANG B，WU J，et al.Deep learning initialized and gradient enhanced level-set based segmentation for liver tumor from CT images[J].IEEE Access，2020，8：76056-76068.
[54] 田宝园，程怿，蔡叶华，等.基于改进U-Net深度网络的超声正中神经图像分割[J].自动化仪表，2020，41（8）：36-41.
TIAN B Y，CHENG Y，CAI Y H，et al.Ultrasound image segmentation for median nerves based on improved U-Net deep network[J].Process Automation Instrumentation，2020，41（8）：36-41.
[55] SHORTEN C，KHOSHGOFTAAR T M.A survey on image data augmentation for deep learning[J].Journal of Big Data，2019，6（1）：1-48.
[56] 牟海维，郭颖，全星慧，等.一种基于改进U-Net脑肿瘤MRI图像分割方法[J/OL].激光与光电子学进展：1-13[2021-07-27].http：//kns.cnki.net/kcms/detail/31.1690.TN.20200910.
0915.008.html.
MOU H W，GUO Y，QUAN X H，et al.A brain tumor MRI image segmentation method based on improved U-Net[J/OL].Laser & Optoelectronics Progress：1-13[2021-07-27].http：//kns.cnki.net/kcms/detail/31.1690.TN.20200910.0915.
008.html.