注意力机制在视网膜血管分割中的应用综述

doi:10.3778/j.issn.1002-8331.2311-0049

摘要/Abstract

摘要： 视网膜血管的自动分割在眼科和心血管疾病的计算机辅助诊断中发挥着重要作用。注意力机制能够提高经典神经网络模型对图像特征提取的效率和精度，因此注意力机制在视网膜血管分割模型中广泛使用。首先回顾了视网膜血管分割的常用数据集及评价指标，接着根据工作机理将注意力分为选择性注意力机制和自注意力机制两类；根据计算机视觉任务中的作用域将注意力方法分为通道注意力、空间注意力以及混合注意力三类，结合视网膜血管分割任务重点介绍了以上三类方法的代表性注意力模型的具体应用，并对相关模型进行性能对比和评价。最后，对注意力机制存在的问题以及未来的发展趋势进行了讨论。

关键词: 视网膜血管分割, 神经网络, 注意力机制, 计算机视觉

Abstract: Automatic segmentation of retinal vessels plays an important role in computer-aided diagnosis of ophthalmology and cardiovascular diseases. Attention mechanism can improve the efficiency and accuracy of image feature extraction in classical neural network models, so attention mechanism is widely used in retinal vessel segmentation models. This paper firstly reviews the commonly used datasets and evaluation metrics for retinal vessel segmentation, subsequently, attention mechanisms are categorized into two types: selective attention mechanisms and self-attention mechanisms, based on their working principles. Meanwhile, according to the data domain of computer vision tasks, attention methods are divided into three categories: channel attention, spatial attention, and mixed attention. Combined with the task of retinal vessel segmentation, the paper highlights the specific applications of representative attention models of these three types and conducts performance comparisons and evaluations of relevant models. Finally, the problems of attention mechanism and the development trend in the future are discussed.

Key words: retinal vessel segmentation, neural network, attention mechanism, computer vision

裴峻鹏, 汪有崧, 李增辉, 王伟. 注意力机制在视网膜血管分割中的应用综述[J]. 计算机工程与应用, 2024, 60(14): 50-65.

PEI Junpeng, WANG Yousong, LI Zenghui, WANG Wei. Comprehensive Review on Application of Attention Mechanism in Retinal Vessel Segmentation[J]. Computer Engineering and Applications, 2024, 60(14): 50-65.

参考文献

[1] MACGILLIVRAY T J, TRUCCO E, CAMERON J R, et al. Retinal imaging as a source of biomarkers for diagnosis, characterization and prognosis of chronic illness or long-term conditions[J]. British Journal of Radiology, 2014, 87: 20130832.
[2] VOSTATEK P, CLARIDGE E, UUSITALO H, et al. Performance comparison of publicly available retinal blood vessel segmentation methods[J]. Computerized Medical Imaging and Graphics, 2017, 55: 2-12.
[3] LI T, BO W, HU C, et al. Applications of deep learning in fundus images: a review[J]. Medical Image Analysis, 2021, 69: 101971.
[4] CHEN X X, WANG X M, ZHANG K, et al. Recent advances and clinical applications of deep learning in medical image analysis[J]. Medical Image Analysis, 2022, 79: 102444.
[5] KHANDOUZI A, ARIAFAR A, MASHAYEKHPOUR Z, et al. Retinal vessel segmentation, a review of classic and deep methods[J]. Annals of Biomedical Engineering, 2022, 50(10): 1292-1314.
[6] 李兰兰, 张孝辉, 牛得草, 等. 深度学习在视网膜血管分割上的研究进展[J]. 计算机科学与探索, 2021, 15(11): 2063-2076.
LI L L, ZHANG X H, NIU D C, et al. Research progress of deep learning in retinal vessel segmentation[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(11): 2063-2076.
[7] CORBETTA M, SHULMAN G L. Control of goal-directed and stimulus-driven attention in the brain[J]. Nature Reviews Neuroscience, 2002, 3(3): 201-215.
[8] DE SANTANA CORREIA A, COLOMBINI E L. Attention, please! a survey of neural attention models in deep learning[J]. Artificial Intelligence Review, 2022, 55(8): 6037-6124.
[9] YANG X. An overview of the attention mechanisms in computer vision[J]. Journal of Physics: Conference Series, 2020, 1693(1): 012173.
[10] LV H, CHEN J, PAN T, et al. Attention mechanism in intelligent fault diagnosis of machinery: a review of technique and application[J]. Measurement, 2022, 199: 111594.
[11] BADAR M, HARIS M, FATIMA A. Application of deep learning for retinal image analysis: a review[J]. Computer Science Review, 2020, 35: 100203.
[12] STAAL J, ABRAMOFF M D, NIEMEIJER M, et al. Ridge-based vessel segmentation in color images of the retina[J]. IEEE Transactions on Medical Imaging, 2004, 23(4): 501-509.
[13] HOOVER A. Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response[J]. IEEE Transactions on Medical Imaging, 2000, 19(3): 203-210.
[14] OWEN C G, RUDNICKA A R, MULLEN R, et al. Measuring retinal vessel tortuosity in 10-year-old children: validation of the computer-assisted image analysis of the retina (CAIAR) program[J]. Investigative Ophthalmology & Visual Science, 2009, 50(5): 2004-2010.
[15] BUDAI A, BOCK R, MAIER A, et al. Robust vessel segmentation in fundus images[J]. International Journal of Biomedical Imaging, 2013, 154860: 1-11.
[16] ZHANG J, DASHTBOZORG B, BEKKERS E, et al. Robust retinal vessel segmentation via locally adaptive derivative frames in orientation scores[J]. IEEE Transactions on Medical Imaging, 2016, 35(12): 2631-2644.
[17] CERVANTES J, CERVANTES J, GARCíA-LAMONT F, et al. A comprehensive survey on segmentation techniques for retinal vessel segmentation[J]. Neurocomputing, 2023, 556: 126626.
[18] SOYDANER D. Attention mechanism in neural networks: where it comes and where it goes[J]. Neural Computing and Applications, 2022, 34(16): 13371-13385.
[19] NIU Z, ZHONG G, YU H. A review on the attention mechanism of deep learning[J]. Neurocomputing, 2021, 452: 48-62.
[20] CHAUDHARI S, MITHAL V, POLATKAN G, et al. An attentive survey of attention models[J]. ACM Trans on Intell Syst Technol, 2021, 12(5): 1-32.
[21] 皇甫晓瑛, 钱惠敏, 黄敏. 结合注意力机制的深度神经网络综述[J]. 计算机与现代化, 2023(2): 40-49.
HUANGFU X Y, QIAN H M, HUANG M. A review of deep neural networks combined with attention mechanism[J]. Computer and Modernization, 2023(2): 40-49.
[22] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st Annual Conference on Neural Information Processing Systems, 2017: 5999-6009.
[23] WANG X, GIRSHICK R, GUPTA A, et al. Non-Local neural networks[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 7794-7803.
[24] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: transformers for image recognition at scale[J]. arXiv:2010.11929, 2020.
[25] GUO M H, XU T X, LIU J J, et al. Attention mechanisms in computer vision: a survey[J]. Computational Visual Media, 2022, 8(3): 331-368.
[26] HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023.
[27] GUO F, LI W, KUANG Z, et al. MES-Net: a new network for retinal image segmentation[J]. Multimedia Tools and Applications, 2021, 80(10): 14767-14788.
[28] LI D, RAHARDJA S. BSEResU-Net: an attention-based before-activation residual U-Net for retinal vessel segmentation[J]. Computer Methods and Programs in Biomedicine, 2021, 205: 106070.
[29] LIU Y, SHEN J, YANG L, et al. ResDO-UNet: a deep residual network for accurate retinal vessel segmentation from fundus images[J]. Biomedical Signal Processing and Control, 2023, 79: 104087.
[30] LI X, DING J, TANG J, et al. Res2UNet: a multi-scale channel attention network for retinal vessel segmentation[J]. Neural Computing and Applications, 2022, 34(14): 12001-12015.
[31] LI X, WANG W, HU X, et al. Selective kernel networks [C]//Proceedings of the 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, June 16 - 20, 2019: 510-519.
[32] HUANG W, GUO F, YAN Y. Retinal vessel segmentation algorithm based on attention mechanism[C]//Proceedings of the IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress, 2022: 1-5.
[33] HE Y, SUN H, YI Y, et al. Curv-Net: curvilinear structure segmentation network based on selective kernel and multi-Bi-ConvLSTM[J]. Medical Physics, 2022, 49(5): 3144-3158.
[34] WANG Q, WU B, ZHU P, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020: 11531-11539.
[35] WANG X, YU P, LI H, et al. U-Net fundus retinal vessel segmentation method based on multi-scale feature fusion [C]//Proceedings of the 2022 IEEE 10th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), 2022: 28-33.
[36] YANG D, ZHAO H, YU K, et al. NAUNet: lightweight retinal vessel segmentation network with nested connections and efficient attention[J]. Multimedia Tools and Applications, 2023, 82(16): 25357-25379.
[37] ZHAO R, LI Q, WU J, et al. A nested U-shape network with multi-scale upsample attention for robust retinal vascular segmentation[J]. Pattern Recognition, 2021, 120: 107998.
[38] NI J, SUN H, XU J, et al. A feature aggregation and feature fusion network for retinal vessel segmentation[J]. Biomedical Signal Processing and Control, 2023, 85: 104829.
[39] DONG F, WU D, GUO C, et al. CRAUNet: a cascaded residual attention U-Net for retinal vessel segmentation[J]. Computers in Biology and Medicine, 2022, 147: 105651.
[40] GUO C, SZEMENYEI M, HU Y, et al. Channel attention residual U-Net for retinal vessel segmentation[C]//Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Toronto, ON, Canada, June 6-11, 2021.
[41] HAN J, WANG Y, GONG H. Fundus retinal vessels image segmentation method based on improved U-Net[J]. IRBM, 2022, 43(6): 628-639.
[42] SCHLEMPER J, OKTAY O, SCHAAP M, et al. Attention gated networks: learning to leverage salient regions in medical images[J]. Medical Image Analysis, 2019, 53: 197-207.
[43] WANG D Y, HAYTHAM A, POTTENBURGH J, et al. Hard attention net for automatic retinal vessel segmentation[J]. IEEE Journal of Biomedical and Health Informatics, 2020, 24(12): 3384-3396.
[44] DU X F, WANG J S, SUN W Z. UNet retinal blood vessel segmentation algorithm based on improved pyramid pooling method and attention mechanism[J]. Physics in Medicine & Biology, 2021, 66(17): 175013.
[45] LI J, GAO G, YANG L, et al. GDF-Net: a multi-task symmetrical network for retinal vessel segmentation[J]. Biomedical Signal Processing and Control, 2023, 81: 104426.
[46] WU C, ZOU Y, ZHAN J. DA-U-Net: densely connected convolutional networks and decoder with attention gate for retinal vessel segmentation[J]. IOP Conference Series: Materials Science and Engineering, 2019, 533(1): 012053.
[47] ZHANG H, NI W, LUO Y, et al. TUnet-LBF: retinal fundus image fine segmentation model based on transformer Unet network and LBF[J]. Computers in Biology and Medicine, 2023, 159: 106937.
[48] LIU X, ZHANG D, YAO J, et al. Transformer and convolutional based dual branch network for retinal vessel segmentation in OCTA images[J]. Biomedical Signal Processing and Control, 2023, 83: 104604.
[49] DU L, LIU H, ZHANG L, et al. Deep ensemble learning for accurate retinal vessel segmentation[J]. Comput Biol Med, 2023, 158: 106829.
[50] LANG J, LIU Y. LCCF-Net: lightweight contextual and channel fusion network for medical image segmentation[J]. Biomedical Signal Processing and Control, 2023, 86: 105134.
[51] SHI Y, WANG W, YUAN M, et al. Self-paced dual-axis attention fusion network for retinal vessel segmentation[J]. Electronics, 2023, 12(9).
[52] WANG B, WANG S, QIU S, et al. CSU-Net: a context spatial U-Net for accurate blood vessel segmentation in fundus images[J]. IEEE J Biomed Health Inform, 2021, 25(4): 1128-1138.
[53] SU H, GAO L, LU Y, et al. Attention-guided cascaded network with pixel-importance-balance loss for retinal vessel segmentation[J]. Front Cell Dev Biol, 2023, 11: 1196191.
[54] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[J]. arXiv:1807.06521, 2018.
[55] LIU S, GUAN J, HE Y, et al. RCU-Net: a novel network for retinal vessel segmentation[C]//Proceedings of the 2nd International Conference on Advanced Algorithms and Signal Image Processing (AASIP 2022), 2022.
[56] CAI D, FU Y, WANG T, et al. CFCNet: a coarse-to-fine cascade network for retinal vessel segmentation[C]//Proceedings of the 2022 China Automation Congress (CAC), 2022: 2728-2733.
[57] DEARI S, OKSUZ I, ULUKAYA S. Block attention and switchable normalization based deep learning framework for segmentation of retinal vessels[J]. IEEE Access, 2023, 11: 38263-38274.
[58] ALIMANOV A, ISLAM M B. Retinal image restoration and vessel segmentation using modified cycle-CBAM and CBAM-UNet[C]//Proceedings of the 2022 Innovations in Intelligent Systems and Applications Conference (ASYU), 2022: 1-6.
[59] ROY A G, NAVAB N, WACHINGER C. Concurrent spatial and channel “squeeze & excitation” in fully convolutional networks[C]//Proceedings of the 21st International Conference on Medical Image Computing and Computer Assisted Intervention, Granada, Spain, September 16- 20, 2018.
[60] WANG H, XU G, PAN X, et al. Attention-inception-based U-Net for retinal vessel segmentation with advanced residual[J]. Computers & Electrical Engineering, 2022, 98: 107670.
[61] FU J, LIU J, TIAN H, et al. Dual attention network for scene segmentation[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019: 3141-3149.
[62] LIU R, WANG T, ZHANG X, et al. DA-Res2UNet: explainable blood vessel segmentation from fundus images[J]. Alexandria Engineering Journal, 2023, 68: 539-549.
[63] SUN K, CHEN Y, CHAO Y, et al. A retinal vessel segmentation method based improved U-Net model[J]. Biomedical Signal Processing and Control, 2023, 82: 104574.
[64] YUAN Y, ZHANG L, WANG L, et al. Multi-level attention network for retinal vessel segmentation[J]. IEEE J Biomed Health Inform, 2022, 26(1): 312-323.
[65] WANG G, HUANG Y, MA K, et al. Automatic vessel crossing and bifurcation detection based on multi-attention network vessel segmentation and directed graph search[J]. Computers in Biology and Medicine, 2023, 155: 106647.
[66] HU X, WANG L, CHENG S, et al. HDC-Net: a hierarchical dilation convolutional network for retinal vessel segmentation[J]. PLOS One, 2021, 16(9): e0257013.
[67] ZHU C, NIU X, ZUO L, et al. Fused residual attention dense double-U network retinal vessel segmentation algorithm[C]//Proceedings of the 2023 International Conference on Intelligent Supercomputing and BioPharma (ISBP), 2023: 109-115.
[68] JIANG Y, QI S, MENG J, et al. SS-Net: split and spatial attention network for vessel segmentation of retinal OCT angiography[J]. Appl Opt, 2022, 61(9): 2357-2363.
[69] LI K, QI X, LUO Y, et al. Accurate retinal vessel segmentation in color fundus images via fully attention-based networks[J]. IEEE J Biomed Health Inform, 2021, 25(6): 2071-2081.
[70] LI Y, ZHANG Y, LIU J Y, et al. Global transformer and dual local attention network via deep-shallow hierarchical feature fusion for retinal vessel segmentation[J]. IEEE Trans Cybern, 2023, 53(9): 5826-5839.
[71] SHEN X, XU J, JIA H, et al. Self-attentional microvessel segmentation via squeeze-excitation transformer U-Net[J]. Computerized Medical Imaging and Graphics, 2022, 97: 102055.
[72] LI Y, YANG J, NI J, et al. TA-Net: triple attention network for medical image segmentation[J]. Comput Biol Med, 2021, 137: 104836.
[73] LIU M, WANG Z, LI H, et al. AA-WGAN: attention augmented Wasserstein generative adversarial network with application to fundus retinal vessel segmentation[J]. Comput Biol Med, 2023, 158: 106874.
[74] ZHANG Y, LUO L, DOU Q, et al. Triplet attention and dual-pool contrastive learning for clinic-driven multi-label medical image classification[J]. Medical Image Analysis, 2023, 86: 102772.
[75] HUANG Z, WANG X, WEI Y, et al. CCNet: criss-cross attention for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(6): 6896-6908.
[76] LAI Q, KHAN S, NIE Y, et al. Understanding more about human and machine attention in deep neural networks[J]. IEEE Transactions on Multimedia, 2021, 23: 2086-2099.
[77] WANG W G, SHEN J B, LU X K, et al. Paying attention to video object pattern understanding[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(7): 2413-2428.
[78] BRAUWERS G, FRASINCAR F. A general survey on attention mechanisms in deep learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2023, 35(4): 3279-3298.
[79] HE X, DENG Y, FANG L, et al. Multi-modal retinal image classification with modality?specific attention network[J]. IEEE Transactions on Medical Imaging, 2021, 40(6): 1591-1602.
[80] CHEN S, BORTSOVA G, GARCíA-UCEDA JUáREZ A, et al. Multi-task attention-based semi-supervised learning for medical image segmentation[C]//Proceedings of the 22nd International Conference on Medical Image Computing and Computer Assisted Intervention, 2019: 457-465.
[81] MA Z, LI X. An improved supervised and attention mechanism-based U-Net algorithm for retinal vessel segmentation[J]. Computers in Biology and Medicine, 2024, 168: 107770.
[82] QIN D A, BU J J, LIU Z, et al. Efficient medical image segmentation based on knowledge distillation[J]. IEEE Transactions on Medical Imaging, 2021, 40(12): 3820-3831.
[83] WANG L, YOON K J. Knowledge distillation and student-teacher learning for visual intelligence: a review and new outlooks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(6): 3048-3068.
[84] ALI S, ABUHMED T, EL-SAPPAGH S, et al. Explainable artificial intelligence (XAI): what we know and what is left to attain trustworthy artificial intelligence[J]. Information Fusion, 2023, 99: 101805.
[85] PATRíCIO C, NEVES J C, TEIXEIRA L F. Explainable deep learning methods in medical image classification: a survey[J]. ACM Comput Surv, 2023, 56(4): 1-41.