Review of Application of Deep Learning in Detection of Diabetic Retinal Lesions

doi:10.3778/j.issn.1002-8331.2104-0345

Abstract

Abstract:

Diabetic retinopathy is the eye disease with the highest blindness rate in the world, and early diagnosis can significantly reduce the probability of blindness in patients. Deep learning methods can extract hidden features of medical images and complete image detection tasks. Therefore, the application of deep learning to detect diabetic retinal lesions has become a research hotspot. This article mainly summarizes five aspects：data set introduction, fully-supervised detection methods, non-fully-supervised detection methods, handling of small sample problems and model interpretability. It focuses on sorting out the basic ideas of various methods, network structure forms, improvement plans and summaries of advantages and disadvantages, etc. Finally, combined with the challenges faced by current detection methods, the future research directions are analyzed.

Key words: deep learning, diabetic retinopathy, convolution neural network, lesion area detection

摘要：

糖尿病视网膜病变是世界上致盲率最高的眼科疾病，早期诊断可以显著降低患者失明的概率。深度学习方法可以提取医学图像的隐含特征，并完成图像的检测任务，因此应用深度学习实现糖尿病视网膜病灶检测成为研究热点。主要从数据集介绍、全监督检测方法、非完全监督检测方法、小样本问题的处理和模型可解释性五个方面进行详细总结，重点整理各类方法的基本思想、网络结构形式、改进方案及优缺点总结等内容，结合当前检测方法所面临的挑战，对其未来研究方向进行展望。

关键词: 深度学习, 糖尿病视网膜病变, 卷积神经网络, 病灶区域检测

NIE Yongqi, CAO Hui, YANG Feng, LIU Jing. Review of Application of Deep Learning in Detection of Diabetic Retinal Lesions[J]. Computer Engineering and Applications, 2021, 57(20): 25-41.

聂永琦，曹慧，杨锋，刘静. 深度学习在糖尿病视网膜病灶检测中的应用综述[J]. 计算机工程与应用, 2021, 57(20): 25-41.

References

[1] SADDA S R，NITTALA M G，TAWEEBANJONGSIN W，et al.Quantitative assessment of the severity of diabetic retinopathy[J].American Journal of Ophthalmology，2020，218：342-352.
[2] SCHWARTZ R，KHALID M H，FRCOPHTH S S，et al.Objective evaluation of proliferative diabetic retinopathy using OCT[J].Ophthalmology Retina，2020，4（2）：164-174.
[3] KUMAR S，ADARSH A，KUMAR B，et al.An automated early diabetic retinopathy detection through improved blood vessel and optic disc segmentation[J].Optics & Laser Technology，2020，121：105815.
[4] PAN X J，JIN K，CAO J，et al.Multi-label classification of retinal lesions in diabetic retinopathy for automatic analysis of fundus fluorescein angiography based on deep learning[J].Graefe’s Archive for Clinical and Experimental Ophthalmology，2020，258（4）：779-785.
[5] BELLEMO V，LIM G，RIM T H，et al.Artificial intelligence screening for diabetic retinopathy：the real-world emerging application[J].Current Diabetes Reports，2019，19（9）：72.
[6] 张福玲，张少敏.应用于CT图像肺结节检测的深度学习方法综述[J].计算机工程与应用，2020，56（13）：20-32.
ZHANG F L，ZHANG S M.Review of deep learning methods applied to lung nodule detection in CT images[J].Computer Engineering and Applications，2020，56（13）：20-32.
[7] 梁芳烜，杨锋，卢丽云，等.基于卷积神经网络的脑肿瘤分割方法综述[J].计算机工程与应用，2021，57（7）：34-43.
LIANG F X，YANG F，LU L Y，et al.Review of brain tumor segmentation methods based on convolutional neural networks[J].Computer Engineering and Applications，2021，57（7）：34-43.
[8] ASIRI N，HUSSAIN M，ADEL F A，et al.Deep learning based computer-aided diagnosis systems for diabetic retinopathy：a survey[J].Artificial Intelligence in Medicine，2019，99：1-20.
[9] KRIZHEVSKY A，SUTSKEVER I，GEOFFREY E.ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems，2012：1097-1105.
[10] STAAL J，ABRAMOFF M D，NIEMEIJER M，et al.Ridge-based vessel segmentation in color images of the retina[J].IEEE Transactions on Medical Imaging，2004，23（4）：501-509.
[11] HOOVER A D，VALENTINA K，MICHAEL G.Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response[J].IEEE Transactions on Medical imaging，2000，19（3）：203-210.
[12] French Ministry.Methods to evaluate segmentation and indexing techniques in the field of retinal ophthalmology[EB/OL].[2021-01-25].http：//www.adcis.net/en/Download-Third-Party/Messidor.html.
[13] DECENCIERE E.TeleOphta：machine learning and image processing methods for teleophthalmology[EB/OL].[2021-01-25].http：//www.adcis.net/en/Download-Third-Party/E-Ophtha.html.
[14] PRASANNA P.Indian diabetic retinopathy image dataset[EB/OL].[2021-01-25].https：//ieee-dataport.org/open-access/indian-diabetic-retinopathy-image-dataset-iDRid.
[15] KAGGLE[EB/OL].[2021-01-25].http：//www.kagglecom/c/diabetic-retinopathy-detection/discussion/15617.
[16] PRENTASIC P，LONCARIC S，VATAVUK Z，et al.Diabetic Retinopathy Image Database（DRiDB）：a new database for diabetic retinopathy screening programs research[EB/OL].[2021-01-25].https：//ipg.fer.hr/ipg/resources/image_database.
[17] CHIU S J，ALLINGHAM M J，METTU P S，et al.Kernel regression based segmentation of optical coherence tomography images with diabetic macular edema[J].Biomedical Optics Express，2015，6（4）：1172-1194.
[18] YI X，WALIA E，BABYN P.Generative adversarial network in medical imaging：a review[J].Medical Image Analysis，2019，58：101552.
[19] SCK A，JIE H B，HMK C，et al.Deep-hipo：multi-scale receptive field deep learning for histopathological image analysis-ScienceDirect[J].Methods，2020，179：3-13.
[20] FISHER Y，KOLTUN V.Multi-scale context aggregation by dilated convolutions[J].arXiv：1511.07122，2015.
[21] JIANG Y，TAN N，PENG T，et al.Retinal vessels segmentation based on dilated multi-scale convolutional neural network[J].arXiv：1904.05664，2019.
[22] 纪玲玉，高永彬，卫子然.MFA-Net：基于多视野上下文关注的视网膜血管分割[J].微电子学与计算机，2021，38（3）：14-20.
JI L Y，GAO Y B，WEI Z R.MFA-Net：retinal vessel segmentation based on multi-view context attention[J].Microelectronics and Computer，2021，38（3）：14-20.
[23] XIAO Y，WANG X，LI Q，et al.A cascade and heterogeneous neural network for CT pulmonary nodule detection and its evaluation on both phantom and patient data[J].Computerized Medical Imaging and Graphics，2021（3）：101889.
[24] SARHAN M H，ALBARQOUNI S，YIGITSOY M，et al.Multi-scale microaneurysms segmentation using embedding triplet loss[C]//Medical Image Computing and Computer Assisted Intervention，2019.
[25] SUN X，CHEN Q，ZHANG N，et al.Mini lesions detection on diabetic retinopathy images via large scale CNN features[J].arXiv：1911.08588，2019.
[26] QILEI C，LIU P，NI J，et al.Pseudo-labeling for small lesion detection on diabetic retinopathy images[C]//2020 International Joint Conference on Neural Networks，Glasgow，UK，2020：1-8.
[27] 杨知桥，张莹，王新杰，等.改进U型网络在视网膜病变检测中的应用研究[J/OL].计算机科学与探索：1-10（2021-02-04）[2021-04-24].http：//kns.cnki.net/kcms/detail/11.5602.TP.20210204.1820.028.html.
YANG Z Q，ZHANG Y，WANG X J，et al.Application research of improved U-shaped network in the detection of retinopathy[J/OL].Journal of Frontiers of Computer Science and Technology：1-10（2021-02-04）[2021-04-24].http：//kns.cnki.net/kcms/detail/11.5602.TP.20210204.
1820.028.html.
[28] RONNEBERGER O，FISCHER P，BROX T.U-Net：convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham：Springer，2015：234-241.
[29] GU Z，CHENG J，FU H，et al.CE-Net：context encoder network for 2D medical image segmentation[J].IEEE Transactions on Medical Imaging，2019，38（10）：2281-2292.
[30] LI R，LI M，LI J，et al.Connection sensitive attention U-NET for accurate retinal vessel segmentation[J].arXiv：1903.05558，2019.
[31] ALOM M Z，HASAN M，YAKOPCIC C，et al.Recurrent residual convolutional neural network based on U-Net（R2U-Net） for medical image segmentation[J].arXiv：1802.06955，2018.
[32] ZHUANG J.LadderNet：multi-path networks based on U-Net for medical image segmentation[J].arXiv：1810.
07810，2018.
[33] WEI W，YU K，HUGONOT J，et al.Recurrent U-Net for resource-constrained segmentation[C]//2019 IEEE/CVF International Conference on Computer Vision（ICCV），Seoul，Korea（South），2019：2142-2151.
[34] QI W，LI X，YU W，et al.Learn to segment retinal lesions and beyond[J].arXiv：1802.06955，2018.
[35] 耿磊，邱玲，吴骏，等.结合深度可分离卷积与通道加权的全卷积神经网络视网膜图像血管分割[J].生物医学工程学杂志，2019，36（1）：107-115.
GENG L，QIU L，JUN W U，et al.Segmentation of retinal image vessels based on fully convolutional network with depthwise separable convolution and channel weighting[J].Journal of Biomedical Engineering，2019，36（1）：107-115.
[36] 梁礼明，蓝智敏，盛校棋，等.融合轮廓信息与条件生成对抗的视网膜血管分割[J].生物医学工程学杂志，2021，38（2）：276-285.
LIANG L M，LAN Z M，SHENG X Q，et al.Segmentation of retinal vessels by fusing contour information and conditional generative adversarial[J].Journal of Biomedical Engineering，2021，38（2）：276-285.
[37] BOUDEGGA H，ELLOUMI Y，AKIL M，et al.Fast and efficient retinal blood vessel segmentation method based on deep learning network[J].Computerized Medical Imaging and Graphics，2021，90：101902.
[38] REN S，HE K，GIRSHICK R，et al.Faster R-CNN：towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，39（6）：1679-1685.
[39] HOSANG J，BENENSON R，PIOTR D，et al.What makes for effective detection proposals?[J].IEEE Transactions on Pattern Analysis & Machine Intelligence，2016，38（4）：814-830.
[40] 谢云霞，黄海于，胡建斌.基于深度卷积神经网络的糖尿病视网膜病变分期及病灶检测[J].计算机应用，2020，40（8）：2460-2464.
XIE Y X，HUANG H Y，HU J B.Staging and lesion detection of diabetic retinopathy based on deep convolution neural network[J].Journal of Computer Applications，2020，40（8）：2460-2464.
[41] 鲁品肃.基于目标检测的糖尿病视网膜病变多模型检测[D].成都：电子科技大学，2020.
LU P S.Multi-model recognition of diabetic retinopathy based on target detection[D].Chengdu：University of Electronic Science and Technology of China，2020.
[42] HE K，GKIOXARI G，PIOTR D，et al.Mask R-CNN[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2020，42（2）：386-397.
[43] FARZAN S，GHAREH M F，AMINI M H，et al.DRDr II：detecting the severity level of diabetic retinopathy using mask RCNN and transfer learning[J].arXiv：2011.
14733，2020.
[44] NOUSHIN E，HAMID P，EHSAN S.Microaneurysm detection in fundus images using a two-step convolutional neural networks[J].arXiv：1710.05191，2017.
[45] JIANG H，YANG K，GAO M，et al.An interpretable ensemble deep learning model for diabetic retinopathy disease classification[C]//2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society，Berlin，Germany，2019：2045-2048.
[46] MUHAMMAD N B，YOSHINOBU T，MUHAMMAD I N，et al.Combining fine-and coarse-grained classifiers for diabetic retinopathy detection[J].arXiv：2005.14308，2020.
[47] TAJBAKHSH N，JEYASEELAN L，LI Q，et al.Embracing imperfect datasets：a review of deep learning solutions for medical image segmentation[J].Medical Image Analysis，2020，63：101693.
[48] CHEPLYGINA V，BRUIJNE M D，PLUIM J.Not-so-supervised：a survey of semi-supervised，multi-instance，and transfer learning in medical image analysis[J].Medical Image Analysis，2019，54：280-296.
[49] ZHAO H，LI H，MAURER-STROH S，et al.Synthesizing retinal and neuronal images with generative adversarial nets[J].Medical Image Analysis，2018，49：14-26.
[50] REN F，CAO P，ZHAO D，et al.Diabetic macular edema grading in retinal images using vector quantization and semi-supervised learning[J].Technol Health Care，2018，26（S1）：389-397.
[51] COSTA P，GALDRAN A，SMAILAGIC A，et al.A weakly-supervised framework for interpretable diabetic retinopathy detection on retinal images[J].IEEE Access，2018，6：18747-18758.
[52] QUELLEC G A，MATHIEU L B A，CONZE A P H C，et al.Automatic detection of rare pathologies in fundus photographs using few-shot learning-ScienceDirect[J].arXiv：1907.09449，2019.
[53] WALEED M G，KOHLER J M，GRZESZICK R，et al.Weakly-supervised localization of diabetic retinopathy lesions in retinal fundus images[C]//2017 IEEE International Conference on Image Processing，Beijing，China，2017：2069-2073.
[54] 丁英姿，丁香乾，郭保琪.基于弱监督的改进型GoogLeNet在DR检测中的应用[J].计算机应用，2019，39（8）：2484-2488.
DING Y Z，DING X Q，GUO B Q.Application of improved GoogLeNet based on weak supervision in DR detection[J].Journal of Computer Applications，2019，39（8）：2484-2488.
[55] 孟凡奎，银温社，贺建峰.基于深度学习的眼底图像出血点检测方法[J].山东大学学报（理学版），2020，55（9）：62-71.
MENG F K，YIN W S，HE J F.Detection method of hemorrhages of fundus image based on deep learning[J].Journal of Shandong University（Natural Science），2020，55（9）：62-71.
[56] LING D，FANG R，LI H，et al.Clinical report guided retinal microaneurysm detection with multi-sieving deep learning[J].IEEE Transactions on Medical Imaging，2018，37（5）：1149-1161.
[57] ZAMIR A R，SAX A，SHEN W，et al.Taskonomy：disentangling task transfer learning[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition，Salt Lake City，UT，USA，2018：3712-3722.
[58] MIHIR R，ZHU M，WANG T.Conversion and implementation of state-of-the-art deep learning algorithms for the classification of diabetic retinopathy[J].arXiv：2010.
11692，2020.
[59] 连超铭，钟舜聪，张添福，等.光学相干断层扫描视网膜图像的迁移学习分类[J].激光与光电子学进展，2021，58（1）：262-268.
LIAN C M，ZHONG S C，ZHANG T F，et al.Transfer learning-based classification of optical coherence tomography retinal images[J].Laser & Optoelectronics Progress，2021，58（1）：262-268.
[60] SAEED F，HUSSAIN M，ABOALSAMH H A.Automatic diabetic retinopathy diagnosis using adaptive fine-tuned convolutional neural network[J].IEEE Access，2021，9：41344-41359.
[61] 王晋宇，杨海涛，李高源，等.生成对抗网络及其图像处理应用研究进展[J].计算机工程与应用，2021，57（8）：26-35.
WANG J Y，YANG H T，LI G Y，et al.Research progress of generative adversarial network and its application in image processing[J].Computer Engineering and Applications，2021，57（8）：26-35.
[62] ALIREZA T，KAMRAN S A，HOSSAIN K F，et al.A novel deep learning conditional generative adversarial network for producing angiography images from retinal fundus photographs[J].Scientific Reports，2020，10（1）：21580.
[63] 陈志远，金炜，李纲.注意力残差链式融合的彩色眼底图像硬性渗出检测[J].光电子·激光，2020，31（10）：1044-1053.
CHEN Z Y，JIN W，LI G.Hard exudation detection of color fundus images with chained fusion of attentional residua[J].Journal of Optoelectronics·Laser，2020，31（10）：1044-1053.
[64] RAMMY S A，ABBAS W，HASSAN N U，et al.CPGAN：conditional patch-based generative adversarial network for retinal vessel segmentation[J].IET Image Processing，2020，14（6）：1081-1090.
[65] DOSHI-VELEZ F，KIM B.Towards a rigorous science of interpretable machine learning[J].arXiv：1702.08608，2017.
[66] OH K，KIM W，SHEN G，et al.Classification of schizophrenia and normal controls using 3D convolutional neural network and outcome visualization[J].Schizophrenia Research，2019，212：186-195.
[67] WYA B，VC C，MING S，et al.Discriminating schizophrenia using recurrent neural network applied on time courses of multi-site FMRI data[J].EBioMedicine，2019，47：543-552.
[68] KEEL S，WU J，LEE P Y，et al.Visualizing deep learning models for the detection of referable diabetic retinopathy and glaucoma[J].JAMA Ophthalmol，2019，137（3）：288-292.
[69] SON J，SHIN J Y，KIM H D，et al.Development and validation of deep learning models for screening multiple abnormal findings in retinal fundus images[J].Ophthalmology，2020，127（1）：85-94.