Super-Resolution of Fundus Images by Information Distillation and Heterogeneous Up-Sampling

doi:10.3778/j.issn.1002-8331.2105-0325

Abstract

Abstract: Large-scale disease screening in poor areas relies on portable hand-held cameras and remote diagnostics, which often produce poor quality and low-resolution images of the fundus. To solve this problem, a lightweight super-resolution network based on information distillation and heterogeneous up-sampling is designed. Considering the difference between fundus images and natural images, the network makes use of distillation features to supplement coarse features, then up samples coarse features and depth features in a heterogeneous way, and finally integrates two kinds of up sampled features to obtain high-definition fundus images. Compared with the advanced method from three aspects of image quality, parameter memory and run-time, the highest image quality is achieved while the parameter memory and run-time are excellent, which provides an idea for super-resolution algorithm to be embedded in handheld fundus cameras and general medical devices.

Key words: fundus image, super-resolution, information distillation, heterogeneous up-sampling

摘要： 贫困地区的大规模疾病筛查主要依靠便携式手持摄像机和远程诊断来完成，这一过程获得的眼底图像往往质量较差，分辨率较低。针对这一问题，设计了一种基于信息蒸馏与异构上采样的轻量级超分辨网络。网络考虑到眼底图像与自然图像的区别，利用蒸馏特征对粗特征进行补足，然后以异构的方式将粗特征与深度特征分别进行上采样，最后集成两种上采样的特征得到高清眼底图像。从图像质量、参数内存和运行时间三个方面与先进的方法进行比较，在参数内存和运行时间成绩优异的同时取得了最高的图像质量，这为超分辨算法嵌入在手持眼底摄像机和普通医用设备上提供了思路。

关键词: 眼底图像, 超分辨率, 信息蒸馏, 异构上采样

LI Hangyu, XUAN Zuxing, ZHOU Jianping, HU Xiyuan, CHENG Gangwei. Super-Resolution of Fundus Images by Information Distillation and Heterogeneous Up-Sampling[J]. Computer Engineering and Applications, 2022, 58(23): 238-244.

李航宇, 玄祖兴, 周建平, 胡晰远, 程钢炜. 信息蒸馏与异构上采样的眼底图像超分辨重建[J]. 计算机工程与应用, 2022, 58(23): 238-244.

References

[1] ABRàMOFF M D，GARVIN M K，SONKA M.Retinal imaging and image analysis[J].IEEE Reviews in Biomedical Engineering，2010，3：169-208.
[2] QUELLEC G，BAZIN L，CAZUGUEL G，et al.Suitability of a low-cost，handheld，nonmydriatic retinograph for diabetic retinopathy diagnosis[J].Translational Vision Science & Technology，2016，5（2）：16.
[3] CUADROS J，BRESNICK G.Can commercially available handheld retinal cameras effectively screen diabetic retinopathy?[J].Journal of Diabetes Science and Technology，2017，11（1）：135-137.
[4] YUE L，SHEN H，JIE L，et al.Image super-resolution：the techniques，applications，and future[J].Signal Processing，2016，128：389-408.
[5] FARSIU S，ROBINSON M D，ELAD M，et al.Fast and robust multiframe super resolution[J].IEEE Transactions on Image Processing，2004，13（10）：1327-1344.
[6] CHEN Y，XIE Y，ZHOU Z，et al.Brain MRI super resolution using 3D deep densely connected neural networks[C]//2018 IEEE 15th International Symposium on Biomedical Imaging，2018：739-742.
[7] PARKS C，PARK M，KANG M.Super-resolution image reconstruction：a technical overview[J].IEEE Signal Processing Magazine，2003，20：21-36.
[8] LI X，ORCHARD M T.New edge-directed interpolation[J].IEEE Transactions on Image Processing，2001，10（10）：1521-1527.
[9] DONG C，LOY C C，HE K，et al.Image super-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2016，38（2）：295-307.
[10] DONG C，LOY C C，TANG X.Accelerating the super-resolution convolutional neural network[C]//European Conference on Computer Vision，2016：391-407.
[11] ZENG K，ZHENG H，CAI C，et al.Simultaneous single-and multi-contrast super-resolution for brain MRI images based on a convolutional neural network[J].Computers in Biology and Medicine，2018，99：133-141.
[12] KIM J，LEE J K，LEE K M.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition，2016：1646-1654.
[13] LIM B，SON S，KIM H，et al.Enhanced deep residual networks for single image super-resolution[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops（CVPRW），2017.
[14] ZHANGY，LI K，LI K，et al.Image super-resolution using very deep residual channel attention networks[C]//European Conference on Computer Vision，2018.
[15] AHN N，KANG B，SOHN K.Fast，accurate，and lightweight super-resolution with cascading residual network[J].arXiv：1803.08664，2018.
[16] FANZ，DAN T，YU H，et al.Single fundus image super-resolution via cascaded channel-wise attention network[C]//2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society（EMBC），2020：1984-1987.
[17] HUI Z，WANG X，GAO X.Fast and accurate single image super-resolution via information distillation network[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2018：723-731.
[18] HUI Z，GAO X，YANG Y，et al.Lightweight image super-resolution with information multi-distillation network[J].arXiv：1909.11856，2019.
[19] TIAN C，ZHUGE R，WU Z，et al.Lightweight image super-resolution with feature enhancement residual network[J].Neurocomputing，2020，404：50-60.
[20] LEDIG C，THEIS L，HUSZAR F，et al.Photo-realistic single image super-resolution using a generative adversarial network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition，2017：105-114.
[21] WANG X T，YU K，WU S X，et al.ESRGAN：enhanced super-resolution generative adversarial networks[C]//Computer Vision-ECCV 2018 Workshops，2019.
[22] YUANY，LIU S，ZHANG J，et al.Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops，2018：814-81409.
[23] MAHAPATRA D，BOZORGTABAR B，HEWAVITHARANAGE S，et al.Image super resolution using generative adversarial networks and local saliency maps for retinal image analysis[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention，2017：382-390.
[24] ANOOPV，BIPIN P R.Super-resolution based automatic diagnosis of retinal disease detection for clinical applications[J].Neural Processing Letters，2020，2：1-16.
[25] PORWAL P，PACHADE S，KAMBLE R，et al.Indian diabetic retinopathy image dataset（IDRiD）：a database for diabetic retinopathy screening research[J].Data，2018，3（3）：25.
[26] WANG Z，BOVIK A C，SHEIKH H R，et al.Image quality assessment：from error visibility to structural similarity[J].IEEE Transactions on Image Processing，2004，13（4）：600-612.
[27] WOO S，PARK J，LEE J，et al.CBAM：convolutional block attention module[C]//European Conference on Computer Vision，2018：3-19.