Multi-Scale Medical Image Super-Resolution Reconstruction

doi:10.3778/j.issn.1002-8331.2105-0202

Abstract

Abstract: Aiming at the problem that most prior methods based on deep learning focus on the alone scale factor medical image super-resolution task, a multi-scale super-resolution network for medical image is proposed. The model is optimized on the basis of residual dense network. Multiple skip-changed residual dense blocks are cascaded in feature extraction. Connection complexity is reduced to logarithm. The shallow feature in medical image is not processed repeatedly. The schema that meta-information embedding directly is used in feature map reconstruction. The universal knowledge between different scale factor task is learned through a small network. The diverse tasks are integrated. The tasks of different scale factors are transferred to the same dimension. The fractional scale factor is supported. The experimental results show that, compared with canonical method such as VDSR, the peak signal-to-noise ratio increases 0.17~1.57 dB and structural similarity increases 0.002 2~0.042 5.

Key words: super-resolution, deep learning, residual dense, meta-information embedding, medical image

摘要： 针对当前大多数基于深度学习的医学图像超分辨率重建方法存在放大因子单一的问题，提出一种多重放大的医学图像超分辨率重建网络模型。以密集残差网络为基础，特征提取级联多个改进连接的密集残差块，降低连接复杂度至对数级，避免浅层的医学图像特征被反复处理。特征图重建采用元信息直接嵌入模式，利用一个小型网络学习不同放大因子任务间的通用知识，实现不同重建任务的整合。将不同放大因子任务对齐至同一维度，实现对小数重建任务的支持。实验结果表明，所提方法与深度卷积（VDSR）等典型方法相比，在峰值信噪比（PSNR）与结构相似度（SSIM）上仍有0.17~1.57 dB与0.002 2~0.042 5的提升。

关键词: 超分辨率, 深度学习, 密集残差, 元信息嵌入, 医学图像

ZHANG Weifan, ZENG Qingpeng. Multi-Scale Medical Image Super-Resolution Reconstruction[J]. Computer Engineering and Applications, 2022, 58(23): 230-237.

章伟帆, 曾庆鹏. 多重放大的医学图像超分辨率重建[J]. 计算机工程与应用, 2022, 58(23): 230-237.

References

[1] CHRISTENSEN-JEFFRIES K，COUTURE O，DAYTON P A，et al.Super-resolution ultrasound imaging[J].Ultra- sound in Medicine & Biology，2020，46（4）：865-891.
[2] 席志红，侯彩燕，袁昆鹏.基于残差网络的医学图像超分辨率重建[J].计算机工程与应用，2019，55（19）：191-197.
XI Z H，HOU C Y，YUAN K P.Medical image super resolution reconstruction based on residual network[J].Computer Engineering and Applications，2019，55（19）：191-197.
[3] 苏衡，周杰，张志浩.超分辨率图像重建方法综述[J].自动化学报，2013，39（8）：1202-1213.
SU H，ZHOU J，ZHANG Z H.Survey of super-resolution image reconstruction methods[J].Acta Automatica Sinica，2013，39（8）：1202-1213.
[4] HU Z，WANG Y，ZHANG X，et al.Super-resolution of PET image based on dictionary learning and random forests[J].Nuclear Instruments and Methods in Physics Research Section A：Accelerators，Spectrometers，Detectors and Associated Equipment，2019，927：320-329.
[5] 李现国，冯欣欣，李建雄.多尺度残差网络的单幅图像超分辨率重建[J].计算机工程与应用，2021，57（7）：215-221.
LI X G，FENG X X，LI J X.Sigle image super-resolution reconstruction based on multi-scale residual network[J].Computer Engineering and Applications，2021，57（7）：215-221.
[6] QIN J，HUANG Y，WEN W.Multi-scale feature fusion residual network for single image super-resolution[J].Neurocomputing，2020，379：334-342.
[7] 沈瑜，刘成，杨倩.利用子空间稀疏特征的超分辨率图像重建算法[J].计算机工程与应用，2021，57（5）：173-182.
SHEN Y，LIU C，YANG Q.Super-resolution image reconstruction algorithm using sparse features in subspace[J].Computer Engineering and Applications，2021，57（5）：173-182.
[8] LIU B，AIT-BOUDAOUD D.Effective image super resolution via hierarchical convolutional neural network[J].Neurocomputing，2020，374：109-116.
[9] DONG C，LOY C C，HE K，et al.Image super-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，38（2）：295-307.
[10] DONG C，LOY C C，TANG X.Accelerating the super- resolution convolutional neural network[C]//European Conference on Computer Vision.Cham：Springer，2016：391-407.
[11] SHI W，CABALLERO J，HUSZáR F，et al.Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：1874-1883.
[12] KIM J，LEE J K，LEE K M.Accurate image super- resolution using very deep convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：1646-1654.
[13] TONG T，LI G，LIU X，et al.Image super-resolution using dense skip connections[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：4799-4807.
[14] ZHANG Y，TIAN Y，KONG Y，et al.Residual dense network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern recognition，2018：2472-2481.
[15] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[16] HUANG G，LIU Z，VAN DER MAATEN L，et al.Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：4700-4708.
[17] 李凡长，刘洋，吴鹏翔，等.元学习研究综述[J].计算机学报，2021，44（2）：422-446.
LI F C，LIU Y，WU P X，et al.A survey on recent advances in meta-learning[J].Chinese Journal of Computers，2021，44（2）：422-446.
[18] ZHOU M，LI Y，LU H，et al.Semi-supervised meta- learning via self-training[C]//2020 3rd International Conference on Intelligent Autonomous Systems（ICoIAS），2020：1-7.
[19] 刘兵，杨娟，汪荣贵，等.结合记忆与迁移学习的小样本学习[J].计算机工程与应用，2022，58（19）：242-249.
LIU B，YANG J，WANG R G，et al.Memory?based transfer learning for few?shot learning[J].Computer Engineering and Applications，2022，58（19）：242-249.
[20] CAI Q，PAN Y，YAO T，et al.Memory matching networks for one-shot image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：4080-4088.
[21] YANG T，ZHANG X，LI Z，et al.MetaAnchor：learning to detect objects with customized anchors[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems，2018：318-328.