3D-MRI Super-Resolution Algorithm Fusing Attention and Dilated Encoder-Decoder

doi:10.3778/j.issn.1002-8331.2304-0099

Abstract

Abstract: In order to improve the spatial resolution of brain magnetic resonance imaging (MRI) images, the super resolution (SR) reconstruction method based on deep convolutional neural network (CNN) has achieved remarkable results. Increasing network depth or width usually can expand the receptive field of the network to improve the quality of reconstruction, but it is difficult to train the network due to the large parameter quantity and huge computing requirements. Meanwhile, the network does not pay enough attention to the medium and high frequency features of the image, which affects the quality of reconstruction. To solve the above problems, this paper proposes a cascaded channels-space attention and encode-decode network for 3D-MRI image SR reconstruction. Firstly, feature extraction is carried out in the low resolution space, and the image features are extracted by the symmetric connected encoder-decoder with dilated convolution to alleviate the checkerboard artifacts. Secondly, a serial connected channel-space attention module is constructed to capture the interdependence between the feature channels by channel attention, and the spatial attention is increased to assign weight to information of different positions, effectively enhancing the network’s learning of medium and high frequency information. Finally, subpixel convolution is used to sample the feature map to the target image size and meanwhile reduce memory consumption. Experimental results on public Kirby21 and BraTS datasets show that the proposed method is superior in both quantitative peak signal-to-noise ratio (PSNR), structural similarity (SSIM) and subjective visual quality when compared with traditional SR algorithms and mainstream CNN-based SR algorithms.

Key words: super-resolution reconstruction, attention mechanism, subpixel convolution, dilated convolution, 3D-MRI

摘要： 为了提高脑部磁共振成像（MRI）图像的空间分辨率，目前基于深度卷积神经网络（CNN）的超分辨率（SR）重建方法已经取得了显著的效果。为了提高重建质量，可以通过增加深度或宽度扩大网络的感受野，但会因参数量大、消耗资源多等问题导致模型难以训练；同时，网络对图像的中高频特征关注度不足，影响了重建质量。针对以上问题，设计了一种基于注意力和空洞编码解码的3D-MRI图像超分辨率重建网络。在低分辨率空间进行特征提取，通过对称连接的空洞编码解码器提取图像特征，并缓解棋盘效应；构建串联的通道-空间注意力模块，利用通道注意力获取各特征通道间的相互依赖关系，根据空间注意力自动学习不同位置的特征的权重，有效增强网络对中高频信息的学习；使用亚像素卷积将特征图上采样到目标图像尺寸，降低模型的内存消耗。基于公开数据集Kirby21和BraTS的实验结果表明，相较于传统SR算法和基于CNN的主流SR算法，所提出的算法在峰值信噪比（PSNR）、结构相似度（SSIM）和主观视觉效果上均优于对比算法。

关键词: 超分辨率重建, 注意力机制, 亚像素卷积, 空洞卷积, 3D-MRI

ZHANG Jindi, JIA Yuanyuan, ZHU Huazheng, LI Hongbi, DU Jinglong. 3D-MRI Super-Resolution Algorithm Fusing Attention and Dilated Encoder-Decoder[J]. Computer Engineering and Applications, 2024, 60(13): 228-236.

张金迪, 贾媛媛, 祝华正, 李洪碧, 杜井龙. 融合注意力和空洞编码解码的3D-MRI超分辨率算法[J]. 计算机工程与应用, 2024, 60(13): 228-236.

References

[1] YANG W, ZHANG X, TIAN Y, et al. Deep learning for single image super-resolution: a brief review[J]. IEEE Transactions on Multimedia, 2019, 12(21): 3106-3121.
[2] YUE L, SHEN H, JIE L, et al. Image super-resolution: the techniques, applications, and future[J]. Signal Processing, 2016, 128: 389-408.
[3] ZOU W, YUEN P C. Very low resolution face recognition problem[J]. IEEE Transactions on Image Processing, 2012, 21(1): 327-340.
[4] HUNG K W, QIU C, JIANG J. Video super resolution via deep global-aware network[J]. IEEE Access, 2019, 7: 74711-74720.
[5] MASUTANI E M, NAEIM B, ALBERT H. Deep learning single-frame and multiframe super-resolution for cardiac MRI[J]. Radiology, 2020, 295(3): 552-561.
[6] SHUM H Y, LIN Z. Fundamental limits of reconstruction[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(1): 83-97.
[7] KEY R. Cubic convolution interpolation for digital Image processing[J]. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1981, 29(6): 1153-1160.
[8] 陆志芳, 钟宝江. 基于预测梯度的图像插值算法[J]. 自动化学报, 2018, 44(6): 1072-1085.
LU Z F, ZHONG B J. Image interpolation with predicted gradients[J]. Acta Automatica Sinica, 2018, 44(6): 1072-1085.
[9] PURKAIT P, CHANDA B. Super resolution image reconstruction through bregman iteration using morphologic regularization[J]. IEEE Transactions on Image Processing, 2012, 21(9): 4039-4209.
[10] SHI F, CHENG J, WANG L, et al. LRTV: MR image super-resolution with low-rank and total variation regularizations[J]. IEEE Transactions on Medical Imaging, 2015, 34(12): 2459-2466.
[11] RAHIMAN V A, GEORGE S N. Single image super resolution using neighbor embedding and statistical prediction model[J]. Computers & Electrical Engineering, 2017: 281-292.
[12] LAI W S, HUANG J B, AHUJA N, et al. Deep Laplacian pyramid networks for fast and accurate super-resolution[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 5835-5843.
[13] YANG X, ZHANG Y, ZHOU D, et al. An improved iterative back projection algorithm based on ringing artifacts suppression[J]. Neurocomputing, 2015: 171-179.
[14] DONG C, LOY C C, HE K, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295-307.
[15] KIM J, LEE J K, LEE K M. Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016: 1637-1645.
[16] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016: 770-778.
[17] LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern, 2017: 1132-1140.
[18] PHAM C, TOR-DíEZ C, MEUNIER H, et al. Multiscale brain MRI super-resolution using deep 3D convolutional networks[J]. Computerized Medical Imaging and Graphics, 2019, 77: 101647.
[19] DU J, WANG L, LIU Y, et al. Brain MRI super-resolution using 3D dilated convolutional encoder-decoder network[J]. IEEE Access, 2020, 8: 18938-18950.
[20] XIA Y, RAVIKUMAR N, GREENWOOD J P, et al. Super-resolution of cardiac MR cine imaging using conditional GANs and unsupervised transfer learning[J]. Medical Image Analysis, 2021, 71: 102037.
[21] TAI Y, YANG J, LIU X M. Image super-resolution via deep recursive residual network[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 2017: 3147-3155.
[22] LIN G, WU Q, QIU L, et al. Image super-resolution using a dilated convolutional neural network[J]. Neurocomputing, 2018, 275: 1219-1230.
[23] HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023.
[24] ZHANG Y L, LI K P, LI K, et al. Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the European Conference on Computer Vision, 2018: 286-301.
[25] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the 15th European Conference on Computer Vision, 2018: 3-19.
[26] 印珏泽, 周宁宁. 基于对偶回归和注意力机制的图像超分辨率重建网络[J]. 计算机系统应用, 2023, 32(2): 111-118.
YIN J Z, ZHOU N N. Image super-resolution reconstruction network based on dual regression and attention mechanism[J]. Computer Systems & Applications, 2023, 32(2): 111-118.
[27] ZHANG Z D, WANG X R, JUNG C. Dilated convolutions for single image super-resolution[J]. IEEE Transactions on Image Processing, 2018, 28(4): 1625-1635.
[28] FENG S, JIAN C, LI W, et al. MR image super-resolution with low-rank and total variation regularizations[J]. IEEE Transactions on Medical Imaging, 2015, 34(12): 2459-2466.
[29] 樊帆, 高媛, 秦品乐, 等. 基于并行通道-空间注意力机制的腹部MRI影像多尺度超分辨率重建[J]. 计算机应用, 2020, 40(12): 3624-3630.
FAN F, GAO Y, QIN P L, et al. Abdominal MRl image multi-scale super-resolution reconstruction based on parallel channel-spatial attention mechanism[J]. Journal of Computer Applications, 2020, 40(12): 3624-3630.
[30] WANG H, WU C D, CHI J N, et al. Image super-resolution using multi-granularity perception and pyramid attention networks[J]. Neurocomputing, 2021, 443: 247-261.
[31] ZHANG L, NIE J, WEI W, et al. Deep blind hyperspectral image super-resolution[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020: 2388-2400.
[32] PHAM C, DUCOURNAU A, FABLET R, et al. Brain MRI super-resolution using deep 3D convolutional networks[C]//Proceedings of the IEEE International Symposium on Biomedical Imaging, Melbourne, 2017: 197-200.
[33] SHI W, CABALLERO J, HUSZáR F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016: 1874-1883.
[34] LANDMAN B A, HUANG A J, GIFFORD A, et al. Multi-parametric neuroimaging reproducibility: a 3-T resource study[J]. Neuroimage, 2011, 54(4): 2854-2866.
[35] HANG Z, ORAZIO G, IURI F, et al. Loss functions for image restoration with neural networks[J]. IEEE Transactions on Computational Imaging, 2017, 3(1): 47-57.
[36] MANJóN J V, COUPé P, BUADES A, et al. Non-local MRI upsampling[J]. Medical Image Analysis, 2010, 14(6): 784-792.
[37] DU J, WANG L, GHOLIPOUR A, et al. Accelerated super-resolution MR image reconstruction via a 3D densely connected deep convolutional neural network[C]//Proceedings of the 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Madrid, 2018: 349-355.