Image Super-Resolution with Light-Weighted Pyramid Pooling-Based Attention Network

doi:10.3778/j.issn.1002-8331.2203-0266

Abstract

Abstract: In the task of image super-resolution reconstruction based on deep learning, most of the current algorithms improve their performance by expanding the network scale, which leads to the increase of computing resources. To solve the problem mentioned above, a light-weighted pyramid pooling-based attention network（LiPAN） is proposed which is composed of information distillation block, pyramid pooling and backward attention fusion module. The attention mechanism ensures that the network extracts important features, the pyramid pooling structure can get more context information and obtain more accurate reconstruction results, and the distillation structure can effectively improve the network performance and reduce network parameters. Compared with state-of-the-art lightweight network models, quantitative evaluation among the scale factor of 2, 3, 4 on four public datasets, including Set5, Set14, BSD100 and Urban100, the proposed LiPAN model is able to achieve superior PSNR and SSIM values. It is shown that LiPAN has better super-resolution reconstruction performance when the model parameter is comparable to the current mainstream light-weighted network.

Key words: super-resolution reconstruction, attention mechanism, distillation network, light-weighted, pyramid pooling

摘要： 在基于深度学习的图像超分辨率重建领域，通过扩大网络规模以提高性能将导致计算资源损耗增加。为此，提出了一种轻量级的基于金字塔池化注意力机制网络（light-weighted pyramid pooling-based attention network，LiPAN），该算法模型由融合注意力机制的信息蒸馏块、多层金字塔池化结构和反向注意力融合模块组成。注意力机制确保了网络对重要特征的提取，金字塔池化结构可获取更多的上下文信息，得到更准确的重建结果，蒸馏结构的引入可有效地提高网络性能并减少网络参数。与目前主流的轻量级网络模型相比，提出的LiPAN模型在Set5、Set14、BSD100及Urban100四个公共数据集分别进行2倍、3倍和4倍下采样重建并定量评估，获得最优峰值信噪比和结构相似度。由此表明，提出的LiPAN在网络模型参数与当前主流的轻量级网络相当的情况下，具有更优的超分辨率重建性能。

关键词: 超分重建, 注意力机制, 蒸馏网络, 轻量级, 金字塔池化

FANG Jinsheng, ZHU Gupei. Image Super-Resolution with Light-Weighted Pyramid Pooling-Based Attention Network[J]. Computer Engineering and Applications, 2022, 58(20): 197-205.

方金生, 朱古沛. 轻型金字塔池化注意力机制网络实现图像超分[J]. 计算机工程与应用, 2022, 58(20): 197-205.

References

[1] 李祥霞，谢娴，李彬，等.生成对抗网络在医学图像处理中的应用[J].计算机工程与应用，2021，57（18）：24-37.
LI X X，XIE X，LI B，et al.Application of generative adversarial networks in medical image processing[J].Computer Engineering and Applications，2021，57（18）：24-37.
[2] 张德，林青宇，郭茂祖.单幅图像超分辨重建的深度学习方法综述[J].计算机工程与应用，2021，57（22）：28-41.
ZHANG D，LIN Q Y，GUO M Z.Review of single image super-resolution based on deep learning[J].Computer Engineering and Applications，2021，57（22）：28-41.
[3] ZHOU B，YE D J，WEI W，et al.Alternating direction projections onto convex sets for super-resolution image reconstruction[J].Information Technology and Control，2020，49（1）：179-190.
[4] LI B，ZHOU Y，ZHANG Y，et al.Depth image super-resolution based on joint sparse coding[J].Pattern Recognition Letters，2020：13021-13029.
[5] ALZUBAIDI L，ZHANG J，HUMAIDI A J，et al.Review of deep learning：concepts，CNN architectures，challenges，applications，future directions[J].Journal of Big Data，2021，8（1）：53.
[6] 孙超文，陈晓.基于多尺度特征融合反投影网络的图像超分辨率重建[J].自动化学报，2021，47（7）：1689-1700.
SUN C W，CHEN X.Multiscale feature fusion backprojection network for image super-resolution[J].Acta Automatica Sinica，2021，47（7）：1689-1700.
[7] 刘璟，宋海川，黄建设，等.通道与空间注意力图像超分辨率网络[J].计算机工程与应用，2021，57（2）：209-216.
LIU J，SONG H C，HUANG J S，et al.Channel and spatial attention image super-resolution network[J].Computer Engineering and Applications，2021，57（2）：209-216.
[8] LUO X，XIE Y，ZHANG Y，et al.Latticenet：towards lightweight image super-resolution with lattice block[C]//Proceedings of the European Conference on Computer Vision，2020：272-289.
[9] MEI Y，FAN Y，ZHOU Y.Image super-resolution with non-local sparse attention[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：3517-3526.
[10] 唐家军，刘辉，胡雪影.功能型复合深度网络的图像超分辨率重建[J].计算机科学与探索，2020，14（8）：1368-1379.
TANG J J，LIU H，HU X Y.Image super-resolution reconstruction of functional composite deep network[J].Journal of Frontiers of Computer Science and Technology，2020，14（8）：1368-1379.
[11] 彭晏飞，高艺，杜婷婷，等.生成对抗网络的单图像超分辨率重建方法[J].计算机科学与探索，2020，14（9）：1612-1620.
PENG Y F，GAO Y，DU T T，et al.Single image super-resolution reconstruction method for generative adversarial network[J].Journal of Frontiers of Computer Science and Technology，2020，14（9）：1612-1620.
[12] DONG C，LOY C C，HE K，et al.Image super-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，38（2）：295-307.
[13] XIN L，ORCHARD M T.New edge directed interpolation[J].IEEE Transactions on Image Processing，2001，10（10）：1521-1527.
[14] DONG C，LOY C C，TANG X.Accelerating the super-resolution convolutional neural network[C]//Proceedings of the European Conference on Computer Vision，2016：394-407.
[15] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[16] KIM J，LEE J K，LEE K M.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016.
[17] LIM B，SON S，KIM H，et al.Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops（CVPRW），2017：136-144.
[18] TAI Y，YANG J，LIU X.Image super-resolution via deep recursive residual network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：3147-3155.
[19] ZHANG Y L，LI K P，LI K，et al.Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the European Conference on Computer Vision，2018：286-301.
[20] LAI W S，HUANG J B，AHUJA N，et al.Deep Laplacian pyramid networks for fast and accurate super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：5835-5843.
[21] AHN N，KANG B，SOHN K A.Fast，accurate，and lightweight super-resolution with cascading residual network[C]//Proceedings of the European Conference on Computer Vision，2018：252-268.
[22] HUI Z，WANG X，GAO X.Fast and accurate single image super-resolution via information distillation network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：723-731.
[23] HUI Z，GAO X，YANG Y，et al.Lightweight image super-resolution with information multi-distillation network[C]//Proceedings of the 27th ACM International Conference on Multimedia，2019：2024-2032.
[24] LIU J，TANG J，WU G.Residual feature distillation network for lightweight image super-resolution[C]//Proceedings of the European Conference on Computer Vision，2020：41-55.
[25] HE K，ZHANG X，REN S，et al.Spatial pyramid pooling in deep convolutional networks for visual recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，37（9）：1904-1916.
[26] ZHAO H，SHI J，QI X，et al.Pyramid scene parsing network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：2881-2890.
[27] HU J，SHEN L，SUN G.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：7132-7141.
[28] LIU J，ZHANG W，TANG Y，et al.Residual feature aggregation network for image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：2359-2368.
[29] AGUSTSSON E，TIMOFTE R.Ntire 2017 challenge on single image super-resolution：dataset and study[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops，2017：126-135.
[30] BEVILACQUA M，ROUMY A，GUILLEMOT C，et al.Low-complexity single image super-resolution based on nonnegative neighbor embedding[C]//Proceedings of the British Machine Vision Conference，2012.
[31] ZEYDE R，ELAD M，PROTTER M.On single image scale-up using sparse-representations[C]//Proceedings of the International Conference on Curves and Surfaces，2010：711-730.
[32] TIMOFTE R，DE SMET V，VAN GOOL L.A+：adjusted anchored neighborhood regression for fast super-
resolution[C]//Proceedings of the Asian Conference on Computer Vision，2014：111-126.
[33] HUANG J B，SINGH A，AHUJA N.Single image super-resolution from transformed self-exemplars[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2015：5197-5206.
[34] TAI Y，YANG J，LIU X，et al.MemNet：a persistent memory network for image restoration[C]//Proceedings of the IEEE International Conference on Computer Vision，2017.