层次信息自适应聚合的图像超分辨率重建算法

doi:10.3778/j.issn.1002-8331.2210-0155

摘要/Abstract

摘要： 随着卷积神经网络的发展，图像超分辨率重建算法取得了一定的突破。尽管如此，现有的图像超分辨率算法很少区分利用层次特征，并且存在多尺度特征提取代价大的问题。针对这些问题，提出了层次信息自适应聚合的图像超分辨率重建算法。具体来说，采用多层次信息精炼机制对不同层次的特征进行自适应增强处理，解决层次特征没有区分利用的问题。构造细粒度的多尺度信息聚合块，解决多尺度信息提取代价大，特征表征能力弱的问题。提出对比度增强的重组注意力块，以较低的代价同时利用特征的通道和空间信息，实现对特征的自适应校准。大量实验表明，相比其他先进的算法，所提方法在Urban100等五个基准数据集上能取得更好的指标和视觉效果。

关键词: 超分辨率, 多层次信息精炼, 多尺度信息, 重组注意力

Abstract: With the development of convolutional neural networks, image super-resolution reconstruction algorithms have made some breakthroughs. Nevertheless, the existing image super-resolution algorithms rarely distinguish the use of hierarchical features and suffer from the problem of costly multi-scale feature extraction. To address these problems, this paper proposes an image super-resolution reconstruction algorithm with adaptive aggregation of hierarchical information. Specifically, the algorithm applies a multi-level information refinement mechanism for the adaptive enhancement of features at different levels to solve the problem that the hierarchical features are not distinguishably utilized. In addition, it is proposed to construct a fine-grained multi-scale information aggregation block to solve the problem of costly multi-scale information extraction and poor feature representation capability. Finally, the algorithm focuses on contrast-enhanced recombinant attention blocks to achieve adaptive calibration of features at a lower cost by exploiting channel and spatial information. Extensive experiments show that compared with other advanced algorithms, the proposed method can achieve better metrics and visual results on five benchmark datasets such as Urban100.

Key words: super-resolution, multi-level information refinement, multi-scale information, recombinant attention

陈伟杰, 黄国恒, 莫非, 林俊宇. 层次信息自适应聚合的图像超分辨率重建算法[J]. 计算机工程与应用, 2024, 60(5): 221-231.

CHEN Weijie, HUANG Guoheng, MO Fei, LIN Junyu. Image Super-Resolution Reconstruction Algorithm with Adaptive Aggregation of Hierarchical Information[J]. Computer Engineering and Applications, 2024, 60(5): 221-231.

参考文献

[1] DONG C, LOY C C, HE K, et al. Learning a deep convolutional network for image super-resolution[C]//Proceedings of the European Conference on Computer Vision, 2014: 184-199.
[2] KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1646-1654.
[3] LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017: 136-144.
[4] ZHANG Y, LI K, LI K, et al. Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 286-301.
[5] ZHANG Y, TIAN Y, KONG Y, et al. Residual dense network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 2472-2481.
[6] DAI T, CAI J, ZHANG Y, et al. Second-order attention network for single image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 11065-11074.
[7] HUI Z, GAO X, YANG Y, et al. Lightweight image super-resolution with information multi-distillation network[C]//Proceedings of the 27th ACM International Conference on Multimedia, 2019: 2024-2032.
[8] LIU J, TANG J, WU G. Residual feature distillation network for lightweight image super-resolution[C]//Proceedings of the European Conference on Computer Vision, 2020: 41-55.
[9] LIU J, ZHANG W, TANG Y, et al. Residual feature aggregation network for image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 2359-2368.
[10] ZHAO H, KONG X, HE J, et al. Efficient image super-resolution using pixel attention[C]//Proceedings of the European Conference on Computer Vision, 2020: 56-72.
[11] ZHANG Q L, YANG Y B. SA-Net: shuffle attention for deep convolutional neural networks[C]//Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021: 2235-2239.
[12] WANG Z, Bovik A C, Sheikh H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.
[13] ZHANG L, WU X. An edge-guided image interpolation algorithm via directional filtering and data fusion[J]. IEEE Transactions on Image Processing, 2006, 15(8): 2226-2238.
[14] RHEE S, KANG M G. Discrete cosine transform based regularized high-resolution image reconstruction algorithm[J]. Optical Engineering, 1999, 38(8): 1348-1356.
[15] 丁子轩, 张娟, 李想, 等. 基于注意力引导的轻量级图像超分辨率网络[J]. 激光与光电子学进展, 2023, 60(14):95-103.
DING Z X, ZHANG J, LI X, et al. Lightweight attention-guided network for image super-resolution[J]. Laser & Optoelectronics Progress, 2023, 60(14):95-103.
[16] TIAN C, XU Y, ZUO W, et al. Coarse-to-fine CNN for image super-resolution[J]. IEEE Transactions on Multimedia, 2021, 23: 1489-1502.
[17] TIAN C, ZHUGE R, WU Z, et al. Lightweight image super-resolution with enhanced CNN[J]. Knowledge-Based Systems, 2020, 205: 106235.
[18] 程德强, 赵佳敏, 寇旗旗, 等. 多尺度密集特征融合的图像超分辨率重建[J]. 光学精密工程, 2022, 30(20): 2489-2500.
CHENG D Q, ZHAO J M, KOU Q Q, et al. Multi-scale dense feature fusion network for image super-resolution[J]. Optics and Precision Engineering, 2022, 30(20): 2489-2500.
[19] 张帅勇, 刘美琴, 姚超, 等. 分级特征反馈融合的深度图像超分辨率重建[J]. 自动化学报, 2022, 48(4): 992-1003.
ZHANG S Y, LIU M Q, YAO C, et al. Hierarchical feature feedback network for depth super-resolution reconstruction[J]. Acta Automatica Sinica, 2022, 48(4): 992-1003.
[20] 杨才东, 李承阳, 李忠博, 等. 深度学习的图像超分辨率重建技术综述[J]. 计算机科学与探索, 2022, 16(9): 1990-2010.
YANG C D, LI C Y, LI Z B, et al. Review of image super-resolution reconstruction algorithms based on deep learning[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 1990-2010.
[21] KIM J, LEE J K, LEE K M. Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1637-1645.
[22] TAI Y, YANG J, LIU X. Image super-resolution via deep recursive residual network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 3147-3155.
[23] DONG C, LOY C C, TANG X. Accelerating the super-resolution convolutional neural network[C]//Proceedings of the European Conference on Computer Vision, 2016: 391-407.
[24] SHI W, CABALLERO J, HUSZáR F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1874-1883.
[25] 许娇, 袁三男. 增强型多尺度残差网络的图像超分辨率重建算法[J]. 激光与光电子学进展, 2023, 60(4): 297-305.
XU J, YUAN S N. Image super-resolution reconstruction algorithm based on enhanced multi-scale residual network[J]. Laser & Optoelectronics Progress, 2023, 60(4): 297-305.
[26] AHN N, KANG B, SOHN K A. Fast, accurate, and lightweight super-resolution with cascading residual network[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 252-268.
[27] TIAN C, XU Y, ZUO W, et al. Asymmetric CNN for image super-resolution[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2021, 52(6): 3718-3730.
[28] HUI Z, WANG X, GAO X. Fast and accurate single image super-resolution via information distillation network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 723-731.
[29] QIN J, LIU F, LIU K, et al. Lightweight hierarchical residual feature fusion network for single-image super-resolution[J]. Neurocomputing, 2022, 478: 104-123.
[30] LI J, FANG F, MEI K, et al. Multi-scale residual network for image super-resolution[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 517-532.
[31] FENG X, LI X, LI J. Multi-scale fractal residual network for image super-resolution[J]. Applied Intelligence, 2021, 51(4): 1845-1856.
[32] GAO S H, CHENG M M, ZHAO K, et al. Res2net: a new multi-scale backbone architecture[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 43(2): 652-662.
[33] YUAN P, LIN S, CUI C, et al. HS-ResNet: hierarchical-split block on convolutional neural network[J]. arXiv:2010.07621,2020.
[34] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7132-7141.
[35] LAN R, SUN L, LIU Z, et al. MADNet: a fast and lightweight network for single-image super resolution[J]. IEEE Transactions on Cybernetics, 2021, 51(3): 1443-1453.
[36] SCHWARTZ I, SCHWING A, HAZAN T. High-order attention models for visual question answering[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017: 3667-3677.
[37] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 3-19.
[38] BA J L, KIROS J R, HINTON G E. Layer normalization[J]. arXiv:1607.06450,2016.
[39] AGUSTSSON E, TIMOFTE R. NTIRE 2017 challenge on single image super-resolution: dataset and study[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017: 126-135.
[40] 钟梦圆, 姜麟. 超分辨率图像重建算法综述[J]. 计算机科学与探索, 2022, 16(5): 972-990.
ZHONG M Y, JIANG L. Review of super-resolution image reconstruction algorithms[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(5): 972-990.
[41] BEVILACQUA M, ROUMY A, GUILLEMOT C, et al. Low-complexity single-image super-resolution based on non-negative neighbor embedding[C]//Proceedings of the British Machine Vision Conference (BMVC), 2012: 1-10.
[42] ZEYDE R, ELAD M, PROTTER M. On single image scale-up using sparse-representations[C]//Proceedings of the International Conference on Curves and Surfaces, 2010: 711-730.
[43] MARTIN D, FOWLKES C, TAL D, et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//Proceedings 8th IEEE International Conference on Computer Vision, 2001: 416-423.
[44] HUANG J B, SINGH A, AHUJA N. Single image super-resolution from transformed self-exemplars[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015: 5197-5206.
[45] MATSUI Y, ITO K, ARAMAKI Y, et al. Sketch-based manga retrieval using manga109 dataset[J]. Multimedia Tools and Applications, 2017, 76(20): 21811-21838.
[46] KINGMA D P, BA J. ADAM: a method for stochastic optimization[J]. arXiv:1412.6980,2014.
[47] LAI W S, HUANG J B, AHUJA N, et al. Deep Laplacian pyramid networks for fast and accurate super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 624-632.
[48] TAI Y, YANG J, LIU X, et al. MemNET: a persistent memory network for image restoration[C]//Proceedings of the IEEE International Conference on Computer Vision, 2017: 4539-4547.