具有自适应激活层和感受野块的超分辨网络

doi:10.3778/j.issn.1002-8331.2306-0104

摘要/Abstract

摘要： 基于生成对抗网络的超分辨网络在提升图像感知质量等方面获得了重大突破，解决了SR图像中的边缘平滑问题，但依然存在纹理细节缺失、噪点和伪影等问题。在此提出一种含有学习层的自适应激活层MetaAconc与残差网络重构结合，得到一种新的残差块ResMA。此外感受野块已经在目标检测等方面取得了不错的效果，为了增强特征细节，提出了增强感受野密集残差块（enhanced residual of receptive field dense block，ERRFDB）。将ResMA和ERRFDB进行网络结构重组，提出了一种全新的超分辨率生成器模型（SRRMA-RFB）。在SRGAN网络基础上，将生成器替换为SRRMA-RFB的网络模型称为SRGAN-ARF，为验证其重建图像的视觉效果和评价指标都有所提高，将其与SRGAN和ESRGAN算法进行对比。实验证明，所提算法在提高网络性能并且控制计算量的同时，使重建图像拥有更好的感知质量和纹理细节并且在减轻噪声方面具有一定的优势。

关键词: 图像超分辨重建, 自适应激活层, 感受野块

Abstract: The super-resolution network based on generative adversarial network has obtained a significant breakthrough in improving image perception quality. It solves the edge smoothing problem in SR images, but there are still problems such as missing texture details, noise and artifacts. Here, a new residual block ResMA is proposed by combining MetaAconc, an adaptive activation layer containing a learning layer, with the reconstruction of the residual network. In addition, the receptive field block has achieved good results in object detection, and in order to enhance the feature details, the enhanced residual of receptive field dense block (ERRFDB) is proposed. A new super-resolution generator model (SRRMA-RFB) is proposed by reorganizing the network structure of ResMA and ERRFDB. Based on the SRGAN network, the network model that replaces the generator with SRRMA-RFB is called SRGAN-ARF, and in order to verify that the visual effect and evaluation index of its reconstructed image are improved, it is compared with SRGAN and ESRGAN algorithms. Experiments show that the proposed algorithm not only improves the network performance and controls the amount of calculation, but also makes the reconstructed image have better perception quality and texture details and has certain advantages in reducing noise.

Key words: super-resolution image reconstruction, adaptive activation layer, receptive field block

关宇珅, 梁玉琦. 具有自适应激活层和感受野块的超分辨网络[J]. 计算机工程与应用, 2024, 60(18): 230-238.

GUAN Yushen, LIANG Yuqi. Super-Resolution Network with Adaptive Activation Layer and Receptive Field Block[J]. Computer Engineering and Applications, 2024, 60(18): 230-238.

参考文献

[1] 蔚瑞, 谢卓辰, 刘洁, 等. 基于图像超分辨重建的低轨卫星频谱感知数据空间分辨率提高方法[J]. 中国科学院大学学报, 2022, 39(3): 386-392.
WEI R, XIE Z C, LIU J, et al. Spatial resolution improvement of spectrum sensing data of LEO satellite based on image super-resolution[J]. Journal of the University of Chinese Academy of Sciences, 2022, 39(3): 386-392.
[2] GENDY G, MOHAMMED H, SABOR N, et al. A deep pyramid attention network for single image super-resolution[C]//2021 9th International Japan-Africa Conference on Electronics, Communications, and Computations (JAC-ECC), 2021: 4-19.
[3] 吴靖, 叶晓晶, 黄峰, 等. 基于深度学习的单帧图像超分辨率重建综述[J]. 电子学报, 2022, 50(9): 2265-2294.
WU J, YE X J, HUANG F, et al. A review of single image super?resolution reconstruction based on deep learning[J]. Acta Electronica Sinica, 2022, 50(9): 2265-2294.
[4] DONG C, CHEN C L, HE K, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 38(2): 295-307.
[5] CHEN Y, LIU L, PHONEVILAY V, et al. Image super-resolution reconstruction based on feature map attention mechanism[J]. Applied Intelligence, 2021, 51: 4367-4380.
[6] KIM J, LEE J K, LEE K M. Accurate image super resolution using very deep convolutional networks[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016: 1646-1654.
[7] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016: 770-778.
[8] LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2017: 151-158.
[9] GENDY G, SABOR N, HOU J C, et al. Balanced spatial feature distillation and pyramid attention network for lightweight image super-resolution[J]. Neurocomputing, 2022, 509: 157-166.
[10] HUANG J B, SINGH A, AHUJA N. Single image super resolution from transformed self-exemplars[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015: 23-32.
[11] 杨记鑫, 胡伟霞, 赵杰, 等. 基于生成对抗网络的图像超分辨算法[J]. 计算机技术与发展, 2022, 32(4): 57-62.
YANG J X, HU W X, ZHAO J, et al. Image super resolution algorithm based on generative countermeasure network [J]. Computer Technology and Development, 2022, 32(4): 57-62.
[12] LEDIG C, THEIS L, HUSZAR F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017: 19-27.
[13] WANG X T, YU K, WU S X, et al. ESRGAN: enhanced super-resolution generative adversarial networks[J]. arXiv:1809.00219, 2018.
[14] KONG F, LI M, LIU D, et al. Residual local feature network for efficient super-resolution[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2022: 766-776.
[15] SUN L, LIU K, LIU Z, et al. A lightweight multi-scale feature integration network for real-time single image super-resolution[J]. Real-Time Image Process, 2021, 18(4): 1221-1234.
[16] DU Z, LIU D, LIU J, et al. Fast and memory-efficient network towards efficient image super-resolution[C]//IEEE Conference on Computer Vision and Pattern Recognition, 2022: 817-823.
[17] SUN L, LIU Z, SUN X, et al. Lightweight image super-resolution via weighted multi-scale residual network[J]. IEEE/CAA Journal of Automatica Sinica, 2021, 8(7): 1271-1280.
[18] GENDY G, HE G H, SABOR N. Lightweight image super-resolution based on deep learning: state-of-the-art and future directions[J]. Information Fusion, 2023, 94: 284-310.
[19] MANDY L, DIETLIND Z, OLAF H, et al. Applications of norms and their smooth approximations for gradient based learning vector quantization[C]//22th European Symposium on Artificial Neural Networks (ESANN 2014), Bruges, Belgium, April 23-25, 2014: 1-13.
[20] MA N N, ZHANG X Y, LIU M, et al. Activate or not: learning customized activation[J]. Proceedings of the IEEE, 2021, 10(4): 550-559.
[21] PRAJIT R, BARRET Z, QUOC V L. Searching for activation functions[J]. arXiv:1710.05941, 2017.
[22] LIU S, HUANG D, WANG Y. Receptive field block net for accurate and fast object detection[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 385-400.
[23] 杨才东, 李承阳, 李忠博, 等. 深度学习的图像超分辨率重建技术综述[J]. 计算机科学与探索, 2022, 16(9): 1990-2010.
YANG C D, LI C Y, LI, Z B, et al. Review of image super-resolution reconstruction algorithms based on deep learning[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 1990-2010.
[24] BLAU Y, MICHAELI T. The perception-distortion tradeoff [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 6228-6237.
[25] YOCHAI B, ROEY M, RADU T, et al. Challenge on perceptual image super resolution[J]. arXiv:1809.07517, 2018.