Dual Regression Networks for Single Image Super-Resolution

doi:10.3778/j.issn.1002-8331.2112-0308

Abstract

Abstract: There are three limitations to the existing single-image super-resolution model based on convolutional neural networks. Theoretically, there is an infinite HR image that can be downsampled to the same LR image, and the possible function space is very large. Because the potential downsampling method of the real world is unknown, the model of data training paired with a specific method has poor generalization ability in practical application, which causes adaptability problems. The high-frequency hierarchical characteristics of residual branches are ignored. In view of the above problems, a dual regression scheme is proposed. In addition to learn the original regression mapping from LR to HR images, an additional pair regression map is learned to estimate the subsampling core and reconstruct the LR image, forming a closed loop to provide additional supervision, and introducing Fourier transformations on the residual structure to enhance the ability of model to express high-frequency information. Compared with other advanced models, HR images are reconstructed with fewer parameters and rich in high-frequency texture details.

Key words: single image super-resolution, deep learning, convolutional neural network

摘要： 现有基于卷积神经网络的单图像超分辨率模型存在三个限制。理论上存在无限的HR图像，可以下采样到相同的LR图像，可能的函数空间非常大。因为现实世界潜在的下采样方法是未知的，使用特定方法配对的数据训练的模型在实际应用中泛化能力差，产生适应性问题。忽视残差分支的高频层次特征。针对上述问题，提出双重回归方案。除了学习从LR到HR图像的原始回归映射之外，额外学习一个对偶回归映射来估计下采样核并重建LR图像，形成一个闭环提供额外的监督，并在残差结构上引入了傅里叶变换，增强模型对高频信息的表达能力。相比其他先进模型以更少的参数重建HR图像，且拥有丰富的高频纹理细节。

关键词: 单图像超分辨率重建, 深度学习, 卷积神经网络

ZHANG Yong, LYU Geng. Dual Regression Networks for Single Image Super-Resolution[J]. Computer Engineering and Applications, 2022, 58(18): 277-283.

张永, 吕庚. 双回归网络的单图像超分辨率重建[J]. 计算机工程与应用, 2022, 58(18): 277-283.

References

[1] DONG C，LOY C C，HE K，et al.Learning a deep convolutional network for image super-resolution[C]//European Conference on Computer Vision.Cham：Springer，2014：184-199.
[2] KIM J，LEE J K，LEE K M.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：1646-1654.
[3] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[4] HARIS M，SHAKHNAROVICH G，UKITA N.Deep back-projection networks for super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：1664-1673.
[5] ZHANG Y L，LI K P，LI K，et al.Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：286-301.
[6] ZHOU R，SUSSTRUNK S.Kernel modeling super-resolution on real low-resolution images[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：2433-2443.
[7] GUO Y，CHEN J，WANG J，et al.Closed-loop matters：dual regression networks for single image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：5407-5416.
[8] 席志红，侯彩燕，袁昆鹏，等.基于深层残差网络的加速图像超分辨率重建[J].光学学报，2019，39（2）：89-98.
XI Z H，HOU C Y，YUAN K P，et al.Super-resolution reconstruction of accelerated image based on deep residual network[J].Acta Optica Sinica，2019，39（2）：89-98.
[9] LI Z，LI Q，WU W，et al.Deep recursive up-down sampling networks for single image super-resolution[J].Neurocomputing，2020，398：377-388.
[10] 彭晏飞，高艺，杜婷婷，等.生成对抗网络的单图像超分辨率重建方法[J].计算机科学与探索，2020，14（9）：1612-1620.
PENG Y F，GAO Y，DU T T，et al.Single image super-resolution reconstruction method for generative adversarial network[J].Journal of Frontiers of Computer Science and Technology，2020，14（9）：1612-1620.
[11] HE D，XIA Y，QIN T，et al.Dual learning for machine translation[J].Advances in Neural Information Processing Systems，2016，29：820-828.
[12] XIA Y，QIN T，CHEN W，et al.Dual supervised learning[C]//International Conference on Machine Learning，2017：3789-3798.
[13] XIA Y，TAN X，TIAN F，et al.Model-level dual learning[C]//International Conference on Machine Learning，2018：5383-5392.
[14] ZHANG Y，XIANG T，HOSPEDALES T M，et al.Deep mutual learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：4320-4328.
[15] ZHU J Y，PARK T，ISOLA P，et al.Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：2223-2232.
[16] YI Z，ZHANG H，TAN P，et al.Dualgan：unsupervised dual learning for image-to-image translation[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：2849-2857.
[17] YUAN Y，LIU S，ZHANG J，et al.Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops，2018：701-710.
[18] MOHRI M，ROSTAMIZADEH A，TALWALKAR A.Foundations of machine learning[M].[S.l.]：MIT Press，2012：35-38.
[19] AGUSTSSON E，TIMOFTE R.Ntire 2017 challenge on single image super-resolution：dataset and study[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops，2017：126-135.
[20] BEVILACQUA M，ROUMY A，GUILLEMOT C，et al.Low-complexity single-image super-resolution based on nonnegative neighbor embedding[C]//British Machine Vision Conference，Surrey，September 3-7，2012：1-10.
[21] ZEYDE R，ELAD M，PROTTER M.On single image scale-up using sparse-representations[C]//International Conference on Curves and Surfaces.Berlin，Heidelberg：Springer，2010：711-730.
[22] MATSUI Y，ITO K，ARAMAKI Y，et al.Sketch-based manga retrieval using manga109 dataset[J].Multimedia Tools and Applications，2017，76（20）：21811-21838.