计算机工程与应用 ›› 2023, Vol. 59 ›› Issue (8): 28-40.DOI: 10.3778/j.issn.1002-8331.2208-0223
张艳青,马建红,韩颖,曹仰杰,李颉,杨聪
出版日期:
2023-04-15
发布日期:
2023-04-15
ZHANG Yanqing, MA Jianhong, HAN Ying, CAO Yangjie, LI Jie, YANG Cong
Online:
2023-04-15
Published:
2023-04-15
摘要: 单幅图像超分辨率是近几十年来计算机视觉领域的一个重要研究课题,基于深度学习的超分辨率重建算法已经取得突破性进展,但当大多数算法应用到真实场景中的图像时效果会大大降低,出现严重模糊、振铃效应等。在此背景下越来越多研究人员致力于研究真实场景下的图像超分辨率算法(real-world single image super-resolution,RSISR)。聚焦于真实场景下图像超分辨率重建算法,介绍了常用公共图像数据集和图像评估指标,从基于外部数据集SR方法和基于内部数据集SR方法两大方向分析对比了各种方法的特点、性能和不足。讨论了RSISR存在的困难和挑战,并对未来的发展趋势进行了思考与展望。
张艳青, 马建红, 韩颖, 曹仰杰, 李颉, 杨聪. 真实场景下图像超分辨率重建研究综述[J]. 计算机工程与应用, 2023, 59(8): 28-40.
ZHANG Yanqing, MA Jianhong, HAN Ying, CAO Yangjie, LI Jie, YANG Cong. Review of Research on Real-World Single Image Super-Resolution Reconstruction[J]. Computer Engineering and Applications, 2023, 59(8): 28-40.
[1] ZHANG L,ZHANG H,SHEN H,et al.A super-resolution reconstruction algorithm for surveillance images[J].Signal Processing,2010,90(3):848-859. [2] GREENSPAN H.Super-resolution in medical imaging[J].The Computer Journal,2009,52(1):43-63. [3] 黄健,赵元元,郭苹,等.深度学习的单幅图像超分辨率重建方法综述[J].计算机工程与应用,2021,57(18):13-23. HUANG J,ZHAO Y Y,GUO P,et al.Survey of single image super-resolution based on deep learning[J].Computer Engineering and Applications,2021,57(18):13-23. [4] LEI S,SHI Z,ZOU Z.Super-resolution for remote sensing images via local-global combined network[J].IEEE Geoscience and Remote Sensing Letters,2017,14(8):1243-1247. [5] ZHANG S,YUAN Q,LI J,et al.Scene-adaptive remote sensing image super-resolution using a multiscale attention network[J].IEEE Transactions on Geoscience and Remote Sensing,2020,58(7):4764-4779. [6] DAI D,WANG Y,CHEN Y,et al.Is image super-resolution helpful for other vision tasks?[C]//2016 IEEE Winter Conference on Applications of Computer Vision(WACV),2016:1-9. [7] GUO Z,WU G,SONG X,et al.Super-resolution integrated building semantic segmentation for multi-source remote sensing imagery[J].IEEE Access,2019,7:99381-99397. [8] WANG L,LI D,ZHU Y,et al.Dual super-resolution learning for semantic segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2020:3774-3783. [9] PANG Y,CAO J,WANG J,et al.JCS-Net:joint classification and super-resolution network for small-scale pedestrian detection in surveillance images[J].IEEE Transactions on Information Forensics and Security,2019,14(12):3322-3331. [10] ZHANG Y,BAI Y,DING M,et al.KGSnet:key-point-guided super-resolution network for pedestrian detection in the wild[J].IEEE Transactions on Neural Networks and Learning Systems,2020,32(5):2251-2265. [11] YANG X,WU W,LIU K,et al.Long-distance object recognition with image super resolution:a comparative study[J].IEEE Access,2018,6:13429-13438. [12] SIADARI T S,HAN M,YOON H.GSR-MAR:global super-resolution for person multi-attribute recognition[C]//ICCV Workshops,2019:1098-1103. [13] LI X,ORCHARD M T.New edge-directed interpolation[J].IEEE Transactions on Image Processing,2001,10(10):1521-1527. [14] SUN J,XU Z,SHUM H Y.Image super-resolution using gradient profile prior[C]//2008 IEEE Conference on Computer Vision and Pattern Recognition,2008:1-8. [15] FORD C,ETTER D.Wavelet basis reconstruction of nonuniformly sampled data[J].IEEE Transactions on Circuits and Systems II:Analog and Digital Signal Processing,1998,45(8):1165-1168. [16] BLU T,THéVENAZ P,UNSER M.Linear interpolation revitalized[J].IEEE Transactions on Image Processing,2004,13(5):710-719. [17] KEYS R.Cubic convolution interpolation for digital image processing[J].IEEE Transactions on Acoustics,Speech,and Signal Processing,1981,29(6):1153-1160. [18] SCHULTZ R R,STEVENSON R L.Extraction of high-resolution frames from video sequences[J].IEEE Transactions on Image Processing,1996,5(6):996-1011. [19] BANHAM M R,KATSAGGELOS A K.Digital image restoration[J].IEEE Signal Processing Magazine,1997,14(2):24-41. [20] NASONOV A V,KRYLOV A S.Fast super-resolution using weighted median filtering[C]//2010 20th International Conference on Pattern Recognition,2010:2230-2233. [21] K?HLER T,B?TZ M,NADERI F,et al.Toward bridging the simulated-to-real gap:benchmarking super-resolution on real data[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2019,42(11):2944-2959. [22] SCHULTZ R R,STEVENSON R L.A Bayesian approach to image expansion for improved definition[J].IEEE Transactions on Image Processing,1994,3(3):233-242. [23] IRANI M,PELEG S.Super resolution from image sequences[C]//10th International Conference on Pattern Recognition,1990. [24] STARK H,OSKOUI P.High-resolution image recovery from image-plane arrays,using convex projections[J].JOSA A,1989,6(11):1715-1726. [25] MEI Y,FAN Y,ZHOU Y.Image super-resolution with non-local sparse attention[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2021:3517-3526. [26] GUO Y,CHEN J,WANG J,et al.Closed-loop matters:dual regression networks for single image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2020:5407-5416. [27] ZHANG Y,TIAN Y,KONG Y,et al.Residual dense network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2018:2472-2481. [28] LIM B,SON S,KIM H,et al.Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops,2017:136-144. [29] KIM J,LEE J K,LEE K M.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2016:1646-1654. [30] WANG Y,PERAZZI F,MCWILLIAMS B,et al.A fully progressive approach to single-image super-resolution[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW),2018. [31] CHEN H,HE X,QING L,et al.Real-world single image super-resolution:a brief review[J].Information Fusion,2022,79:124-145. [32] ZHANG X,DONG H,HU Z,et al.Gated fusion network for degraded image super resolution[J].International Journal of Computer Vision,2020,128(6):1699-1721. [33] DAI T,CAI J,ZHANG Y,et al.Second-order attention network for single image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2019:11065-11074. [34] HARIS M,SHAKHNAROVICH G,UKITA N.Deep back-projection networks for super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2018:1664-1673. [35] ZHANG K,WANG B,ZUO W,et al.Joint learning of multiple regressors for single image super-resolution[J].IEEE Signal Processing Letters,2015,23(1):102-106. [36] TIMOFTE R,DE SMET V,VAN GOOL L.Anchored neighborhood regression for fast example-based super-resolution[C]//Proceedings of the IEEE International Conference on Computer Vision,2013:1920-1927. [37] ZHANG Y,LI K,LI K,et al.Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the European Conference on Computer Vision(ECCV),2018:286-301. [38] LIU J,ZHANG W,TANG Y,et al.Residual feature aggregation network for image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2020:2359-2368. [39] BELL-KLIGLER S,SHOCHER A,IRANI M.Blind super-resolution kernel estimation using an internal-GAN[C]//Advances in Neural Information Processing Systems,2019. [40] CAI J,ZENG H,YONG H,et al.Toward real-world single image super-resolution:a new benchmark and a new model[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision,2019:3086-3095. [41] WEI P,XIE Z,LU H,et al.Component divide-and-conquer for real-world image super-resolution[C]//European Conference on Computer Vision.Cham:Springer,2020:101-117. [42] ZHANG X,CHEN Q,NG R,et al.Zoom to learn,learn to zoom[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2019:3762-3770. [43] CHEN C,XIONG Z,TIAN X,et al.Camera lens super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2019:1652-1660. [44] IGNATOV A,KOBYSHEV N,TIMOFTE R,et al.Dslr-quality photos on mobile devices with deep convolutional networks[C]//ICCV,2017:3297-3305. [45] LI X,CAO G,ZHANG Y,et al.Combining synthesis sparse with analysis sparse for single image super-resolution[J].Signal Processing:Image Communication,2020,83:115805. [46] SHEIKH H R,SABIR M F,BOVIK A C.A statistical evaluation of recent full reference image quality assessment algorithms[J].IEEE Transactions on image processing,2006,15(11):3440-3451. [47] WANG Z,BOVIK A C,SHEIKH H R,et al.Image quality assessment:from error visibility to structural similarity[J].IEEE Transactions on Image Processing,2004,13(4):600-612. [48] SHEIKH H R,BOVIK A C,DE VECIANA G.An information fidelity criterion for image quality assessment using natural scene statistics[J].IEEE Transactions on Image Processing,2005,14(12):2117-2128. [49] ZHANG R,ISOLA P,EFROS A A,et al.The unreasonable effectiveness of deep features as a perceptual metric[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2018:586-595. [50] EFRAT N,GLASNER D,APARTSIN A,et al.Accurate blur models vs. image priors in single image super-resolution[C]//Proceedings of the IEEE International Conference on Computer Vision,2013:2832-2839. [51] SHAO W Z,ELAD M.Simple,accurate,and robust nonparametric blind super-resolution[M]//Image and graphics.Berlin:Springer,2015:333-348. [52] GU J,LU H,ZUO W,et al.Blind super-resolution with iterative kernel correction[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2019:1604-1613. [53] CORNILLERE V,DJELOUAH A,YIFAN W,et al.Blind image super-resolution with spatially variant degradations[J].ACM Transactions on Graphics(TOG),2019,38(6):1-13. [54] HUANG Y,LI S,WANG L,et al.Unfolding the alternating optimization for blind super resolution[C]//Advances in Neural Information Processing Systems,2020:5632-5643. [55] KIM S Y,SIM H,KIM M.Koalanet:blind super-resolution using kernel-oriented adaptive local adjustment[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2021:10611-10620. [56] ZHOU R,SUSSTRUNK S.Kernel modeling super-resolution on real low-resolution images[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision,2019:2433-2443. [57] JI X,CAO Y,TAI Y,et al.Real-world super-resolution via kernel estimation and noise injection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops,2020:466-467. [58] WANG X,XIE L,DONG C,et al.Real-esrgan:training real-world blind super-resolution with pure synthetic data[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision,2021:1905-1914. [59] LIANG J,SUN G,ZHANG K,et al.Mutual affine network for spatially variant kernel estimation in blind image super-resolution[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision,2021:4096-4105. [60] ZHANG K,LIANG J,VAN GOOL L,et al.Designing a practical degradation model for deep blind image super-resolution[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision,2021:4791-4800. [61] AAKERBERG A,NASROLLAHI K,MOESLUND T B.Real‐world super‐resolution of face‐images from surveillance cameras[J].IET Image Processing,2022,16(2):442-452. [62] WANG L,WANG Y,DONG X,et al.Unsupervised degradation representation learning for blind super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2021:10581-10590. [63] BULAT A,YANG J,TZIMIROPOULOS G.To learn image super-resolution,use a GAN to learn how to do image degradation first[C]//Proceedings of the European Conference on Computer Vision(ECCV),2018:185-200. [64] ZHU J Y,PARK T,ISOLA P,et al.Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE International Conference on Computer Vision,2017:2223-2232. [65] PRAJAPATI K,CHUDASAMA V,PATEL H,et al.Unsupervised single image super-resolution network(USISResNet) for real-world data using generative adversarial network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops,2020:464-465. [66] YOU C,LI G,ZHANG Y,et al.CT super-resolution GAN constrained by the identical,residual,and cycle learning ensemble(GAN-CIRCLE)[J].IEEE Transactions on Medical Imaging,2019,39(1):188-203. [67] MAEDA S.Unpaired image super-resolution using pseudo-supervision[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2020:291-300. [68] LIAN S,ZHOU H,SUN Y.FG-SRGAN:a feature-guided super-resolution generative adversarial network for unpaired image super-resolution[C]//International Symposium on Neural Networks.Cham:Springer,2019:151-161. [69] YOON K.Simple and efficient unpaired real-world super-resolution using image statistics[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision,2021:1983-1990. [70] YUAN Y,LIU S,ZHANG J,et al.Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops,2018:701-710. [71] KIM G,PARK J,LEE K,et al.Unsupervised real-world super resolution with cycle generative adversarial network and domain discriminator[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops,2020:456-457. [72] WANG W,ZHANG H,YUAN Z,et al.Unsupervised real-world super-resolution:a domain adaptation perspective[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision,2021:4318-4327. [73] MOSSERI I,ZONTAK M,IRANI M.Combining the power of internal and external denoising[C]//IEEE International Conference on Computational Photography(ICCP),2013:1-9. [74] ZONTAK M,IRANI M.Internal statistics of a single natural image[C]//CVPR 2011,2011:977-984. [75] GLASNER D,BAGON S,IRANI M.Super-resolution from a single image[C]//IEEE International Conference on Computer Vision,2009. [76] MICHAELI T,IRANI M.Nonparametric blind super-resolution[C]//IEEE International Conference on Computer Vision,2013. [77] SHOCHER A,COHEN N,IRANI M.“Zero-shot” super-resolution using deep internal learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2018:3118-3126. [78] YAMAC M,ATAMAN B,NAWAZ A.Kernelnet:a blind super-resolution kernel estimation network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2021:453-462. [79] EMAD M,PEEMEN M,CORPORAAL H.Dualsr:zero-shot dual learning for real-world super-resolution[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision,2021:1630-1639. [80] SOH J W,CHO S,CHO N I.Meta-transfer learning for zero-shot super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2020:3516-3525. [81] PARK S,YOO J,CHO D,et al.Fast adaptation to super-resolution networks via meta-learning[C]//European Conference on Computer Vision.Cham:Springer,2020:754-769. [82] LI S,ZHANG G,LUO Z,et al.From general to specific:online updating for blind super-resolution[J].Pattern Recognition,2022,127:108613. |
[1] | 陈吉尚, 哈里旦木·阿布都克里木, 梁蕴泽, 阿布都克力木·阿布力孜, 米克拉依·艾山, 郭文强. 深度学习在符号音乐生成中的应用研究综述[J]. 计算机工程与应用, 2023, 59(9): 27-45. |
[2] | 姜秋香, 郭伟鹏, 王子龙, 欧阳兴涛, 隆睿睿. Python语言在水文水资源领域中的应用与展望[J]. 计算机工程与应用, 2023, 59(9): 46-58. |
[3] | 罗会兰, 陈翰. 时空卷积注意力网络用于动作识别[J]. 计算机工程与应用, 2023, 59(9): 150-158. |
[4] | 刘华玲, 皮常鹏, 赵晨宇, 乔梁. 基于深度域适应的跨域目标检测算法综述[J]. 计算机工程与应用, 2023, 59(8): 1-12. |
[5] | 何家峰, 陈宏伟, 骆德汉. 深度学习实时语义分割算法研究综述[J]. 计算机工程与应用, 2023, 59(8): 13-27. |
[6] | 岱超, 刘萍, 史俊才, 任鸿杰. 利用U型网络的遥感影像建筑物规则化提取[J]. 计算机工程与应用, 2023, 59(8): 105-116. |
[7] | 王静, 金玉楚, 郭苹, 胡少毅. 基于深度学习的相机位姿估计方法综述[J]. 计算机工程与应用, 2023, 59(7): 1-14. |
[8] | 蒋玉英, 陈心雨, 李广明, 王飞, 葛宏义. 图神经网络及其在图像处理领域的研究进展[J]. 计算机工程与应用, 2023, 59(7): 15-30. |
[9] | 周玉蓉, 张巧灵, 于广增, 徐伟强. 基于声信号的工业设备故障诊断研究综述[J]. 计算机工程与应用, 2023, 59(7): 51-63. |
[10] | 韦健, 赵旭, 李连鹏. 融合位置信息注意力的孪生弱目标跟踪算法[J]. 计算机工程与应用, 2023, 59(7): 198-206. |
[11] | 赵宏伟, 郑嘉俊, 赵鑫欣, 王胜春, 李浥东. 基于双模态深度学习的钢轨表面缺陷检测方法[J]. 计算机工程与应用, 2023, 59(7): 285-293. |
[12] | 高腾, 张先武, 李柏. 深度学习在安全帽佩戴检测中的应用研究综述[J]. 计算机工程与应用, 2023, 59(6): 13-29. |
[13] | 蒋心璐, 陈天恩, 王聪, 李书琴, 张宏鸣, 赵春江. 农业害虫检测的深度学习算法综述[J]. 计算机工程与应用, 2023, 59(6): 30-44. |
[14] | 江倩殷, 余志, 李熙莹. 标签差网络在噪声标签数据集中的应用[J]. 计算机工程与应用, 2023, 59(6): 92-100. |
[15] | 李宇, 韩晓红, 张玲, 张海轩, 李钢. 融合时空注意力机制的P波到时拾取网络[J]. 计算机工程与应用, 2023, 59(6): 113-124. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||