计算机工程与应用 ›› 2024, Vol. 60 ›› Issue (16): 49-60.DOI: 10.3778/j.issn.1002-8331.2403-0230
朱新峰,宋健
出版日期:
2024-08-15
发布日期:
2024-08-15
ZHU Xinfeng, SONG Jian
Online:
2024-08-15
Published:
2024-08-15
摘要: 基于深度学习的图像超分辨率(super-resolution,SR)受到广泛关注,其目的是提高图像的分辨率,以便对图像做进一步的处理,如目标检测、图像分类和人脸识别等。图像SR领域相关研究近年来取得了迅猛发展,但有关轻量级SR模型的相关综述还不多见。对基于深度学习的轻量级SR方法研究现状和损失函数进行了分析,并对目前轻量级SR方法进行了新的分类,分别为传统卷积方法和注意力机制方法。系统梳理了图像轻量级SR方法的发展历程和最新进展,指出了每一种方法存在的优势和缺陷。最后对当前轻量级SR技术存在的问题进行了分析,并给出了轻量级图像SR方法未来的研究方向。
朱新峰, 宋健. 轻量级图像超分辨率研究综述[J]. 计算机工程与应用, 2024, 60(16): 49-60.
ZHU Xinfeng, SONG Jian. Review of Research on Lightweight Image Super-Resolution[J]. Computer Engineering and Applications, 2024, 60(16): 49-60.
[1] 李云红, 王珍, 张凯兵, 等. 基于学习的图像超分辨重建方法综述[J]. 计算机工程与应用, 2018, 54(15): 13-21. LI Y H, WANG Z, ZHANG K B, et al. Survey on example learning-based single image super-resolution technique[J]. Computer Engineering and Applications, 2018, 54(15): 13-21. [2] 孙旭, 李晓光, 李嘉锋, 等. 基于深度学习的图像超分辨率复原研究进展[J]. 自动化学报, 2017, 43(5): 697-709. SUN X, LI X G, LI J F, et al. Review on deep learning based image super-resolution restoration algorithms[J] Acta Automatica Sinica, 2017, 43(5): 697-709. [3] DONG C, LOY C C, HE K, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295-307. [4] YANG W M, ZHANG X C, TIAN Y P, et al. Deep learning for single image super-resolution: a brief review[J]. IEEE Transactions on Multimedia, 2019, 21(12): 3106-3121. [5] CHEN H G, HE X H, QING L B, et al. Real-world single image super-resolution: a brief review[J]. Information Fusion, 2022, 79: 124-145. [6] JIANG J J, WANG C Y, LIU X M, et al. Deep learning-based face super-resolution: a survey[J]. ACM Computing Surveys, 2023, 55(1): 13. [7] LIU A R, LIU Y H, GU J J, et al. Blind image super-resolution: a survey and beyond[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(5): 5461-5480. [8] 杨才东, 李承阳, 李忠博, 等. 深度学习的图像超分辨率重建技术综述[J]. 计算机科学与探索, 2022, 16(9): 1990-2010. YANG C D, LI C Y, LI Z B, et al. Review of image super-resolution reconstruction algorithms based on deep learning[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 1990-2010. [9] GENDY G, HE G, SABOR N. Lightweight image super-resolution based on deep learning: state-of-the-art and future directions[J]. Information Fusion, 2023, 94: 284-310. [10] DONG C, LOY C C, TANG X. Accelerating the super-resolution convolutional neural network[C]//Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 11-14, 2016. Cham: Springer, 2016: 391-407. [11] SHI W, CABALLERO J, HUSZAR F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1874-1883. [12] KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1646-1654. [13] KIM J, LI J K, LI K K M. Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1637-1645. [14] LI H L, LIU Z H, LIU Y, et al. Lightweight image super-resolution network using 3D convolutional neural networks[J]. Journal of Electronic Imaging, 2024, 33(1): 013016. [15] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778. [16] LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017: 136-144. [17] YING T, JIAN Y, XIAOMING L. Image super-resolution via deep recursive residual network[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2790-2798. [18] AHN N, KANG B, SOHN K A. Fast, accurate, and lightweight super-resolution with cascading residual network[C]//Proceedings of the 15th European Conference on Computer Vision, 2018: 252-268. [19] ZHANG Y, TIAN Y, KONG Y, et al. Residual dense network for image restoration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(7): 2480-2495. [20] SUN B, ZHANG Y, JIANG S, et al. Hybrid pixel-unshuffled network for lightweight image super?resolution[C]//Proceedings of the 37th AAAI Conference on Artificial Intelligence, 2023: 2375-2383. [21] ZHANG Y, BAI H M, BING Y X, et al. Lightweight image super-resolution with ConvNeXt residual network[J]. Neural Processing Letters, 2023, 55(7): 9545-9561. [22] GAO H, ZHUANG L, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2261-2269. [23] TONG T, LI G, LIU X, et al. Image super-resolution using dense skip connections[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, 2017: 4799-4807. [24] FENG Z, LAI J, XIE X, et al. Image super-resolution via a densely connected recursive network[J]. Neurocomputing, 2018, 316: 270-276. [25] ANWAR S, BARNES N. Densely residual laplacian super-resolution[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(3): 1192-1204. [26] LV X, WANG C Z, FAN X D, et al. A novel image super-resolution algorithm based on multi-scale dense recursive fusion network[J]. Neurocomputing, 2022, 489: 98-111. [27] HINTON G, VINYALS O, DEAN J. Distilling the knowledge in a neural network[J]. arXiv:1503.02531, 2015. [28] GOU J, YU B, MAYBANK S J, et al. Knowledge distillation: a survey[J]. International Journal of Computer Vision, 2021, 129(6): 1789-1819. [29] ZHENG H, XIUMEI W, XINBO G. Fast and accurate single image super-resolution via information distillation network[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 723-731. [30] YANG X, GUO Y, LI Z, et al. MRDN: a lightweight multi-stage residual distillation network for image super-resolution[J]. Expert Systems with Applications, 2022, 204: 117594. [31] TIAN C, YUAN Y, ZHANG S, et al. Image super-resolution with an enhanced group convolutional neural network[J]. Neural Networks, 2022, 153: 373-385. [32] GAO X, ZHOU Y, WU S N, et al. Residual multi-branch distillation network for efficient image super-resolution[J]. Multimedia Tools and Applications, 2024. DOI:10.1007/s11042-024-18504-5. [33] GAO Q, ZHAO Y, LI G, et al. Image super-resolution using knowledge distillation[C]//Proceedings of the 2018 Asian Conference on Computer Vision. Cham: Springer, 2018: 527-541. [34] HE Z, DAI T, LU J, et al. FAKD: feature-affinity based knowledge distillation for efficient image super-resolution[C]//Proceedings of the 2020 IEEE International Conference on Image Processing, 2020: 518-522. [35] WANG C, CHEN D, MEI J P, et al. SemCKD: semantic calibration for cross-layer knowledge distillation[J]. IEEE Transactions on Knowledge and Data Engineering, 2023, 35(6): 6305-6319. [36] FANG H X, LONG Y W, HU X Y, et al. Dual cross knowledge distillation for image super-resolution[J]. Journal of Visual Communication and Image Representation, 2023, 95: 103858. [37] FAN Y C, SHI H H, YU J H, et al. Balanced two-stage residual networks for image super-resolution[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition: Workshops, 2017: 1157-1164. [38] LIU H, CAO F L, WEN C L, et al. Lightweight multi-scale residual networks with attention for image super-resolution[J]. Knowledge-Based Systems, 2020, 203: 106103. [39] ZONG Z, ZHA L, JIANG J, et al. Asymmetric information distillation network for lightweight super resolution[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 1249-1258. [40] HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023. [41] ZHANG Y L, LI K P, LI K, et al. Image super-resolution using very deep residual channel attention networks[C]//LNCS 11211: Proceedings of the 15th European Conference on Computer Vision, 2018: 294-310. [42] YANG A P, YANG B W, JI Z, et al. Lightweight group convolutional network for single image super-resolution[J]. Information Sciences, 2020, 516: 220-233. [43] PENG C M, SHU P, HUANG X Y, et al. LCRCA: image super-resolution using lightweight concatenated residual channel attention networks[J]. Applied Intelligence, 2022, 52(9): 10045-10059. [44] LU J, LUO X. Image super-resolution with enhanced channel attention residual network[C]//Proceedings of the 2023 8th International Conference on Intelligent Computing and Signal Processing, 2023: 16501-16504. [45] YASIR M, ULLAH I, CHOI C. Depthwise channel attention network (DWCAN): an efficient and lightweight model for single image super-resolution and metaverse gaming[J]. Expert Systems, 2024, 41(4): e13516. [46] SOH J W, CHO N I. Lightweight single image super-resolution with multi-scale spatial attention networks[J]. IEEE Access, 2020, 8: 35383-35391. [47] CAI Y X, LAI H C, JIA Z H, et al. Lightweight spatial-channel adaptive coordination of multilevel refinement enhancement network for image reconstruction[J]. Knowledge-Based Systems, 2022, 256: 109824. [48] CHEN J C, WANG W L, XING F S, et al. Multi-feature fusion attention network for single image super-resolution[J]. IET Image Processing, 2023, 17(5): 1389-1402. [49] TOMAR A S, ARYA K V, RAJPUT S S. Noise robust face super-resolution via learning of spatial attentive features[J]. Multimedia Tools and Applications, 2023, 82(16): 25449-25465. [50] ZHAO H, KONG X, HE J, et al. Efficient image super-resolution using pixel attention[C]//Proceedings of the 16th European Conference on Computer Vision Workshops, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 56-72. [51] ZHAO K H, LU T, ZHANG Y D, et al. Face super-resolution via triple-attention feature fusion network[J]. IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences, 2022, E105A(4): 748-752. [52] WANG X, ZHANG S F, LIN Y Y, et al. Pixel attention convolutional network for image super-resolution[J]. Neural Computing & Applications, 2023, 35(11): 8589-8599. [53] JIANG L F,ZHONG M Z,QIU F C. Single-image super-resolution based on a self-attention deep neural network[C]//Proceedings of the 2020 13th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics, 2020: 387-391. [54] LIU Y H, CHU Z Z, LI B. A local and non-local features based feedback network on super-resolution[J]. Sensors, 2022, 22(24): 9604. [55] ZENG K, LIN H J, YAN Z Q, et al. Densely connected transformer with linear self-attention for lightweight image super-resolution[J]. IEEE Transactions on Instrumentation and Measurement, 2023, 72: 1-12. [56] PARK K, SOH J W, CHO N I. A dynamic residual self-attention network for lightweight single image super-resolution[J]. IEEE Transactions on Multimedia, 2023, 25: 907-918. [57] HAN X W, WANG L, WANG X P, et al. A multi-scale recursive attention feature fusion network for image super-resolution reconstruction algorithm[J]. Sensors, 2023, 23(23): 9458. [58] LI H A, WANG D, ZHANG J, et al. Image super-resolution reconstruction based on multi-scale dual-attention[J]. Connection Science, 2023, 35(1): 2182487. [59] XIE F, LU P, LIU X Y. Multi-scale convolutional attention network for lightweight image super-resolution[J]. Journal of Visual Communication and Image Representation, 2023, 95: 103889. [60] WU H P, GUI J, ZHANG J, et al. Pyramidal dense attention networks for single image super-resolution[J]. IET Image Processing, 2022, 16(12): 3247-3257. [61] GENDY G, SABOR N, HOU J C, et al. Balanced spatial feature distillation and pyramid attention network for lightweight image super-resolution[J]. Neurocomputing, 2022, 509: 157-166. [62] CHEN S, BI X P, ZHANG L F. Fused pyramid attention network for single image super-resolution[J]. IET Image Processing, 2023, 17(6): 1681-1693. [63] MEI Y Q, FAN Y C, ZHANG Y L, et al. Pyramid attention network for image restoration[J]. International Journal of Computer Vision, 2023, 131(12): 3207-3225. [64] LAI W S, HUANG J B, AHUJA N, et al. Fast and accurate image super-resolution with deep laplacian pyramid networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(11): 2599-2613. [65] WU Q, FAN C X, LI Y, et al. A novel perceptual loss function for single image super-resolution[J]. Multimedia Tools and Applications, 2020, 79(29/30): 21265-21278. [66] JIANG D H, SAI Z, LEI D, et al. Multi-scale generative adversarial network for image super-resolution[J]. Soft Computing, 2022, 26(8): 3631-3641. [67] 吴靖, 叶晓晶, 黄峰, 等. 基于深度学习的单帧图像超分辨率重建综述[J]. 电子学报, 2022, 50(9): 2265-2294. WU J, YE X J, HUANG F, et al. A review of single image super-resolution reconstruction based on deep learning[J]. Acta Electronica Sinica, 2022, 50(9): 2265-2294. [68] RADOSAVOVIC I, KOSARAJU R P, GIRSHICK R, et al. Designing network design spaces[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 10428-10436. |
[1] | 王彩玲, 闫晶晶, 张智栋. 基于多模态数据的人体行为识别方法研究综述[J]. 计算机工程与应用, 2024, 60(9): 1-18. |
[2] | 廉露, 田启川, 谭润, 张晓行. 基于神经网络的图像风格迁移研究进展[J]. 计算机工程与应用, 2024, 60(9): 30-47. |
[3] | 杨晨曦, 庄旭菲, 陈俊楠, 李衡. 基于深度学习的公交行驶轨迹预测研究综述[J]. 计算机工程与应用, 2024, 60(9): 65-78. |
[4] | 史涛, 崔杰, 李松. 优化改进YOLOv8实现实时无人机车辆检测的算法[J]. 计算机工程与应用, 2024, 60(9): 79-89. |
[5] | 窦智, 高浩然, 刘国奇, 常宝方. 轻量化YOLOv8的小样本钢板缺陷检测算法[J]. 计算机工程与应用, 2024, 60(9): 90-100. |
[6] | 蔡腾, 陈慈发, 董方敏. 结合Transformer和动态特征融合的低照度目标检测[J]. 计算机工程与应用, 2024, 60(9): 135-141. |
[7] | 许智宏, 张天润, 王利琴, 董永峰. 融合图谱重构的时序知识图谱推理[J]. 计算机工程与应用, 2024, 60(9): 181-187. |
[8] | 宋建平, 王毅, 孙开伟, 刘期烈. 结合双曲图注意力网络与标签信息的短文本分类方法[J]. 计算机工程与应用, 2024, 60(9): 188-195. |
[9] | 杨文涛, 雷雨琦, 李星月, 郑天成. 融合汉字输入法的BERT与BLCG的长文本分类研究[J]. 计算机工程与应用, 2024, 60(9): 196-202. |
[10] | 张洋宁, 朱静, 董瑞, 尤泽顺, 王震. 多层级信息增强异构图的篇章级话题分割模型[J]. 计算机工程与应用, 2024, 60(9): 203-211. |
[11] | 陶林娟, 华庚兴, 李波. 基于位置增强词向量和GRU-CNN的方面级情感分析模型研究[J]. 计算机工程与应用, 2024, 60(9): 212-218. |
[12] | 江结林, 朱永伟, 许小龙, 崔燕, 赵英男. 混合特征及多头注意力的中文短文本分类[J]. 计算机工程与应用, 2024, 60(9): 237-243. |
[13] | 车运龙, 袁亮, 孙丽慧. 基于强语义关键点采样的三维目标检测方法[J]. 计算机工程与应用, 2024, 60(9): 254-260. |
[14] | 邱云飞, 王宜帆. 双分支结构的多层级三维点云补全[J]. 计算机工程与应用, 2024, 60(9): 272-282. |
[15] | 叶彬, 朱兴帅, 姚康, 丁上上, 付威威. 面向桌面交互场景的双目深度测量方法[J]. 计算机工程与应用, 2024, 60(9): 283-291. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||