Research on Multi-Branch Image Denoising Algorithm

doi:10.3778/j.issn.1002-8331.2207-0146

Abstract

Abstract: In recent years, deep convolutional neural network（CNN） has caused a great sensation in the field of image denoising. However, for the Gaussian image denoising task, it has some problems：（1） most of the single-branch models can not make full use of image features and are often affected by information loss. （2） Most deep CNNs have the problem of insufficient edge feature extraction and performance saturation. In order to solve these two problems, a multi-branch network model based on deep learning （MBNet） is proposed. Firstly, in order to solve the problem of insufficient feature extraction of single-branch network model, MBNet introduces multiple different and complementary networks to combine and then perform feature fusion to enhance the denoising effect and generalization ability. Secondly, in order to solve the problem of inadequate edge feature extraction, MBNet introduces multiple cavity convolution with different expansion rates to increase the receptive field and extract more context information. Finally, in order to solve the performance saturation problem of deep CNN, MBNet also adopts multi-local residual learning and global residual learning. A large number of experimental results show that when [σ=15], the average PSNR values of MBNet on Set12, BSD68, CBSD68, Kodak24 and McMaster datasets are 32.981 dB, 31.750 dB, 34.001 dB, 34.709 dB and 34.394 dB, respectively. MBNet has better performance than the current advanced image denoising methods, and can obtain clearer image and edge texture features in subjective visual effects.

Key words: convolutional neural network, residual learning, batch normalization, dilated convolution

摘要： 近些年，深卷积神经网络（CNN）在图像去噪领域引起了极大的轰动。然而，对于高斯图像去噪任务来说，它有一些问题：（1）绝大多数的单分支模型不能充分地利用图像特征，经常受到信息丢失的影响；（2）大多数深度CNN存在边缘特征提取不足且性能饱和问题。为了解决这两个问题，提出了基于深度学习的多分支网络模型（multi-branch network model based on deep learning，MBNet）。为了解决单分支网络模型提取特征不充分问题，MBNet引入多个不同且互补的网络相结合再进行特征融合来增强去噪效果和泛化能力；为了解决边缘特征提取不充分问题，MBNet引入多个不同扩张率的空洞卷积来增大感受野，提取更多的上下文信息；为了解决深度CNN性能饱和问题，MBNet还采用了多局部残差学习和整体残差学习的方式。大量实验结果表明，当[σ=15]时，MBNet在Set12、BSD68、CBSD68、Kodak24、McMaster数据集上的平均PSNR值分别为32.981?dB、31.750?dB、34.001?dB、34.709?dB、34.394?dB。MBNet比目前先进的图像去噪方法具有更好的性能，并且在主观视觉效果上得到更加清晰的图像和边缘纹理特征。

关键词: 卷积神经网络, 残差学习, 批量归一化, 空洞卷积

GENG Jun, LI Wenhai, WU Zihao, SUN Xinjie. Research on Multi-Branch Image Denoising Algorithm[J]. Computer Engineering and Applications, 2023, 59(24): 196-208.

耿俊, 李文海, 吴子豪, 孙鑫杰. 多分支图像去噪算法研究[J]. 计算机工程与应用, 2023, 59(24): 196-208.

References

[1] ZHU H，JIANG Y，ZHANG C，et al.Research on mosaic method of UAV low-altitude remote sensing image based on SIFT and SURF[J].Journal of Physics：Conference Series，2022，2203（1）：012027.
[2] WANG G，LI W，GAO X，et al.Multimodal medical image fusion based on multichannel coupled neural P systems and max-cloud models in spectral total variation domain[J].Neurocomputing，2022，480：61-75.
[3] DABOV K，FOI A，KATKOVNIK V，et al.Image denoising by sparse 3-D transform-domain collaborative filtering[J].IEEE Transactions on Image Processing，2007，16（8）：2080-2095.
[4] MAIRAL J，BACH F，PONCE J，et al.Non-local sparse models for image restoration[C]//2009 IEEE 12th International Conference on Computer Vision，2009：2272-2279.
[5] DONG W，ZHANG L，SHI G，et al.Nonlocally centralized sparse representation for image restoration[J].IEEE Transactions on Image Processing，2012，22（4）：1620-1630.
[6] GU S，ZHANG L，ZUO W，et al.Weighted nuclear norm minimizationwithapplication to image denoising[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2014：2862-2869.
[7] MALFAIT M，ROOSE D.Wavelet-based image denoising using a Markov random field a priori model[J].IEEE Transactions on Image Processing，1997，6（4）：549-565.
[8] LAI W S，HUANG J B，AHUJA N，et al.Deep laplacian pyramid networks for fast and accurate super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：624-632.
[9] WANG Y，LI X，NAN F，et al.Image super-resolution reconstruction based on generative adversarial network model with feedback and attention mechanisms[J].Multimedia Tools and Applications，2022，81（5）：6633-6652.
[10] REN C，HE X，PU Y.Nonlocal similarity modeling and deep CNN gradient prior for super resolution[J].IEEE Signal Processing Letters，2018，25（7）：916-920.
[11] ZHANG Y，TIAN Y，KONG Y，et al.Residual dense network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：2472-2481.
[12] ZHANG K，ZUO W，GU S，et al.Learning deep CNN denoiser prior for image restoration[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：3929-3938.
[13] JIAO J，TU W C，HE S，et al.Formresnet：formatted residual learning for image restoration[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops，2017：38-46.
[14] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[15] CHEN L C，PAPANDREOU G，SCHROFF F，et al.Rethinking atrous convolution for semantic image segmentation[J].arXiv：1706.05587，2017.
[16] YU F，KOLTUN V.Multi-scale context aggregation by dilatedconvolutions[J].arXiv：1511.07122，2015.
[17] ZHANG K，ZUO W，CHEN Y，et al.Beyond a Gaussian denoiser：residual learning of deep CNN for image denoising[J].IEEE Transactions on Image Processing，2017，26（7）：3142-3155.
[18] ZHANG K，ZUO W，ZHANG L.FFDNet：toward a fast and flexible solution for CNN-based image denoising[J].IEEE Transactions on Image Processing，2018，27（9）：4608-4622.
[19] TIAN C，XU Y，ZUO W.Image denoising using deep CNN with batch renormalization[J].Neural Networks，2020，121：461-473.
[20] ZHANG K，LI Y，ZUO W，et al.Plug-and-play image restoration with deep denoiser prior[J].IEEE Transactions on Pattern Analysis & Machine Intelligence，2022，44（10）：6360-6376.
[21] ZHAO M，CAO G，HUANG X，et al.Hybrid transformer-CNN for real image denoising[J].IEEE Signal Processing Letters，2022，29：1252-1256.
[22] ZHANG Q，XIAO J，TIAN C，et al.A robust deformed convolutional neural network（CNN） for image denoising[J].CAAI Transactions on Intelligence Technology，2022，6：101049.
[23] TIAN C，XU Y，ZUO W，et al.Designing and training of a dual CNN for image denoising[J].Knowledge-Based Systems，2021，226：106949.
[24] IOFFE S，SZEGEDY C.Batch normalization：accelerating deep network training by reducing internal covariate shift[C]//International Conference on Machine Learning，2015：448-456.
[25] DUBNOV Y A.The feature selection method based on a probabilistic approach and a cross-entropy metric for the image recognition problem[J].Scientific and Technical Information Processing，2021，48（6）：430-435.
[26] LIN G，MILAN A，SHEN C，et al.Refinenet：multi-path refinement networks for high-resolution semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：1925-1934.
[27] DUBEY A，SANTRA A，FUCHS J，et al.HARadNet：anchor-free target detection for radar point clouds using hierarchical attention and multi-task learning[J].Machine Learning with Applications，2022，8：100275.
[28] VENKATA LAVANYA P，VENKATA NARASIMHULU C，SATYA PRASAD K.Image denoising using an artificial neural network and genetic optimization algorithm based dual-tree complex wavelet transform[M]//Soft computing and signal processing.Singapore：Springer，2022：393-401.
[29] WANG T，SUN M，HU K.Dilated deep residual network for image denoising[C]//2017 IEEE 29th International Conference on Tools with Artificial Intelligence（ICTAI），2017：1272-1279.
[30] TAI Y，YANG J，LIU X.Image super-resolution via deep recursive residual network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：3147-3155.
[31] ZORAN D，WEISS Y.From learning models of natural image patches to whole image restoration[C]//2011 International Conference on Computer Vision，2011：479-486.
[32] KINGMA D P，BA J.Adam：a method for stochastic optimization[J].arXiv：1412.6980，2014.
[33] BURGER H C，SCHULER C J，HARMELING S.Image denoising：can plain neural networks compete with BM3D?[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition，2012：2392-2399.
[34] CHEN Y，POCK T.Trainable nonlinear reaction diffusion：a flexible framework for fast and effective image restoration[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2016，39（6）：1256-1272.
[35] SCHMIDT U，ROTH S.Shrinkage fields for effective image restoration[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2014：2774-2781.
[36] ZHANG K，ZUO W，ZHANG L.Learning a single convolutional super-resolution network for multiple degradations[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：3262-3271.