Multi-scale Image Deblurring Model with DenseNet

doi:10.3778/j.issn.1002-8331.2007-0453

Abstract

Abstract:

Multi-scale convolutional neural network is widely used in the field of image deblurring, but the method of setting network parameters independently at different scales will lead to difficult network training, and lead to problems such as too large parameters, decreased stability and unconstrained solution space. According to the above problems of multi-scale algorithm, an image deblurring algorithm that shares network weight across scales and fuses DenseNet is proposed. In this model, encoder-decoder structure is adopted, and dense blocks are introduced to improve the structure, it can obtain the in-depth characteristic information to the greatest extent. At the same time, the method of sharing network weight across scales is proposed to share parameters in the process of scale iteration, which significantly reduces the training difficulty and significantly improves the stability. The advantages are two fold. Experiments are carried out on GOPRO and Kohler of large scale moving image deblurring data set and blind deblurring data set of image. The results show that the model is superior to the existing methods qualitatively and quantitatively, and superior to other algorithms in both subjective vision and experimental data. Compared with other methods in this field in recent years, this method has the characteristics of simpler network structure, fewer parameters and easier training. The proposed algorithm performs well in subjective and objective evaluation, and can deal with a variety of fuzzy kernels, and has strong robustness, which can be applied to deblurring moving images.

Key words: image deblurring, multi-scale, encoder-decoder, weight shared, denseblock

摘要：

多尺度卷积神经网络被广泛应用在图像去模糊领域，但在不同尺度上对网络参数进行独立设定的方法会导致网络训练难，并且产生参数过大、稳定性降低、无约束解空间等问题。针对多尺度算法存在的上述问题提出了跨尺度共享网络权重并融合DenseNet的图像去模糊算法。该模型采用编码器-解码器结构，并通过引入密集块来改进该结构，从而形成独特的编解码器密集网络，能最大程度获取深层次特征信息。同时提出跨尺度权重共享的方法，使得在尺度迭代的过程中共享参数，显著降低了训练难度，明显提升了稳定性，优势是双重的。将训练所得模型在大规模运动图像去模糊数据集GOPRO和图像盲去模糊数据集Kohler上进行实验，结果表明，该模型在定性和定量条件下明显优于现有方法，并且能够同时在主观视觉和实验数据上优于其他算法。相比近年来该领域出现的其他方法，该方法具有更简单的网络结构、更少的参数和更容易训练等特点。提出的算法在主客观评价上都表现良好，能够处理多种模糊核，鲁棒性强，可应用于运动图像的去模糊处理。

关键词: 图像去模糊, 多尺度, 编解码器, 权重共享, 密集块

Multi-scale Image Deblurring Model with DenseNet. Multi-scale Image Deblurring Model with DenseNet[J]. Computer Engineering and Applications, 2021, 57(24): 219-226.

刘万军，张正寰，曲海成. 融合DenseNet的多尺度图像去模糊模型[J]. 计算机工程与应用, 2021, 57(24): 219-226.

References

[1] KIM T H，AHN B，LEE K M.Dynamic scene deblurring[C]//IEEE International Conference on Computer Vision，2013：3160-3167.
[2] 谭海鹏，曾炫杰，牛四杰.基于正则化约束的遥感图像多尺度去模糊[J].中国图象图形学报，2015，20（3）：386-394.
TAN H P，ZENG X J，NIU S J.Remote sensing image multi-scale deblurring based on regularization constraint[J].Journal of Image and Graphics，2015，20（3）：386-394.
[3] 刘伟豪，梅林，蔡烜，等.稀疏梯度先验模型的正则化图像复原[J].中国图象图形学报，2012，17（12）：1485-1491.
LIU W H，MEI L，CAI X.Regularized image restoration algorithm on sparse gradient prior model[J].Journal of Image and Graphics，2012，17（12）：1485-1491.
[4] PAN J，SUN D，PFISTER H.Blind image deblurring using dark channel prior[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2016.
[5] SUN J，CAO W，XU Z.Learning a convolutional neural network for non-uniform motion blur removal[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2015：769-777.
[6] NAH S，KIM T H，LEE K M.Deep multi-scale convolutional neural network for dynamic scene deblurring[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2017：3883-3891.
[7] SCHULER C，HIRSCH M，HARMELING S.Learning to deblur[J].IEEE Transactions on Pattern Analysis & Machine Intelligence，2016，38（7）：1439-1451.
[8] CHEN Q，KOLTUN V.Photographic image synthesis with cascaded refinement networks[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：1511-1520.
[9] MAO X，SHEN C，YANG Y B.Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections[C]//Advances in Neural Information Processing Systems，2016：2802-2810.
[10] XU N，PRICE B，COHEN S.Deep image matting[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2017：2970-2979.
[11] TAO X，GAO H，LIAO R.Detail-revealing deep video super-resolution[C]//2017 IEEE International Conference on Computer Vision（ICCV），2017：4472-4480.
[12] HUANG G，LIU Z，VAN DER MAATEN L，et al.Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：4700-4708.
[13] SHI X J，CHEN Z，WANG H，et al.Convolutional LSTM network：a machine learning approach for precipitation nowcasting[C]//Advances in Neural Information Processing Systems，2015：802-810.
[14] SHI W，CABALLERO J，HUSZáR F，et al.Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：1874-1883.
[15] CHAKRABARTI A.A neural approach to blind motion deblurring[C]//European Conference on Computer Vision.Cham：Springer，2016：221-235.
[16] ABADI M，BARHAM P，CHEN J，et al.Tensorflow：a system for large-scale machine learning[C]//12th {USENIX} Symposium on Operating Systems Design and Implementation，2016：265-283.
[17] K?HLER R，HIRSCH M，MOHLER B.Recording and playback of camera shake：benchmarking blind deconvolution with a real-world database[C]//European Conference on Computer Vision，2012：27-40.
[18] LAI W S，HUANG J B，HU Z，et al.A comparative study for single image blind deblurring[C]//IEEE Conference on Computer Vision and Pattern Recognition，2016.