Snow and Haze Elimination Model Based on Function Decoupling and Edge Feature Fusion

doi:10.3778/j.issn.1002-8331.2203-0566

Abstract

Abstract: Aiming at the problem of snowflake occlusion and haze veil effect in the collected images by the vehicle-mounted camera for the influence of snowflakes and haze, this paper proposes a decoupling and double-supervised snow haze elimination network based on the idea of image edge texture and image color separation and reconstruction. The proposed algorithm separates and reconstructs image edge texture and color information, decouples the snow haze removal task into two sub-tasks, background texture inpainting and color reconstruction, and uses dual generative adversarial networks to reconstruct edge texture and color features separately. The algorithm has tested on the SRRS-6000 dataset, which verifies the effectiveness of dual supervision on network acceleration convergence and the significant effect for noise removal. The peak signal to noise ratio（PSNR） and structural similarity（SSIM） has reached 33.29 dB and 0.94, 32.8?dB and 0.931 6, 30.13 dB and 0.93, 25.88 dB and 0.82 on Snow100K-S, Snow100K-M, Snow100K-L and I&O-Haze datasets, respectively. Experimental results show that by method of decoupling and double supervision in image denoising task, efficient snowflake and haze removal performance have achieved, and the adaptability of unmanned assistance system under complex weather conditions have been enhanced.

Key words: generative adversarial network, desnow, dehaze, noise removal, image denoising

摘要： 针对车载相机受雪花、雾霾影响，导致采集图像出现雪花遮挡和雾霾面纱效应问题，基于图像边缘纹理和图像色彩分离重建的思想，提出功能解耦、双重监督的雪霾消除网络。所提算法通过对图像边缘纹理和色彩信息进行分离重建，将雪霾消除任务解耦为背景纹理修复与色彩重建两个子任务，并用双生成对抗网络分别进行边缘纹理和色彩特征的协同重建。算法在SRRS-6000数据集上进行消融测试，验证了双重监督对网络加速收敛的有效性和噪声消除的显著效果，模型在Snow100K-S、Snow100K-M、Snow100K-L、I&O-Haze数据集上进行测试，峰值信噪比（peak signal to noise ratio，PSNR）和结构相似性（structural similarity，SSIM）分别达到33.29?dB和0.94、32.8?dB和0.931?6、30.13?dB和0.93、25.88?dB和0.82。实验结果表明，通过对图像去噪任务进行解耦和双重监督，取得了高效的雪花、雾霾消除效果，增强了无人驾驶辅助系统在复杂天气条件下的适应性。

关键词: 生成对抗网络, 去雪, 去雾, 噪声消除, 图像去噪

BAO Xianfu, QIANG Zanxia, YANG Guan. Snow and Haze Elimination Model Based on Function Decoupling and Edge Feature Fusion[J]. Computer Engineering and Applications, 2023, 59(13): 211-219.

鲍先富, 强赞霞, 杨关. 功能解耦和谱特征融合的雪霾消除模型[J]. 计算机工程与应用, 2023, 59(13): 211-219.

References

[1] ZHENG X，LIAO Y，GUO W，et al.Single-image-based rain and snow removal using multi-guided filter[C]//Proceedings of International Conference on Neural Information Processing，2013：258-265.
[2] WANG Y，LIU S，CHEN C，et al.A hierarchical approach for rain or snow removing in a single color image[J].IEEE Transactions on Image Processing，2017，26（8）：3936-3950.
[3] LI Z，ZHANG J，FANG Z，et al.Single image snow removal via composition generative adversarial networks[J].IEEE Access，2019，7：25016-25025.
[4] LIU Y F，JAW D W，HUANG S C，et al.DesnowNet：context-aware deep network for snow removal[J].IEEE Transactions on Image Processing，2018，27（6）：3064-3073.
[5] CHEN W T，FANG H Y，DING J J，et al.JSTASR：joint size and transparency-aware snow removal algorithm based on modified partial convolution and veiling effect removal[C]//Proceedings of European Conference on Computer Vision，2020：754-770.
[6] 郑凤仙，王夏黎，何丹丹，等.单幅图像去雾算法研究综述[J].计算机工程与应用，2022，58（3）：1-14.
ZHENG F X，WANG X L，HE D D，et al.Survey of single image defogging algorithm[J].Computer Engineering and Applications，2022，58（3）：1-14.
[7] LIZUKA S，SIMO-SERRA E，ISHIKAWA H.Globally and locally consistent image completion[J].ACM Transactions on Graphics（ToG），2017，36（4）：1-14.
[8] YANG C，LU X，LIN Z，et al.High-resolution image inpainting using multi-scale neural patch synthesis[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：6721-6729.
[9] LI H，LI J，ZHAO D，et al.DehazeFlow：multi-scale conditional flow network for single image dehazing[C]//Proceedings of the 29th ACM International Conference on Multimedia，2021：2577-2585.
[10] ARJOVSKY M，CHINTALA S，BOTTOU L.Wasserstein generative adversarial networks[C]//Proceedings of International Conference on Machine Learning（PMLR），2017：214-223.
[11] 魏富强，古兰拜尔·吐尔洪，买日旦·吾守尔.生成对抗网络及其应用研究综述[J].计算机工程与应用，2021，57（19）：18-31.
WEI F Q，TUERHONG G，WUSHOUER M.Review of research on generative adversarial networks and its application[J].Computer Engineering and Applications，2021，57（19）：18-31.
[12] DOLHANSKY B，FERRER C C.Eye in-painting with exemplar generative adversarial networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：7902-7911.
[13] LIANG T，JIN Y，LI Y，et al.EDCNN：edge enhancement-based densely connected network with compound loss for low-dose CT denoising[C]//Proceedings of the 15th IEEE International Conference on Signal Processing（ICSP），2020：193-198.
[14] SONG Y，YANG C，LIN Z，et al.Contextual-based image inpainting：infer，match，and translate[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：3-19.
[15] TAO S，SHEN C，ZHU L，et al.SVD-CNN：a convolutional neural network model with orthogonal constraints based on SVD for context-aware citation recommendation[J].Computational Intelligence and Neuroscience，2020，2020：5343214.
[16] ANCUTI C，ANCUTI C O，TIMOFTE R，et al.I-HAZE：a dehazing benchmark with real hazy and haze-free indoor images[C]//Proceedings of the International Conference on Advanced Concepts for Intelligent Vision Systems，2018：620-631.
[17] ANCUTI C O，ANCUTI C，TIMOFTE R，et al.O-haze：a dehazing benchmark with real hazy and haze-free outdoor images[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops，2018：754-762.
[18] FU X，HUANG J，DING X，et al.Clearing the skies：a deep network architecture for single-image rain removal[J].IEEE Transactions on Image Processing，2017，26（6）：2944-2956.
[19] CAI B，XU X，JIA K，et al.Dehazenet：an end-to-end system for single image haze removal[J].IEEE Transactions on Image Processing，2016，25（11）：5187-5198.
[20] BHARATH R N，VENKETESWARAN N.Single image haze removal using a generative adversarial network[J].arXiv：1810.09479，2018.
[21] LIU X，SUGANUMA M，SUN Z，et al.Dual residual networks leveraging the potential of paired operations for image restoration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：7007-7016.
[22] DONG H，PAN J，XIANG L，et al.Multi-scale boosted dehazing network with dense feature fusion[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：2157-2167.
[23] HONG M，XIE Y，LI C，et al.Distilling image dehazing with heterogeneous task imitation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：3462-3471.