Omni-Frequency Image Denoising with Multi-Head Attention

doi:10.3778/j.issn.1002-8331.2311-0430

Abstract

Abstract: In recent years, deep convolutional neural networks (CNN) have achieved significant results in the field of image denoising. However, most existing denoising methods directly input noisy images into CNN models for training, relying on cropping a large number of image training blocks. The repeatedly cropped regions not only waste computing resources, but also limit the diversity of feature extraction, resulting in the loss of image texture details. To address these issues, this paper proposes an omni-frequency enhanced multi-head attention network (OEMANet) for removing additive Gaussian white noise and real-world noise. The noisy image is decomposed into low- and high-frequency components, these two components and the noisy image are then simultaneously input into OEMANet for training. By increasing the network width, richer image features are extracted. An enhanced multi-head attention mechanism focuses on features at the image level, and recovers more texture details. To obtain accurate noise mappings, a noise learning module is used to remove redundant features and optimize the remaining features of the image. In this paper, the effectiveness of OEMANet is verified on multiple datasets such as Set12 and CBSD68. The experimental results show that the method proposed in this paper is superior to mainstream denoising methods such as ADNet, AMDNet, MWDCNN in terms of grayscale noise image denoising, color noise image denoising, and real image denoising. Moreover, the image denoised by OEMANet has a clearer visual performance.

Key words: image denoising, Gaussian noise, multi-head attention, residual learning, frequency decomposition

摘要： 近年来，深度卷积神经网络（convolutional neural network，CNN）在图像去噪领域取得了显著成果。然而，现有的大部分去噪方法都是将噪声图像直接输入CNN模型训练，依赖于裁剪大量的图像训练块，重复裁剪的区域不仅浪费计算资源，还限制了特征提取的多样性，导致图像纹理细节的丢失。为解决这些问题，提出一种适用于去除加性高斯白噪声和真实图像噪声的全频增强多头注意力去噪网络。该方法将噪声图像分解为低频和高频分量，并与噪声图像一起输入网络进行训练，通过增加网络宽度来提取更丰富的图像特征。特征增强多头注意力机制关注图像级别的特征，能够保留更多的纹理细节。为了得到干净噪声映射，还设计了噪声学习模块来去除冗余特征并优化图像的残差特征。在Set12、CBSD68等多个数据集上验证了所提出方法的有效性。实验结果显示，该方法在灰度噪声图像去噪、彩色噪声图像去噪以及真实图像去噪方面均优于ADNet、AMDNet、MWDCNN等主流去噪方法，而且使用该方法去噪后的图像具有更清晰的视觉效果。

关键词: 图像去噪, 高斯噪声, 多头注意力, 残差学习, 频率分解

JIANG Jielin, SHI Mingyue, YANG Haidong, CUI Yan. Omni-Frequency Image Denoising with Multi-Head Attention[J]. Computer Engineering and Applications, 2024, 60(16): 236-247.

江结林, 史明月, 杨海东, 崔燕. 多头注意力机制的全频图像去噪算法[J]. 计算机工程与应用, 2024, 60(16): 236-247.

References

[1] TIAN C, XU Y, LI Z, et al. Attention-guided CNN for image denoising[J]. Neural Networks, 2020, 124: 117-129.
[2] BAUER P, THORPE A, BRUNET G. The quiet revolution of numerical weather prediction[J]. Nature, 2015, 525(7567): 47-55.
[3] JAMJOOM M, MAHMOUD A M, ABBAS S, et al. Gaussian mixture with max expectation guide for stacked architecture of denoising autoencoder and DRBM for medical chest scans and disease identification[J]. Electronics, 2022, 12(1): 105.
[4] ZEB M H, ALOBEIDAT F, TUBAISHAT A, et al. Denoising histopathology images for the detection of breast cancer[J]. Neural Computing and Applications, 2023. DOI:10.1007/s00521-023-08771-y.
[5] TIAN C, FEI L, ZHENG W, et al. Deep learning on image denoising: an overview[J]. Neural Networks, 2020, 131: 251-275.
[6] BUADES A, COLL B, MOREL J M. Nonlocal image and movie denoising[J]. International Journal of Computer Vision, 2008, 76: 123-139.
[7] GAI S, BAO Z, ZHANG K. Vector extension of quaternion wavelet transform and its application to colour image denoising[J]. IET Signal Processing, 2019, 13(2): 133-140.
[8] WU J F, LEE X D. An improved WNNM algorithm for image denoising[J]. Journal of Physics: Conference Series, 2019, 1237(2): 022037.
[9] 李潇瑶, 王炼红, 周怡聪, 等. 自适应非局部 3 维全变分彩色图像去噪[J]. 中国图象图形学报, 2022, 27(12): 3450-3460.
LI X Y, WANG L H, ZHOU Y C, et al. Adaptive nonlocal 3D total variation color image denoising[J]. Chinese Journal of Image and Graphics, 2022, 27(12): 3450-3460.
[10] MAO X, SHEN C, YANG Y B. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections[C]//Advances in Neural Information Processing Systems 29, 2016.
[11] 盖杉, 鲍中运. 基于深度学习的高噪声图像去噪算法[J]. 自动化学报, 2020, 46(12): 2672-2680.
GAI S, BAO Z Y. High noise image denoising algorithm based on deep learning[J]. Acta Automatica Sinica, 2020, 46(12): 2672-2680.
[12] ZHANG Q, ZHAO J, TIAN C, et al. A robust deformed convolutional neural network (CNN) for image denoising[J]. CAAI Transactions on Intelligence Technology, 2023, 8(2): 331-342.
[13] FU X, HUANG J, DING X, et al. Clearing the skies: a deep network architecture for single-image rain removal[J]. IEEE Transactions on Image Processing, 2017, 26(6): 2944-2956.
[14] XIAO J, LIU E, ZHAO L, et al. Detail enhancement of image super-resolution based on detail synthesis[J]. Signal Processing: Image Communication, 2017, 50: 21-33.
[15] 吕承侃, 沈飞, 张正涛, 等. 图像异常检测研究现状综述[J]. 自动化学报, 2022, 48(6): 1402-1428.
LV C K, SHEN F, ZHANG Z T, et al. Overview of research status on image anomaly detection[J]. Acta Automatica Sinica, 2022, 48(6): 1402-1428.
[16] SHI Z, METTES P, ZHENG G, et al. Frequency-supervised MR-to-CT image synthesis[C]//Deep Generative Models, and Data Augmentation, Labelling, and Imperfections, First Workshop, DGM4MICCAI 2021, Strasbourg, 2021: 3-13.
[17] RHEE H, JANG Y I, KIM S, et al. LC-FDNet: learned lossless image compression with frequency decomposition network[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 6033-6042.
[18] LI X, JIN X, YU T, et al. Learning omni-frequency region-adaptive representations for real image super-resolution[C]//Proceedings of the 35th AAAI Conference on Artificial Intelligence, 2021: 1975-1983.
[19] ZHANG X, ZENG H, GUO S, et al. Efficient long-range attention network for image super-resolution[C]//Proceedings of the 17th European Conference on Computer Vision. Cham: Springer, 2022: 649-667.
[20] GUO M H, LIU Z N, MU T J, et al. Beyond self-attention: external attention using two linear layers for visual tasks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45(5): 5436-5447.
[21] LI X, WANG W, HU X, et al. Selective kernel networks[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 510-519.
[22] CHEN L, LU X, ZHANG J, et al. HINet: half instance normalization network for image restoration[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 182-192.
[23] ALLEN D M. Mean square error of prediction as a criterion for selecting variables[J]. Technometrics, 1971, 13(3): 469-475.
[24] ABDELHAMED A, LIN S, BROWN M S. A high-quality denoising dataset for smartphone cameras[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, 2018: 1692-1700.
[25] MA K, DUANMU Z, WU Q, et al. Waterloo exploration database: new challenges for image quality assessment models[J]. IEEE Transactions on Image Processing, 2016, 26(2): 1004-1016.
[26] LI H, XIA S, ZHOU B, et al. The growth mechanism of grain boundary carbide in Alloy 690[J]. Materials Characterization, 2013, 81: 1-6.
[27] ZHANG L, WU X, BUADES A, et al. Color demosaicking by local directional interpolation and nonlocal adaptive thresholding[J]. Journal of Electronic Imaging, 2011, 20(2): 023016.
[28] PLOTZ T, ROTH S. Benchmarking denoising algorithms with real photographs[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 1586-1595.
[29] DABOV K, FOI A, KATKOVNIK V, et al. Image denoising by sparse 3-D transform-domain collaborative filtering[J]. IEEE Transactions on Image Processing, 2007, 16(8): 2080-2095.
[30] GU S, ZHANG L, ZUO W, et al. Weighted nuclear norm minimization with application to image denoising[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014: 2862-2869.
[31] ZHANG K, ZUO W, CHEN Y, et al. Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising[J]. IEEE Transactions on Image Processing, 2017, 26(7): 3142-3155.
[32] ZHANG K, ZUO W, ZHANG L. FFDNet: toward a fast and flexible solution for CNN-based image denoising[C]// IEEE Transactions on Image Processing, 2018, 27(9): 4608-4622.
[33] TIAN C, XU Y, ZUO W. Image denoising using deep CNN with batch renormalization[J]. Neural Networks, 2020, 121: 461-473.
[34] YUE Z, YONG H, ZHAO Q, et al. Variational denoising network: toward blind noise modeling and removal[C]// Advances in Neural Information Processing Systems 32, 2019.
[35] QUAN Y, CHEN Y, SHAO Y, et al. Image denoising using complex-valued deep CNN[J]. Pattern Recognition, 2021, 111: 107639.
[36] ZHONG R, ZHANG Q. DRFENet: an improved deep learning neural network via dilated skip convolution for image denoising application[J]. Applied Sciences, 2022, 13(1): 28.
[37] TIAN C, ZHENG M, ZUO W, et al. Multi-stage image denoising with the wavelet transform[J]. Pattern Recognition, 2023, 134: 109050.
[38] BIAN S, HE X, XU Z, et al. Hybrid dilated convolution with attention mechanisms for image denoising[J]. Electronics, 2023, 12(18): 3770.
[39] GUO S, YAN Z, ZHANG K, et al. Toward convolutional blind denoising of real photographs[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 1712-1722.
[40] ANWAR S, BARNES N. Real image denoising with feature attention[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, 2019: 3155-3164.