Attention Exposure Fusion Network for Low-Illumination Image Enhancement

doi:10.3778/j.issn.1002-8331.2206-0132

Abstract

Abstract: Low-illumination image enhancement is used to improve the brightness of low-illumination images and fully exploit the hidden information in the images. Recently, low-light image enhancement methods based on deep learning have achieved significant advantages over traditional methods. In order to solve the problem that existing methods may lead to overexposure of bright details or loss of dark details in images, and cannot take into account both dark and bright details in images. A multi-scale attention exposure fusion network is proposed. First, a set of images with different exposure magnifications are generated from a single image. Then, each image is passed through the proposed multiscale attention block to improve the low-illumination image enhancement performance. In particular, the proposed residual context block models the global context of the input features to obtain rich global features. The selective kernel block reselects and calibrates multiple features to enhance useful information while suppressing useless information to improve the processing power of the network. Meanwhile, the fusion block constructs an accurately exposed, detail-rich image by fusing well-exposed regions in images with different exposure magnifications. The PSNR and SSIM performance are improved by 0.72 dB and 0.08 respectively compared to the SID method when compared with the mainstream methods in recent years on the SID dataset. Experimental results show that the improved method has better detail reproduction and color reproduction than the existing methods.

Key words: attention mechanisms, image enhancement, low-illumination images, exposure fusion

摘要： 低照度图像增强用于提高低照度图像的亮度并充分挖掘图像中的隐藏信息。最近，基于深度学习的低照度图像增强方法相对于传统方法取得了显著的优势。为了解决现有方法可能会导致图像亮部细节过曝或暗部细节丢失、无法兼顾图像暗部和亮部细节的问题。提出一种多尺度注意力曝光融合网络。从单个图像中生成一组具有不同曝光倍率的图像。每张图像通过提出的多尺度注意力模块提高低照度图像增强性能。提出的残差上下文模块对输入特征进行全局上下文建模，获取丰富的全局特征。选择性内核模块利用注意力机制结合具有不同感受野的特征，同时保留了它们独特的互补特征。融合模块通过融合不同曝光倍率图像中曝光良好的区域来构建一张曝光准确、细节丰富的图像。在SID数据集上与近年来主流方法进行了对比，PSNR和SSIM性能相比SID方法分别提升了0.72?dB和0.08。实验结果表明，改进的方法比现有方法拥有更好的细节还原和色彩还原能力。

关键词: 注意力机制, 图像增强, 低照度图像, 曝光融合

BAO Yifeng, YANG Degang. Attention Exposure Fusion Network for Low-Illumination Image Enhancement[J]. Computer Engineering and Applications, 2023, 59(20): 237-244.

包易峰, 杨德刚. 面向低照度图像增强的注意力曝光融合网络[J]. 计算机工程与应用, 2023, 59(20): 237-244.

References

[1] GOW R D，RENSHAW D，FINDLATER K，et al.A comprehensive tool for modeling cmos image-sensor-noise performance[J].IEEE Transactions on Electron Devices，2007，54（6）：1321-1329.
[2] CAO Y，XU J R，L S，et al.Global context networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2023，45（6）：6881-6895.
[3] LI X，WANG W H，HU X L，et al.Selective kernel networks[C]//Conference on Computer Vision and Pattern Recognition（CVPR），2019：510-519.
[4] LI S，YIN H，FANG L.Group-sparse representation with dictionary learning for medical image denoising and fusion[J].IEEE Transactions on Biomedical Engineering，2012，59（12）：3450-3459.
[5] SONI V，BHANDARI A K，KUMAR A，et al.Improved sub-band adaptive thresholding function for denoising of satellite image based on evolutionary algorithms[J].IET Signal Processing，2013，7（8）：720-730.
[6] KWON Y，KIM K I，TOMPKIN J，et al.Efficient learning of image super-resolution and compression artifact removal with semi-local Gaussian processes[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，37（9）：1792-1805.
[7] REN S，HE K，GIRSHICK R，et al.FasterR-CNN：towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，39（6）：1137-1149.
[8] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Computer Vision and Pattern Recognition，Las Vegas，NV，USA，2016.
[9] PORTILLA J，STRELA V，WAINWRIGHT M J，et al.Image denoising using scale mixtures of Gaussians in the wavelet domain[J].IEEE Transactions on Image Processing，2003，12（11）：1338-1351.
[10] ELAD M，AHARON M.Image denoising via sparse and redundant representations over learned dictionaries[J].IEEE Transactions on Image Processing，2006，15（12）：3736-3745.
[11] DABOV K，FOI A，KATKOVNIK V，et al.Image denoising by sparse 3D transform-domain collaborative filtering[J].IEEE Transactions on Image Processing，2007，16（8）：2080-2095.
[12] CHEN Y，POCK T.Trainable nonlinear reaction diffusion：a flexible framework for fast and effective image restoration[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，39（6）：1256-1272.
[13] BURGER H C，SCHULER C J，HARMELING S.Image denoising：can plain neural networks compete with BM3D?[C]//IEEE Conference on Computer Vision and Pattern Recognition，2012：2392-2399.
[14] LORE G K，AKINTAYO A，SARKAR S.LLNet：a deep autoencoder approach to natural low-light image enhancement[J].Pattern Recognition，2017，61（1）：650-662.
[15] JAIN V，SEUNG H S.Natural image denoising with convolutional networks[C]//NIPS’08：Proceedings of the 21st International Conference on Neural Information Processing Systems，2008：769-776.
[16] LONG，SHELHAMER J，DARRELL E.Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2015：3431-3440.
[17] LEE C，KIM C.Contrast enhancement based on layered difference representation of 2D histogram[J].IEEE Transactions on Image Processing，2013，22（12）：5372-5384.
[18] LAND E H.The retinex theory of color vision[J].Scientific American，1986，237（6）：108-129.
[19] CHENG H D，SHI X J，et al.A simple and effective histogram equalization approach to image enhancement[J].Digital Signal Processing，2004，14（2）：158-170.
[20] YING Z Q，LI G，GAO W，et al.A bio-inspired multi-exposure fusion framework for low-light image enhancement[J].arXiv：1711.00591，2017.
[21] TAO L，ZHU C，XIANG G Q，et al.LLCNN：a convolutional neural network for low-light image enhancement[C]//IEEE Visual Communications and Image Processing（VCIP），2017：1-4.
[22] WEI C，WANG W J，YANG W H，et al.Deep retinex decomposition for low-light enhancement[J].arXiv：1808.
04560，2018.
[23] CHEN Q F，XU J，KOLTUN V，et al.Fast image processing with fully-convolutional networks[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：2516-2525.
[24] CHEN C，CHEN Q F，XU J，et al.Learning to see in the dark[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：3291-3300.
[25] RONNEBERGER O，FISCHER P，BROX T.U-Net：convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention，2015：234-241.
[26] SHI W Z，CABALLERO J，HUSZáR F，et al.Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2016：1874-1883.
[27] AITTALA M，DURAND F.Burst image deblurring using permutation invariant convolutional neural networks[C]//Proceedings of the European Conference on Computer Vision，2018：731-747.
[28] KINGMA D，BA J.Adam：a method for stochastic optimization[J].arXiv：1412.6980，2014.
[29] ZHAO H，GALLO O，FROSIO I，et al.Loss functions for image restoration with neural networks[J].IEEE Transactions on Computational Imaging，2017，3（1）：47-57.
[30] GUO X J，LI Y，LING H.LIME：low-light image enhancement via illumination map estimation[J].IEEE Transactions on Image Processing，2017，26（2）：982-993.
[31] GUO C，LI C Y，GUO J C H.Zero-reference deep curve estimation for low-light image enhancement[C]//Computer Vision and Pattern Recognition（CVPR），2020：1780-1789.
[32] LAMBA M，MITRA K.Restoring extremely dark images in real time[C]//Computer Vision and Pattern Recognition（CVPR），2021：3486-3496.
[33] GU S H，LI Y W，GOOL V，et al.Self-guided network for fast image denoization[C]//IEEE/CVF International Conference on Computer Vision（ICCV），2019：2511-2520.
[34] XHU K，YANG X，YIN B C.et al.Learning to restore low-light images via decomposition-and-enhancement[C]//Computer Vision and Pattern Recognition（CVPR），2020：2278-2287.
[35] HUYNH-THU Q，GHANBARI M.Scope of validity of PSNR in image/video quality assessment[J].Electronics Letters，2008，44（13）：800-801.
[36] WANG Z，BOVIK A C，SHEIKH H R.Image quality assessment：from error visibility to structural similarity[J].IEEE Transactions on Image Processing，2004，13（4）：600-612.
[37] MITTAL A，SOUNDARARAJAN R，BOVIK A C，et al.Making a “completely blind” image quality analyzer[J].IEEE Signal Processing Letters，2013，20（3）：209-212.