Low-Light Image Enhancement Using Brightness and Signal-to-Noise Ratio Guided Transformer

doi:10.3778/j.issn.1002-8331.2312-0361

Abstract

Abstract: The enhanced images generated by some existing low-light image enhancement methods have problems such as uneven brightness, poor denoising effect, and lack of detailed information. To solve these issues, this paper proposes a low-light image enhancement network based on brightness and signal-to-noise ratio guided Transformer. This network has the following advantages: a brightness and signal-to-noise ratio generation sub-network is designed to extract global illumination information and locate dark areas with missing information. The Transformer is guided by brightness and signal-to-noise ratio feature maps to extract long-distance features only from dark areas with missing information to reduce the calculation complexity. Meanwhile, the subsequent feature fusion module is guided to enrich the details of dark areas with the help of bright area information and achieve information sharing. A cross-fusion attention module is designed and introduced between the encoder and decoder, thereby the ability of network is improved to retain image details. Experimental results on four public datasets show that BSGFormer can achieve better enhancement effects than the popular methods in both subjective vision and objective evaluation.

Key words: low-light image, image enhancement, Transformer, residual convolution

摘要： 一些低照度图像增强方法生成的增强图像存在亮度不均、去噪效果较差和缺少细节的问题。为了解决上述问题，提出了基于亮度和信噪比引导Transformer的低照度图像增强网络BSGFormer。该网络具有三个优势：设计了亮度信噪比生成子网络，旨在提取全局光照信息和定位信息量缺失的暗区域；通过亮度和信噪比特征引导Transformer，仅对信息量缺失的暗区提取长距离特征以减少计算量，同时引导后续特征融合模块，借助亮区信息来丰富暗区细节并实现信息共享；设计了交叉融合注意力模块并将其引入到编解码器之间，改善网络对图像细节信息的保留能力。在四个公开数据集上进行实验表明，与主流方法相比，BSGFormer在主观视觉和客观评价两方面均得到了更好的增强效果。

关键词: 低照度图像, 图像增强, Transformer, 卷积残差

DU Xiaogang, LU Wenjie, LEI Tao, WANG Yingbo. Low-Light Image Enhancement Using Brightness and Signal-to-Noise Ratio Guided Transformer[J]. Computer Engineering and Applications, 2025, 61(6): 263-272.

杜晓刚, 路文杰, 雷涛, 王营博. 亮度信噪比引导Transformer的低照度图像增强[J]. 计算机工程与应用, 2025, 61(6): 263-272.

References

[1] TAN S F, ISA N A M. Exposure based multi-histogram equalization contrast enhancement for non?uniform illumination images[J]. IEEE Access, 2019, 7: 70842-70861.
[2] PARIHAR A S, VERMA O P. Contrast enhancement using entropy?based dynamic sub?histogram equalisation[J]. IET Image Processing, 2016, 10(11): 799-808.
[3] LI M D, LIU J Y, YANG W H, et al. Structure-revealing low-light image enhancement via robust retinex model[J]. IEEE Transactions on Image Processing, 2018, 27(6): 2828-2841.
[4] YANG J Y, XU Y W, YUE H J, et al. Low?light image enhancement based on Retinex decomposition and adaptive gamma correction[J]. IET Image Processing, 2021, 15(5): 1189-1202.
[5] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems, 2017: 5998-6008.
[6] LORE K G, AKINTAYO A, SARKAR S. LLNet: a deep autoencoder approach to natural low-light image enhancement[J]. Pattern Recognition, 2017, 61: 650-662.
[7] LV F, LU F, WU J, et al. MBLLEN: low-light image/video enhancement using CNNs[C]//Proceedings of the British Machine Vision Conference, 2018: 220-228.
[8] ZHANG Y H, ZHANG J W, GUO X J. Kindling the darkness: a practical low-light image enhancer[C]//Proceedings of the 27th ACM International Conference on Multimedia, 2019: 1632-1640.
[9] JIANG Y F, GONG X Y, LIU D, et al. EnlightenGAN: deep light enhancement without paired supervision[J]. IEEE Transactions on Image Processing, 2021, 30: 2340-2349.
[10] GUO C L, LI C Y, GUO J C, et al. Zero-reference deep curve estimation for low-light image enhancement[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020: 1780-1789.
[11] ZHENG S, GUPTA G. Semantic-guided zero-shot learning for low-light image/video enhancement[C]//Proceedings of the IEEE Winter Conference on Applications of Computer Vision, 2022: 581-590.
[12] WANG Y F, WAN R J, YANG W H, et al. Low-light image enhancement with normalizing flow[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2022: 2604-2612.
[13] WU W H, WENG J, ZHANG P P, et al. Uretinex-Net: retinex-based deep unfolding network for low-light image enhancement[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2022: 5901-5910.
[14] LIU Z, LIN Y T, CAO Y, et al. Swin Transformer: hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE International Conference on Computer Vision, 2021: 10012-10022.
[15] WANG Z D, CUN X D, BAO J M, et al. Uformer: a general u-shaped transformer for image restoration[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2022: 17683-17693.
[16] ZAMIR S W, ARORA A, KHAN S, et al. Restormer: efficient transformer for high-resolution image restoration[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2022: 5728-5739.
[17] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: Transformers for image recognition at scale[J]. arXiv:2010.11929, 2020.
[18] CUI Z T, LI K C, GU L, et al. You only need 90K parameters to adapt light: a light weight transformer for image enhancement and exposure correction[C]//Proceedings of the British Machine Vision Conference, 2022.
[19] HAN Q H, FAN Z J, DAI Q, et al. On the connection between local attention and dynamic depth-wise convolution[J]. arXiv:2106.04263, 2021.
[20] LI K C, WANG Y L, ZHANG J H, et al. Uniformer: unifying convolution and self-attention for visual recognition[J]. arXiv:2201.09450, 2022.
[21] WANG T, ZHANG K H, SHEN T R, et al. Ultra-high-definition low?light image enhancement: a benchmark and transformer?based method[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2023: 2654-2662.
[22] CAI Y H, BIAN H, LIN J, et al. Retinexformer: one-stage retinex-based transformer for low-light image enhancement[J]. arXiv:2303.06705, 2023.
[23] BUADES A, COLL B, MOREL J M. A non-local algorithm for image denoising[C]//Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005: 60-65.
[24] DABOV K, FOI A, KATKOVNIK V, et al. Image denoising with block-matching and 3D filtering[C]//Proceedings of the Image Processing: Algorithms and Systems, Neural Networks, and Machine Learning, 2006: 354-365.
[25] AZAD R, ARIMOND R, AGHDAM E K, et al. DAE-Former: dual attention-guided efficient transformer for medical image segmentation[C]//Proceedings of the International Workshop on Predictive Intelligence In Medicine, 2023: 83-95.
[26] SHEN Z S, ZHANG M Y, ZHAO H Y, et al. Efficient attention: attention with linear complexities[C]//Proceedings of the IEEE Winter Conference on Applications of Computer Vision, 2021: 3531-3539.
[27] LAI W S, HUANG J B, AHUJA N, et al. Fast and accurate image super-resolution with deep laplacian pyramid networks[C]//Proceedings of the British Machine Vision Conference, 2018: 2599-2613.
[28] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. arXiv:1409. 1556, 2014.
[29] YANG W H, WANG W J, HUANG H F, et al. Sparse gradient regularized deep retinex network for robust low-light image enhancement[J]. IEEE Transactions on Image Processing, 2021, 30: 2072-2086.
[30] WEI C, WANG W J, YANG W H, et al. Deep retinex decomposition for low?light enhancement[J]. arXiv:1808. 04560, 2018.
[31] CHEN C, CHEN Q F, XU J, et al. Learning to see in the dark[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 3291-3300.
[32] WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.
[33] ZHANG R, ISOLA P, EFROS A A, et al. The unreasonable effectiveness of deep features as a perceptual metric[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 586-595.
[34] GUO X J, LI Y, LING H B. LIME: low-light image enhancement via illumination map estimation[J]. IEEE Transactions on Image Processing, 2016, 26(2): 982-993.
[35] MA L, MA T Y, LIU R S, et al. Toward fast, flexible, and robust low-light image enhancement[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 5637-5646.
[36] LIU R S, MA L, ZHANG J A, et al. Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 10561-10570.
[37] XU X G, WANG R X, FU C W, et al. SNR-aware low-light image enhancement[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2022: 17714-17724.