Survey of Research Methods for Low Light Image Enhancement

doi:10.3778/j.issn.1002-8331.2210-0143

Abstract

Abstract: The purpose of low-light image enhancement is to restore images with complete details in low-light conditions, and it has gradually become a hot spot in computer image processing research. The quality of image imaging is crucial to intelligent security, video surveillance, and other scenarios and has a very broad application prospect in related industries. In order to study low-light image enhancement in depth, firstly, the traditional low-light image enhancement methods are classified and analyzed in detail, and then the image enhancement methods based on deep learning are listed, and the various networks used and the problems solved are detailed and compared the mentioned methods in detail. Then, the data set is analyzed and studied in detail, and some commonly used evaluation indicators are briefly sorted out. Finally, it summarizes the content, points out the difficulties in the current research, and points out the research goals for the future.

Key words: low-light image enhancement, deep learning, Retinex theory, low-light image dataset, image processing

摘要： 低光照图像增强目的是从低光照条件下恢复细节完整的图像，并逐渐成为计算机图像处理研究的热点。图像成像的质量对于智能安防、视频监控等场景至关重要，且在相关行业中有着十分广阔的应用前景。为了深入研究低光照图像增强，对传统低光照图像增强方法进行详细地分类阐述与分析，列举了基于深度学习的图像增强方法，对所用到的各种网络以及所解决的问题进行了详细的梳理，并将所提到的方法进行了细致的对比。又对数据集进行了细致的分析和研究，并对一些常用的评价指标进行了简单梳理。对所述内容做出总结以及指出了当前研究中存在的困难，并指出了未来的研究目标。

关键词: 低光照图像增强, 深度学习, Retinex理论, 低光照图像数据集, 图像处理

PENG Daxin, ZHEN Tong, LI Zhihui. Survey of Research Methods for Low Light Image Enhancement[J]. Computer Engineering and Applications, 2023, 59(18): 14-27.

彭大鑫, 甄彤, 李智慧. 低光照图像增强研究方法综述[J]. 计算机工程与应用, 2023, 59(18): 14-27.

References

[1] ZHANG Z，ZHENG H，HONG R，et al.Deep color consistent network for low-light image enhancement[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2022：1899-1908.
[2] 祖佳贞，周永霞，陈乐.结合注意力的双分支残差低光照图像增强[J].计算机应用，2023，43（4）：1240-1247.
ZU J Z，ZHOU Y X，CHEN L.Dual-branch residual low-light image enhancement combined with attention[J].Journal of Computer Applications，2023，43（4）：1240-1247.
[3] KANDULA P，SUIN M，RAJAGOPALAN A N.Illumination-adaptive unpaired low-light enhance-ment[J].IEEE Transactions on Circuits and Systems for Video Technology，2023，33（8）：3726-3736.
[4] 赵军辉，吴玉峰，胡坤融，等.基于Lab色彩空间和色调映射的彩色图像增强算法[J].计算机科学，2018，45（2）：297-300.
ZHAO J H，WU Y F，HU K R，et al.Color image enhancement algorithm based on lab color space and tone mapping[J].Computer Science，2018，45（2）：297-300.
[5] 曹红燕，刘长明，沈小林，等.基于自适应阈值和局部色调映射的低照度图像处理[J].激光与光电子学进展，2021，58（4）：227-234.
CAO H Y，LIU C M，SHEN X L，et al.Low illumination image processing based on adaptive threshold and local tone mapping[J].Laser & Optoelectronics Progress，2021，58（4）：227-234.
[6] 庞泽邦，芦碧波，谷亚楠，等.基于交叉分解的高动态范围图像色调映射算法[J].激光与光电子学进展，2021，58（14）：296-303.
PANG Z B，LU B B，GU Y N，et al.Crossing decomposition based tone mapping algorithm for high dynamic range image[J].Laser & Optoelectronics Progress，2021，58（14）：296-303.
[7] 朱世松，秦嬴，郑艳梅，等.色度亮度协同滤波的色调映射算法[J].液晶与显示，2022，37（1）：77-85.
ZHU S S，QIN Y，ZHENG Y M，et al.Tone mapping algorithm with chromaticity brightness collaborative filtering[J].Chinese Journal of Liquid Crystals and Displays，2022，37（1）：77-85.
[8] 郭永坤，朱彦陈，刘莉萍，等.空频域图像增强方法研究综述[J].计算机工程与应用，2022，58（11）：23-32.
GUO Y K，ZHU Y C，LIU L P，et al.Research review of space-frequency domain image enhancement methods[J].Computer Engineering and Applications，2022，58（11）：23-32.
[9] 江巨浪，刘国明，朱柱，等.基于快速模糊聚类的动态多直方图均衡化算法[J].电子学报，2022，50（1）：167-176.
JIANG J L，LIU G M，ZHU Z，et al.Dynamic multi-histogram equalization based on fast fuzzy clustering[J].Acta Electronica Sinica，2022，50（1）：167-176.
[10] 王利娟，常霞，任旺.基于加权直方图均衡化彩色图像增强仿真[J].计算机仿真，2021，38（12）：126-131.
WANG L J，CHANG X，REN W.Color image enhancement simulation based on weighted histogram equalization[J].Computer Simulation，2021，38（12）：126-131.
[11] 杨嘉能，李华，田宸玮，等.基于自适应校正的动态直方图均衡算法[J].计算机工程与设计，2021，42（5）：1264-1270.
YANG J N，LI H，TIAN C W，et al.Adaptive correction based dynamic histogram equalization[J].Computer Engineering and Design，2021，42（5）：1264-1270.
[12] 康利娟，陈先桥.基于多级直方图形状分割的图像对比度增强技术[J].计算机应用与软件，2022，39（3）：207-212.
KANG L J，CHEN X Q.Image contrast enhancement technology based on multi-level histogram shape segmentation[J].Computer Applications and Software，2022，39（3）：207-212.
[13] JEBADASS J R，BALASUBRAMANIAM P.Low contrast enhancement technique for color images using interval-valued intuitionistic fuzzy sets with contrast limited adaptive histogram equalization[J].Soft Computing，2022，26（10）：4949-4960.
[14] 侯利霞，聂丰英，万里勇.多尺度自适应Gamma矫正的低照图像增强[J].云南大学学报（自然科学版），2023，45（1）：57-66.
HOU L X，NIE F Y，WAN L Y.Low illumination image enhancement based on multi-scale adaptive gamma correction[J].Journal of Yunnan University（Natural Sciences Edition），2023，45（1）：57-66.
[15] 杨先凤，李小兰，贵红军.改进的自适应伽马变换图像增强算法仿真[J].计算机仿真，2020，37（5）：241-245.
YANG X F，LI X L，GUI H J.Image enhancement algorithm simulation based on improved adaptive Gamma transformation[J].Computer Simulation，2020，37（5）：241-245.
[16] DONG X，WANG G，PANG Y，et al.Fast efficient algorithm for enhancement of low lighting video[C]//IEEE International Conference on Multimedia and Expo，2010：37-53.
[17] 刘峰，王信佳，于波，等.基于暗原色先验的低照度视频增强算法[J].计算机系统应用，2019，28（6）：165-171.
LIU F，WANG X J，YU B，et al.Low lighting video enhancement algorithm based on dark channel prior[J].Computer Systems & Applications，2019，28（6）：165-171.
[18] 王硕，陈金玉.自适应校正透射率的暗通道先验去雾算法[J].计算机工程与应用，2021，57（13）：207-211.
WANG S，CHEN J Y.Dark channel prior defogging algorithm for adaptive correction transmittance[J].Computer Engineering and Applications，2021，57（13）：207-211.
[19] LI D，SHI H，WANG H，et al.Image enhancement method based on dark channel prior[C]//2022 International Conference on Computer Engineering and Artificial Intelligence（ICCEAI），2022：200-204.
[20] LAND E H，MCCANN J J.Lightness and retinex theory[J].Journal of the Optical Society of America，1971，61（1）：1-11.
[21] FUNT B V，CIUREA F，MCCANN J J.Retinex in Matlab[J].Journal of Electronic Imaging，2000，13（1）：112-121.
[22] JOBSON D J，RAHMAN Z，WOODEL G A.Properties and performance of a center/surround retinex[J].IEEE Transactions on Image Processing，1997，6（3）：451-462.
[23] JOBSON D J，RAHMAN Z，WOODELL G A.A multiscale retinex for bridging the gap between color images and the human observation of scenes[J].IEEE Transactions on Image processing，1997，6（7）：965-976.
[24] RAHMAN Z，JOBSON D J，WOODELL G A.Retinex processing for automatic image enhancement[J].Journal of Electronic Imaging，2004，13（1）：100-110.
[25] 常戬，刘鑫姝.空间转换与自适应灰度校正的低照度图像增强[J].计算机工程，2023，49（6）：193-200.
CHANG J，LIU X S.Low illumination image enhancement with spatial transformation and adaptive gray correction[J].Computer Engineering，2023，49（6）：193-200.
[26] 苏康友，王晓刚，柳革命.改进Retinex算法的光照不均匀图像增强研究[J].激光杂志，2022，43（9）：103-108.
SU K Y，WANG X G，LIU G M.Improved Retinex algorithm for image enhancement with uneven illumination[J].Laser Journal，2022，43（9）：103-108.
[27] 李淼，周冬明，刘琰煜，等.结合深度残差神经网络与Retinex理论的低照度图像增强[J].云南大学学报（自然科学版），2021，43（4）：690-699.
LI M，ZHOU D M，LIU Y Y，et al.Low-light image enhancement using deep neural network and retinex theory[J].Journal of Yunnan University（Natural Sciences Edition），2021，43（4）：690-699.
[28] PAN X，LI C，PAN Z，et al.Low-light image enhancement method based on retinex theory by improving illumination map[J].Applied Sciences，2022，12（10）：5257.
[29] YANG J，WANG J，DONG L L，et al.Optimization algorithm for low light image enhancement based on Retinex theory[J].IET Image Processing，2022，17（2）：505-517.
[30] MA Q，WANG Y，ZENG T.Retinex-based variational framework for low-light image enhancement and denoising[J].IEEE Transactions on Multimedia，2022：1-9.
[31] LIN Y H，LU Y C.Low-light enhancement using a plug-and-play retinex model with shrink-age mapping for illumination estimation[J].IEEE Transactions on Image Processing，2022，31：4897-4908.
[32] 王殿伟，邢质斌，韩鹏飞，等.基于模拟多曝光融合的低照度全景图像增强[J].光学精密工程，2021，29（2）：349-362.
WANG D W，XING Z B，HAN P F，et al.Low illumination panoramic image enhancement algorithm based on simulated multi-exposure fusion[J].Optics and Precision Engineering，2021，29（2）：349-362.
[33] RAO Y，LIN W Y，CHEN L.Image-based fusion for video enhancement of night-time surveillance[J].Optical Engineering，2010，49（12）：120501.
[34] 翟海祥，何嘉奇，王正家，等.改进Retinex与多图像融合算法用于低照度图像增强[J].红外技术，2021，43（10）：987-993.
ZHAI H X，HE J Q，WANG Z J，et al.Improved Retinex and multi-image fusion algorithm for low illumination image enhancement[J].Infrared Technology，2021，43（10）：987-993.
[35] UEDA Y，KOGA T，SUETAKE N.Fusion-based backlit image enhancement using multiple stype transformations for convex combination coefficients[C]//2022 IEEE International Conference on Image Processing（ICIP），2022：2971-2975.
[36] ZHANG Y，ZHANG J，GUO X.Kindling the darkness：a practical low-light image enhancer[C]//Proceedings of the 27th ACM International Conference on Multimedia，2019：1632-1640.
[37] ZHANG Y，GUO X，MA J，et al.Beyond brightening low-light images[J].International Journal of Computer Vision，2021，129（4）：1013-1037.
[38] WANG Y，WAN R，YANG W，et al.Low-light image enhancement with normalizing flow[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2022：2604-2612.
[39] JIANG K，WANG Z Y，WANG Z，et al.Degrade is upgrade：learning degradation for low-light image enhancement[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2022：1078-1086.
[40] XU K，CHEN H，TAN X，et al.HFMNet：hierarchical feature mining network for low-light image enhancement[J].IEEE Transactions on Instrumentation and Measurement，2022，71.
[41] LU Y，GUO Y，LIU R W，et al.MTRBNet：multi-branch topology residual block-based network for low-light enhancement[J].IEEE Signal Processing Letters，2022，29：1127-1131.
[42] ZHUANG Y，ZHENG Z，LYU C.DPFNet：a dual-branch dilated network with phase-aware fourier convolution for low-light image enhancement[J].arXiv：2209.07937，2022.
[43] FAN G，FAN B，GAN M，et al.Multiscale low-light image enhancement network with illumination constraint[J].IEEE Transactions on Circuits and Systems for Video Technology，2022，32（11）：7403-7417.
[44] GAO S H，CHENG M M，ZHAO K，et al.Res-2-Net：a new multi-scale backbone architecture[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2021，43（2）：652-662.
[45] WU W，WENG J，ZHANG P，et al.URetinex-Net：retinex-based deep unfolding network for low-light image enhancement[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2022：5901-5910.
[46] LU K，ZHANG L.TBEFN：a two-branch exposure-fusion network for low-light image enhancement[J].IEEE Transactions on Multimedia，2020，23：4093-4105.
[47] LU H，GONG J，LIU Z，et al.FDMLNet：a frequency-division and multiscale learning network for enhancing low-light image[J].Sensors，2022，22（21）：8244.
[48] XU K，CHEN H，XU C，et al.Structure-textureaware network for low-light image enhancement[J].IEEE Transactions on Circuits and Systems for Video Technology，2022，32（8）：4983-4996.
[49] GARG A，PAN X W，DUNG L R.LiCENt：low-light image enhancement using the light channel of HSL[J].IEEE Access，2022，10：33547-33560.
[50] CHEN J，LIAN Q，ZHANG X，et al.Hcsam-Net：multistage network with a hybrid of convolution and self-attention mechanism for low-light image enhancement[J].SSRN，2022：4237486.
[51] YANG Y，HU W，HUANG S，et al.Low-light image enhancement network based on multi-stream information supplement[J].Multidimensional Systems and Signal Processing，2022，33：711-723.
[52] JIN Y，YANG W，TAN R T.Unsupervised night image enhancement：when layer decomposition meets light-effects suppression[J].arXiv：2207.10564，2022.
[53] GUO C，Li C，GUO J，et al.Zero-reference deep curve estimation for low-light image enhancement[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：1780-1789.
[54] LI C，GUO C，LOY C C.Learning to enhance low-light image via zero-reference deep curve estimation[J].arXiv：2103.00860，2021.
[55] JIANG Y，GONG X，LIU D，et al.Enlightengan：deep light enhancement without paired supervision[J].IEEE Transactions on Image Processing，2021，30：2340-2349.
[56] FU Y，HONG Y，CHEN L，et al.LE-GAN：unsupervised low-light image enhancement networkusing attention module and identity invariant loss[J].Knowledge-Based Systems，2022，240：108010.
[57] NI Z，YANG W，WANG H，et al.Cycle-interactive generative adversarial network for robust unsupervised low-light enhancement[C]//Proceedings of the 30th ACM International Conference on Multimedia，2022：1484-1492.
[58] WANG R，JIANG B，YANG C，et al.MAGAN：unsupervised low-light image enhancement guided by mixed-attention[J].Big Data Mining and Analytics，2022，5（2）：110-119.
[59] QIAO J，WANG X，CHEN J，et al.Low-light image enhancement with an anti-attention block based generative adversarial network[J].Electronics，2022，11（10）：1627.
[60] XU W，CHEN X，GUO H，et al.Unsupervised image restoration with quality-task-perception loss[J].IEEE Transactions on Circuits and Systems for Video Technology，2022，32（9）：5736-5747.
[61] JIANG Q，MAO Y，CONG R，et al.Unsupervised decomposition and correction network for low-light image enhancement[J].IEEE Transactionson Intelligent Transportation Systems，2022，23（10）19440-19455.
[62] ZHU J Y，PARK T，ISOLA P，et al.Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：2223-2232.
[63] BHATTACHARYA J，MODI S，GREGORAT L，et al.D2BGAN：a dark to bright image conversion model for quality enhancement and analysis tasks without paired supervision[J].IEEE Access，2022，10：57942-57961.
[64] WEI C，WANG W，YANG W，et al.Deep retinex decomposition for low-light enhancement[J].arXiv：1808.04560，2018.
[65] CAI J，GU S，ZHANG L.Learning a deep single image contrast enhancer from multi-exposure images[J].IEEE Transactions on Image Processing，2018，27（4）：2049-2062.
[66] BYCHKOVSKY V，PARIS S，CHAN E，et al.Learning photographic global tonal adjustment with a database of input/output image pairs[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2011：97-104.
[67] CHEN C，CHEN Q，XU J，et al.Learning to see in the dark[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：3291-3300.
[68] CHEN C，CHEN Q，DO M N，et al.Seeing motion in the dark[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：3185-3194.
[69] LOH Y P，CHAN C S.Getting to know low-light images with the exclusively dark dataset[J].Computer Vision and Image Understanding，2019，178：30-42.
[70] KALANTARI N K，RAMAMOORTHI R.Deep high dynamic range imaging of dynamic scenes[J].ACM Transactions on Graphics，2017，36（4）：1-12.
[71] HOWARD A G，ZHU M，CHEN B，et al.Mobilenets：efficient convolutional neural networks for mobile vision applications[J].arXiv：1704.04861，2017.
[72] LIU J J，HOU Q，CHENG M M，et al.Improving convolutional networks with self-calibrated convolutions[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：10096-10105.
[73] LIU C，ZOPH B，NEUMANN M，et al.Progressive neural architecture search[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：19-34.
[74] LIU C，CHEN L C，SCHROFF F，et al.Auto-deeplab：hierarchical neural architecture search for semantic image segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Visionand Pattern Recognition，2019：82-92.
[75] DOSOVITSKIY A，BEYER L，KOLESNIKOV A，et al.An image is worth 16x16 words：transformers for image recognition at scale[J].arXiv：2010.11929，2020.
[76] CHEN H，WANG Y，GUO T，et al.Pretrained image processing transformer[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：12299-12310.
[77] SHEN N，ZHOU B，XIE J，et al.Low-light image enhancement via transformer-based network[C]//2022 4th International Conference on Image，Video and Signal Processing，2022：41-47.
[78] YANG S，ZHOU D，CAO J，et al.Rethinking low-light enhancement via Transformer-GAN[J].IEEE Signal Processing Letters，2022，29：1082-1086.
[79] KHAN S，NASEER M，HAYAT M，et al.Transformers in vision：a survey[J].ACM Computing Surveys（CSUR），2022，54（10s）：1-41.
[80] 刘文婷，卢新明.基于计算机视觉的Transformer研究进展[J].计算机工程与应用，2022，58（6）：1-16.
LIU W T，LU X M.Research progress of Transformer based on computer vision[J].Computer Engineering and Applications，2022，58（6）：1-16.
[81] FANG Y，ZHU H，ZENG Y，et al.Perceptual quality assessment of smartphone photography[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：3677-3686.
[82] TALEBI H，MILANFAR P.NIMA：neural image assessment[J].IEEE Transactions on Image Processing，2018，27（8）：3998-4011.
[83] HYUN K T，MU L K，SCHOLKOPF B，et al.Online video deblurring via dynamic temporal blending network[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：4038-4047.
[84] EHRET T，DAVY A，MOREL J M，et al.Model-blind video denoising via frame-to-frame training[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：11369-11378.
[85] CHAN K C K，WANG X，YU K，et al.BasicVSR：the search for essential components in video super-resolution and beyond[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：4947-4956.
[86] WANG X，YU K，DONG C，et al.Recovering realistic texture in image super-resolution by deep spatial feature transform[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：606-615.
[87] CHAN K C K，WANG X，XU X，et al.Glean：generative latent bank for large-factor image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：14245-14254.
[88] LI X，CHEN C，ZHOU S，et al.Blind face restoration via deep multi-scale component dictionaries[C]//European Conference on Computer Vision.Cham：Springer，2020：399-415.
[89] LIANG D，LI L，WEI M，et al.Semantically contrastive learning for low-light image enhancement[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2022：1555-1563.