结合多分支结构和U-net的低照度图像增强

doi:10.3778/j.issn.1002-8331.2011-0097

摘要/Abstract

摘要： 随着夜景拍摄技术的提高，低照度图像增强成为计算机视觉领域一个新的热点。但是由于光照不足、逆光、聚焦失败等因素的影响会导致光照强度不足，导致图像亮度和对比度过低。为了更好地处理低光照图像，提出了一种基于多分支结构和U-net结合的低照度图像增强算法。利用深度残差网络将图片不同层次的特征提取出来进行交叉合并。将得到的图像通过不同深度和结构的U-net进行增强。将U-net增强后的图像进行融合，最终得到了增强后的低照度图像。通过大量的实验表明，运用深度残差网络和U-net，可以更好地进行特征提取，低照度图像增强的效果也更好，很大程度上优于现有的技术。提出的方法不仅在视觉上提高了亮度和对比度，色彩更真实，更加符合人眼视觉系统特性，而且PSNR、SSIM等七项客观图像质量指标在几种算法中都是最优的。

关键词: 低照度图像, 多分支, U-net网络, 神经网络, 图像增强

Abstract: With the improvement of night scene shooting technology, low-light image enhancement has become a new hot spot in the field of computer vision. However, due to the lack of light, backlighting, focusing failure and other factors will lead to insufficient light intensity, resulting in excessively low brightness and contrast of the image. To better process low-light images, a low-light image enhancement algorithm based on multi-branch structure and U-net is proposed. The image features with different levels which are extracted by deep residual network are used for cross-merging. The obtained images are enhanced by U-net with different depths and structures, and then the images enhanced by U-net are fused. The enhanced low illuminance image is obtained. A mass of experiments show that the use of the deep residual network and U-net can better extract features, and the effect of low-intensity image enhancement is better, which is largely superior to the existing technology. The proposed methods not only improve the brightness and contrast visually, the colors are more realistic, and more in line with the characteristics of the human visual system, but also PSNR, SSIM and other seven objective image quality indexes are optimal in several algorithms.

Key words: low-light image, multi-branch, U-net network, neural network, image enhancement

卫依雪, 周冬明, 王长城, 李淼. 结合多分支结构和U-net的低照度图像增强[J]. 计算机工程与应用, 2022, 58(12): 199-208.

WEI Yixue, ZHOU Dongming, WANG Changcheng, LI Miao. Low-Light Image Enhancement Using Multi-Branch Structure and U-net[J]. Computer Engineering and Applications, 2022, 58(12): 199-208.

参考文献

[1] CAI J，GU S，ZHANG L.Learning a deep single image contrast enhancer from multi-exposure images[J].IEEE Transactions on Image Processing，2018，27（4）：2049-2062.
[2] 赵斌红，马帅.自适应的低照度图像增强变分模型[J].计算机工程与应用，2019，55（9）：144-150.
ZHAO B H，MA S.Adaptive variational model for low light image enhancement[J].Computer Engineering and Applications，2019，55（9）：144-150.
[3] KONG N S P，IBRAHIM H.Color image enhancement using brightness preserving dynamic histogram equalization[J].IEEE Transactions on Consumer Electronics，2008，54（4）：1962-1968.
[4] IBRAHIM H，KONG N S P.Brightness preserving dynamic histogram equalization for Image contrast enhancement[J].IEEE Transactions on Consumer Electronics，2007，53（4）：1752-1758.
[5] 白海平，杨燕.基于暗通道先验的比值重估透射率去雾算法[J].计算机工程与应用，2016，52（13）：212-217.
BAI H P，YANG Y.Ratio revaluating transmission algorithm of image dehazing based on dark channel prior[J].Computer Engineering and Applications，2016，52（13）：212-217.
[6] MCCANN J J.Lightness and retinex theory[J].Journal of the Optical Society of America，1970，61（1）：1-11.
[7] 陈薇，赵方田，陈侃，等.改进单尺度Retinex的光照人脸识别[J].计算机工程与应用，2013，49（12）：151-154.
CHEN W，ZHAO F T，CHEN K，et al.Face recognition in change of illumination based on improved Retinex algorithm[J].Computer Engineering and Applications，2013，49（12）：151-154.
[8] JOBSON D J，RAHMAN Z，WOODELL G A.A multiscale retinex for bridging the gap between color images and the human observation of scenes[J].IEEE Transactions on Image Processing，1997，6（7）：965-976.
[9] RAHMAN Z，JOBSON D J，WOODELL G A.Multi-scale retinex for color image enhancement[C]//Proceedings of 3rd IEEE International Conference on Image Processing，Lausanne，Switzerland，1996：1003-1006.
[10] FU X，ZENG D，HUANG Y，et al.A weighted variational model for simultaneous reflectance and illumination estimation[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），Seattle，WA，2016：2782-2790.
[11] GUO X，LI Y，LING H.LIME：low-light image enhancement via illumination map estimation[J].IEEE Transactions on Image Processing，2017，26（2）：982-993.
[12] DONG X，WANG G，PANG Y，et al.Fast efficient algorithm for enhancement of low lighting video[C]//2011 IEEE International Conference on Multimedia and Expo（ICME），2011.
[13] YING Z，LI G，GAO W.A bio-inspired multi-exposure fusion framework for low-light image enhancement[J].arXiv：1711.
00591，2017.
[14] WANG S，ZHENG J，HU H M，et al.Naturalness preserved enhancement algorithm for non-uniform illumination images[J].IEEE Transactions on Image Processing，2013，22（9）：3538-3548.
[15] 李益红，周晓谊.一种多分辨多尺度的Retinex彩色图像增强算法[J].计算机工程与应用，2017，53（16）：193-198.
LI Y H，ZHOU X Y.Multi-resolution and multi-scale retinex for color image enhancement[J].Computer Engineering and Applications，2017，53（16）：193-198.
[16] LORE K G，AKINTAYO A，SARKAR S.LLNet：a deep autoencoder approach to natural low-light image enhancement[J].arXiv：1511.03995，2015.
[17] TAO L，ZHU C，XIANG G，et al.LLCNN：a convolutional neural network for low-light image enhancement[C]//2017 IEEE Visual Communications and Image Processing（VCIP），St.Petersburg，FL，2017.
[18] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Computer Vision and Pattern Recognition（CVPR），Seattle，WA，2016：770-778.
[19] RONNEBERGER O，FISCHER P，BROX T.U-Net：convolutional networks for biomedical image segmentation[C]//18th International Conference on Medical Image Computing and Computer-Assisted Intervention（MICCAI），Munich，Germany，2015：234-241.
[20] LV F，LU F，WU J，et al.MBLLEN：low-light image/video enhancement using CNNs[C]//29th British Machine Vision Conference 2018（BMVC 2018），2018.
[21] ZHANG Y，ZHANG J，GUO X.Kindling the darkness：a practical low-light image enhancer[C]//Proceedings of the 27th ACM International Conference on Multimedia（MM’19），Nice，France，2019：1632-1640.
[22] LV F，LI Y，LU F.Attention guided low-light image enhancement with a large-scale low-light simulation dataset[J].arXiv：1908.00682，2019.
[23] REN W Q，LIU S F，MA L，et al.Low-light image enhancement via a deep hybrid network[J].IEEE Transactions on Image Processing，2019，28（9）：4364-4375.
[24] CHEN C，CHEN Q，XU J.Learning to see in the dark[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition，2018：3291-3300.
[25] JIANG Y，GONG X，LIU D，et al.EnlightenGAN：deep light enhancement without paired supervision[J].arXiv：1906.06972，2019.
[26] HUYNH-THU Q，GHANBARI M.Scope of validity of PSNR in image/video quality assessment[J].Electronics Letters，2008，44（13）：800-801.
[27] SHEIKH H R，BOVIK A C.Image information and visual quality[J].IEEE Transactions on Image Processing，2006，15（2）：430-444.
[28] SHEIKH H R.An information fidelity criterion for image quality assessment using natural scene statistics[J].IEEE Transactions on Image Processing，2006，14（12）：2117-2128.
[29] WANG Z，BOVIK A C，SHEIKH H R.Image quality assessment：from error visibility to structural similarity[J].IEEE Transactions on Image Processing，2004，13（4）：600-612.
[30] MITTAL A，SOUNDARARAJAN R，BOVIK A C.Making a “completely blind” image quality analyzer[J].IEEE Signal Processing Letters，2013，20（3）：209-212.