Low Illumination Traffic Image Enhancement Model Based on Multi-Scale Dense Blocks

doi:10.3778/j.issn.1002-8331.2204-0527

Abstract

Abstract: Vehicle detection and tracking is an important content in the field of intelligent transportation. The complex illumination intensity and changeable traffic scene make the details of expressway images blurred, the contrast is low, and the image information extraction is highly disturbing. A generation countermeasure network based on attention mechanism and multi-scale residual dense blocks is proposed for low illumination traffic image enhancement. Different illumination images are synthesized by gamma correction, camera response function and manual adjustment as data sets, covering a wider range of brightness curves and simulating the real night scene. The attention mechanism is introduced to represent the correlation between different channels and high-frequency information. At the same time, maximum pooling and average pooling are used to capture the dependency between texture information and background information to enhance the overall integrity of the image. It builds a multi-scale fusion residual dense connection network. Deep extraction of complex image features uses parallel branches to fuse different ranks and gradations of information to improve the overall perception of details of the network and retain the consistency of picture information. Bilinear convolution structure is used to replace deconvolution layer to eliminate artifacts. The experimental results show that compared with the mainstream methods, the enhancement effect evaluation indexes PSNR and SSIM of the network in this paper are improved by 26.37% and 14.14% respectively. The image enhances the detailed texture and improves the natural visual effect of the image, which provides technical support for the visual task in the field of transportation.

Key words: low illumination image enhancement, traffic images, attention mechanism, residual dense blocks, multi-scale fusion, generative adversarial network

摘要： 车辆检测与跟踪是智能交通领域的重要内容，复杂的光照强度与多变的交通场景使高速公路拍摄图像细节模糊、对比度低，图片信息提取干扰性强。提出一种基于注意力机制融合多尺度残差稠密块的生成对抗网络用于低照度交通图像增强。通过伽马校正、相机响应函数和手工调节方法合成不同照度图像作为数据集，涵盖更广泛亮度曲线，模拟真实夜晚场景；引入注意力机制通过表征不同通道与高频信息间关联性，同时采用最大池化和平均池化捕获纹理信息和背景信息间依赖关系增强图像整体完整性；搭建多尺度融合的残差稠密连接网络，深度提取图像复杂特征利用并行支路融合不同级别和层次的信息，提升网络对细节的整体感知力，保留图片信息一致性；采用双线性加卷积结构代替反卷积层消除伪影现象。实验结果表明与主流方法相比，该网络的增强效果评价指标PSNR和SSIM分别提升26.37%、14.14%，图片增强使细节纹理清晰、图像自然视觉效果提高，为交通领域的视觉任务提供技术支持。

关键词: 低照度图像增强, 交通图像, 注意力机制, 残差稠密块, 多尺度融合, 生成对抗网络

WANG Weihao, WANG Xiali, WU Lizhan, ZHANG Qian, LI Chao. Low Illumination Traffic Image Enhancement Model Based on Multi-Scale Dense Blocks[J]. Computer Engineering and Applications, 2023, 59(17): 223-231.

王炜昊, 王夏黎, 武历展, 张倩, 李超. 融合多尺度密集块的低照度交通图像增强模型[J]. 计算机工程与应用, 2023, 59(17): 223-231.

References

[1] BHANDAR A K.A logarithmic law based histogram modification scheme for naturalness image contrast enhancement[J].Journal of Ambient Intelligence and Humanized Computing，2020，11（4）：1605-1627.
[2] PIZER S M，AMBURN E P，AUSTIN J D，et al.Adaptive histogram equalization and its variations[J].Computer Vision，Graphics，and Image Processing，1987，39（3）：355-368.
[3] LAND E H.The retinex theory of color vision[J].Scientific American，1977，237（6）：108-129.
[4] RAHMAN Z，JOBSON D J，WOODELL G A.Multiscale retinex for color rendition and dynamic range compression[C]//Applications of Digital Image Processing XIX，1996：183-191.
[5] ABDULLAH-AL-WADUD M，KABIR M H，DEWAN M A A，et al.A dynamic histogram equalization for image contrast enhancement[J].IEEE Transactions on Consumer Electronics，2007，53（2）：593-600.
[6] IBRAHIM H，KONG N S P.Brightness preserving dynamic histogram equalization for image contrast enhancement[J].IEEE Transactions on Consumer Electronics，2007，53（4）：1752-1758.
[7] 郑凤仙，王夏黎，何丹丹，等.单幅图像去雾算法研究综述[J].计算机工程与应用，2022，58（3）：1-14.
ZHENG F X，WANG X L，HE D D，et al.Survey of single image defogging algorithm[J].Computer Engineering and Applications，2022，58（3）：1-14.
[8] JOBSON D J，RAHMAN Z，WOODELL G A.Properties and performance of a center/surround retinex[J].IEEE Transactions on Image Processing，1997，6（3）：451-462.
[9] JOBSON D J，RAHMAN Z，WOODELL G A.A multiscale retinex for bridging the gap between color images and the human observation of scenes[J].IEEE Transactions on Image Processing，1997，6（7）：965-976.
[10] LANG H，CWC C，ZWA B.Automatic depression recognition using CNN with attention mechanism from videos[J].Neurocomputing，2021，422：165-175.
[11] HUANG H，TAO H，WANG H.A convolutional neural network based method for low-illumination image enhancement[C]//Proceedings of the 2nd International Conference on Artificial Intelligence and Pattern Recognition，2019：72-77.
[12] FENG J，FENG X，CHEN J，et al.Generative adversarial networks based on collaborative learning and attention mechanism for hyperspectral image classification[J].Remote Sensing，2020，12（7）：1149.
[13] 刘佳敏，何宁，尹晓杰.基于Retinex-UNet算法的低照度图像增强[J].计算机工程与应用，2020，56（22）：211-216.
LIU J M，HE N，YIN X J.Low illumination image enhan-cement based on Retinex-UNet algorithm[J].Computer Engineering and Applications，2020，56（22）：211-216.
[14] 李天宇，吴浩，毛艳玲，等.一种改进CycleGAN的低照度图像增强网络[J].无线电工程，2022，52（5）：799-806.
LI T Y，WU H，MAO Y L，et al.A low-light image enhancement network to improve CycleGAN[J].Radio Engineering，2022，52（5）：799-806.
[15] 王照乾，孔韦韦，滕金保，等.DenseNet生成对抗网络低照度图像增强方法[J].计算机工程与应用，2022，58（8）：214-220.
WANG Z Q，KONG W W，TENG J B，et al.Low illumination image enhancement method based on DenseNet GAN[J].Computer Engineering and Applications，2022，58（8）：214-220.
[16] 杨微，张志威，成海秀.PNet：融合注意力机制的多级低照度图像增强网络[J].计算机应用研究，2022，39（5）：1579-1585.
YANG W，ZHANG Z W，CHENG H X.PNET：multi-level low illumination image enhancement network based on attention mechanism[J].Application Research of Computers，2022，39（5）：1579-1585.
[17] 王蓉，杨燕.基于转换域与自适应伽马校正的去雾算法[J].计算机工程与科学，2022，44（9）：1630-1637.
WANG R，YANG Y.A dehazing algorithm based on transform domain and adaptive gamma correction[J].Computer Engineering & Science，2022，44（9）：1630-1637.
[18] 巫玲，陈念年，廖小华.区域自适应多尺度强光融合的红外图像增强[J].红外技术，2020，42（11）：1072-1076.
WU L，CHEN N N，LIAO X H.Infrared image enhancement based on regional adaptive multiscale instense light fusion[J].Infrared Technology，2020，42（11）：1072-1076.
[19] 桑亮，高爽，尹增山.基于生成对抗网络的运动模糊图像复原[J].计算机工程与应用，2019，55（6）：173-177.
SANG L，GAO S，YIN Z S.Motion deblurring based on generative adversarial network[J].Computer Engineering and Applications，2019，55（6）：173-177
[20] GOODFELLOW I，POUGET-ABADIE J，MIRZA M，et al.Generative adversarial nets[C]//Advances in Neural Information Processing Systems，2014.
[21] HU J，SHEN L，SUN G.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：7132-7141.
[22] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[23] HUANG G，LIU Z，VANDER M L，et al.Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：4700-4708.
[24] WANG S，ZHENG J，HU H M，et al.Naturalness preserved enhancement algorithm for non-uniform illumination images[J].IEEE Transactions on Image Processing，2013，22（9）：3538-3548.
[25] WEI C，WANG W，YANG W，et al.Deep retinex decomposition for low-light enhancement[J].arXiv：1808.04560，2018.
[26] GUO X，LI Y，LING H.LIME：low-light image enhancement via illumination map estimation[J].IEEE Transactions on Image Processing，2016，26（2）：982-993.
[27] GEIGER A，LENZ P，STILLER C，et al.Vision meets robotics：the kitti dataset[J].The International Journal of Robotics Research，2013，32（11）：1231-1237.
[28] WEN L，DU D，CAI Z，et al.UA-DETRAC：a new benchmark and protocol for multi-object detection and tracking[J].Computer Vision and Image Understanding，2020，193：102907.
[29] LI B，REN W，FU D，et al.Benchmarking single-image dehazing and beyond[J].IEEE Transactions on Image Processing，2018，28（1）：492-505.
[30] FU X，ZENG D，HUANG Y，et al.A weighted variational model for simultaneous reflectance and illumination estimation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：2782-2790.
[31] MANNOS J，SAKRISON D.The effects of a visual fidelity criterion of the encoding of images[J].IEEE Transactions on Information Theory，1974，20（4）：525-536.
[32] WANG Z，BOVIK A C，SHEIKH H R，et al.Image quality assessment：from error visibility to structural similarity[J].IEEE Transactions on Image Processing，2004，13（4）：600-612.
[33] RADFORD A，METZ L，CHINTALA S.Unsupervised representation learning with deep convolutional generative adversarial networks[J].arXiv：1511.06434，2015.