基于多尺度特征融合生成对抗网络的水下图像增强

doi:10.3778/j.issn.1002-8331.2304-0094

摘要/Abstract

摘要： 针对水下退化图像细节模糊、对比度低和蓝绿色偏问题，提出了一种基于多尺度特征融合生成对抗网络的水下图像增强算法。算法以生成对抗网络为基本框架，结合传统白平衡算法和多尺度增强网络实现对水下退化图像的增强。通过改进的通道补偿白平衡算法矫正蓝绿色偏，并以卷积神经网络提取偏色校正后图像的特征；提取图像多尺度特征，结合提出的残差密集块将每一层的局部特征增强为捕获语义信息的全局特征，并与偏色校正图像的特征相融合；通过重建模块将融合特征重建为清晰图像，恢复图像的细节信息。实验结果表明，该算法增强的水下图像去雾效果较好且颜色更真实，有效改善了水下图像色偏和模糊的问题，在主观指标和客观指标上的实验结果均优于对比算法。

关键词: 水下图像增强, 生成对抗网络, 多尺度, 特征融合

Abstract: Aiming at the problems of detail blur, low contrast and blue-green skew in underwater degraded images, an underwater image enhancement algorithm is proposed based on multi-scale feature fusion to generate antagonistic network. Based on the generated adversarial network, this algorithm combines the traditional white balance algorithm and multi-scale enhancement network to enhance underwater degraded images. Firstly, an white balance algorithm with improved channel compensation is used to correct the color bias of underwater degraded images, and the features of the corrected images are extracted by convolutional neural network. Then, the multi-scale features of the image are extracted, and the local features of each layer are enhanced to capture the global features of semantic information by combining with the proposed residual dense blocks, which are fused with the features of the color correction images. Finally, the fusion features are reconstructed into clear images by the reconstruction module, and the details of the images are recovered. The experimental results show that the enhanced underwater image with this algorithm has better defogging effect and more real color, and effectively improves the problem of color bias and blur of underwater image. The experimental results are superior to the comparison algorithm in both subjective and objective indicators.

Key words: underwater image enhancement, generative adversarial network, multiscale, feature fusion

陈辉, 王硕, 许家昌, 肖哲璇. 基于多尺度特征融合生成对抗网络的水下图像增强[J]. 计算机工程与应用, 2023, 59(21): 231-241.

CHEN Hui, WANG Shuo, XU Jiachang, XIAO Zhexuan. Underwater Image Enhancement Based on Generate Adversarial Network with Multiscale Feature Fusion[J]. Computer Engineering and Applications, 2023, 59(21): 231-241.

参考文献

[1] ZHANG W D，ZHUANG P X，SUN H H，et al.Underwater image enhancement via minimal color loss and locally adaptive contrast enhancement[J].IEEE Transactions on Image Processing，2022，31：3997-4010.
[2] RAVEENDRAN S，PATIL M D，BIRAJDAR G K.Underwater image enhancement：a comprehensive review，recent trends，challenges and applications[J].Artificial Intelligence Review，2021，54：5413-5467.
[3] 林成创，单纯，赵淦森，等.机器视觉应用中的图像数据增广综述[J].计算机科学与探索，2021，15（4）：583-611.
LIN C C，SHAN C，ZHAO G S，et al.Review of image data augmentation in computer vision[J].Journal of Frontiers of Computer Science and Technology，2021，15（4）：583-611.
[4] 胡振宇，陈琦，朱大奇.基于颜色平衡和多尺度融合的水下图像增强[J].光学精密工程，2022，30（17）：2133-2146.
HU Z Y，CHEN Q，ZHU D Q.Underwater image enhancement based on color balance and multi-scale fusion[J].Optics and Precision Engineering，2022，30（17）：2133-2146.
[5] 郭继昌，李重仪，郭春乐，等.水下图像增强和复原方法研究进展[J].中国图象图形学报，2017，22（3）：273-287.
GUO J C，LI C Y，GUO C L，et al.Research progress of underwater image enhancement and restoration methods[J].Journal of Image and Graphics，2017，22（3）：273-287.
[6] BUCHSBAUM G.A spatial processor model for object colour perception[J].Journal of the Franklin Institute，1980，310（1）：1-26.
[7] LAND E H.The retinex theory of color vision[J].Scientific American，1977，237（6）：108-129.
[8] ANCUTI C O，ANCUTI C，VLEESCHOUWER C D，et al.Color balance and fusion for underwater image enhancement[J].IEEE Transactions on Image Processing，2018，27（1）：379-393.
[9] WANG Y，SONG W，FORTINO G，et al.An experimental-based review of image enhancement and image restoration methods for underwater imaging[J].IEEE Access，2019，7：140233-140251.
[10] XU Y，WEN J，FEI L，et al.Review of video and image defogging algorithms and related studies on image restoration and enhancement[J].IEEE Access，2016，4：165-188.
[11] IQBAL K，ODETAYO M O，JAMES A E，et al.Enhancing the low quality images using unsupervised colour correction method[C]//IEEE International Conference on Systems，Man and Cybernetics，2010，4：1703-1709.
[12] PRABHAKAR C J，KUMAR P.Underwater image denoising using adaptive wavelet subband thresholding[C]//International Conference on Signal & Image Processing，2010：322-327.
[13] KHAN A，ALI S，MALIK A S，et al.Underwater image enhancement by wavelet based fusion[C]//2016 IEEE International Conference on Underwater System Technology：Theory and Applications（USYS），2016：83-88.
[14] 晋玮佩，郭继昌，祁清.基于条件生成对抗网络的水下图像增强[J].激光与光电子学进展，2020，57（14）：141002.
JIN W P，GUO J C，QI Q.Underwater image enhancement based on conditional generative adversarial network[J].Laser & Optoelectronics Progress，2020，57（14）：141002.
[15] LI J，SKINNER K A，EUSTICE R M.WaterGAN：unsupervised generative network to enable real-time color correction of monocular underwater images[J].IEEE Robotics and Automation Letters，2018，3（1）：387-394.
[16] ZHU J Y，PARK T，ISOLA P，et al.Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//2017 IEEE International Conference on Computer Vision（ICCV），2017：2242-2251.
[17] LIU X，GAO Z，CHEN B M.MLFcGAN：multi-level feature fusion based conditional gan for underwater image color correction[J].IEEE Geoscience and Remote Sensing Letters，2020，17（9）：1488-1492.
[18] 范新南，杨鑫，史朋飞，等.特征融合生成对抗网络的水下图像增强[J].计算机辅助设计与图形学学报，2022，34（2）：264-272.
FAN X N，YANG X，SHI P F，et al.Underwater image enhancement based on feature fusion generative adversaral networks[J].Journal of Computer-Aided Design & Computer Graphics，2022，34（2）：264-272.
[19] JIE H，LI S，GANG S.Squeeze-and-excitation networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2020，42（8）：2011-2023.
[20] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[21] GUO Y，LI H，ZHUANG P.Underwater image enhancement using a multiscale dense generative adversarial network[J].IEEE Journal of Oceanic Engineering，2020，45（3）：862-870.
[22] LIU X，GAO Z，CHEN B M.MLFcGAN：multi-level feature fusion based conditional GAN for underwater image color correction[J].IEEE Geoscience and Remote Sensing Letters，2020，17（9）：1488-1492.
[23] MAO X，LI Q，XIE H，et al.Least squares generative adversarial networks[C]//2017 IEEE International Conference on Computer Vision（ICCV），2017：2813-2821.
[24] ARJOVSKY M，CHINTALA S，BOTTOU L.Wasserstein GAN[J].arXiv：1701.07875，2017.
[25] GULRAJANI I，AHMED F，ARJOVSKY M，et al.Improved training of Wasserstein GANs[C]//Proceedings of 31st International Conference on Neural Information Processing Systems，2017：5767-5777.
[26] ISLAM M J，XIA Y，SATTAR J.Fast underwater image enhancement for improved visual perception[J].IEEE Robotics and Automation Letters，2020，5（2）：3227-3234.
[27] LI C，GUO C，REN W，et al.An underwater image enhancement benchmark dataset and beyond[J].IEEE Transactions on Image Processing，2020，29：4376-4389.
[28] HUMMEL R.Image enhancement by histogram transformation[J].Computer Graphics and Image Processing，1977，6（2）：184-195.
[29] ZUIDERVELD K.Contrast limited adaptive histogram equalization[M]//Graphics gems.USA：Academic Press Professional Inc，1994：474-485.
[30] DREWS J P，NASCIMENTO E，MORAES F，et al.Transmission estimation in underwater single images[C]//2013 IEEE International Conference on Computer Vision Workshops，2013：825-830.
[31] PENG Y T，COSMAN P C.Underwater image restoration based on image blurriness and light absorption[J].IEEE Transactions on Image Processing，2017，26（4）：1579-1594.
[32] FABBRI C，ISLAM M J，SATTAR J.Enhancing underwater imagery using generative adversarial networks[C]//2018 IEEE International Conference on Robotics and Automation（ICRA），2018：7159-7165.
[33] PANETTA K，GAO C，AGAIAN S.Human-visual-system-inspired underwater image quality measures[J].IEEE Journal of Oceanic Engineering，2016，41（3）：541-551.
[34] YANG M，SOWMYA A.An underwater color image quality evaluation metric[J].IEEE Transactions on Image Processing，2015，24（12）：6062-6071.