DL-GAN生成对抗网络的半监督语义分割模型

doi:10.3778/j.issn.1002-8331.2307-0036

摘要/Abstract

摘要： 语义分割目前主流的全监督学习方式、数据质量和数量决定了网络的训练效果。只有花费大量的标注成本，才能得到质量高且数据量大的训练数据。根据上述情况，出现了基于半监督学习方式的语义分割。半监督学习可以节省数据的标注成本，很好地解决需要大量标注成本的问题，越来越多人开始关注半监督学习的图像语义分割。根据图像语义分割方法目前的发展现状，提出了一种结合DeepLabv2的生成对抗网络（DL-GAN）的半监督语义分割的模型。将DeepLabv2作为生成对抗网络的生成网络，完全卷积的网络作为判别器网络；对生成网络进行改进，首次将CBAM注意力机制和深度可分离卷积结合应用于DeepLabv2，将其作为生成网络，具体一是在DeepLabv2最后的卷积层前添加CBAM注意力机制，二是将DeepLabv2网络中Resnet残差块的标准卷积替换为深度可分离卷积，使整个模型将权重参数更为合理地分配，提高模型的表征能力且计算更为高效，加快训练效率；用空洞卷积替换判别器的标准卷积，提升整个判别器的感受野，提高训练效果，提升语义分割精度。该方法在PASCAL VOC 2012数据集上的实验结果相对于Affinitynet网络平均交并比提高6.3个百分点，证明了提出方法是有效的。

关键词: 生成对抗网络, 注意力机制, 语义分割, 深度可分离卷积

Abstract: Semantic segmentation is currently the mainstream fully supervised learning method, and the quality and quantity of data determine the training effect of the network. High-quality and large-scale training data can be obtained only by spending a lot of annotation costs. Based on the above situation, semantic segmentation based on semi-supervised learning emerged. Semi-supervised learning can save the cost of labeling data and solve the problem of requiring a large amount of labeling costs. More and more people are beginning to pay attention to semi-supervised learning for image semantic segmentation. Based on the current development status of image semantic segmentation methods, a semi-supervised semantic segmentation model combining DeepLabv2’s generative adversarial network (DL-GAN) is proposed. First, it uses DeepLabv2 as the generator network of the generative adversarial network, and a fully convolutional network as the discriminator network of the generator network. Secondly, it improves the generation network by applying the CBAM attention mechanism and deep separable convolution for the first time to DeepLabv2 as a generation network. Specifically, it adds the CBAM attention mechanism before the final convolutional layer of DeepLabv2, and replaces the standard convolution of Resnet residual blocks in the DeepLabv2 network with deep separable convolution, which makes the weight parameters of the entire model more reasonably distributed, improves the model’s representation ability and computational efficiency, and accelerates the training efficiency. Finally, replacing the standard convolution of the discriminator with a hole convolution improves the receptive field of the entire discriminator, enhances training effectiveness, and improves semantic segmentation accuracy. The experimental results of the proposed method on the PASCAL VOC 2012 dataset show an average improvement of 6.3 percentage points compared to the Affinitynet network, proving the effectiveness of the proposed method.

Key words: generative adversarial network, attention mechanism, semantic segmentation, deep separable convolution

刘凡, 段先华, 胡维康. DL-GAN生成对抗网络的半监督语义分割模型[J]. 计算机工程与应用, 2024, 60(19): 221-229.

LIU Fan, DUAN Xianhua, HU Weikang. DL-GAN Semi Supervised Semantic Segmentation Model for Generative Adversarial Network[J]. Computer Engineering and Applications, 2024, 60(19): 221-229.

参考文献

[1] 何雨岩. 基于深度学习的图像语义分割综述[J]. 长江信息通信, 2023, 36(1): 77-79.
HE Y Y. A survey of image semantic segmentation based on deep learning[J]. Yangtze River Information and Communication, 2023, 36(1): 77-79.
[2] 谭见文. 基于机器视觉的变电站巡检机器人自主导航技术研究[D]. 长春: 长春工程学院, 2022.
TAN J W. Research on autonomous navigation technology of substation inspection robot based on machine vision[D]. Changchun: Changchun Institute of Technology, 2022.
[3] 张晴晴, 史健芳. 基于语义分割网络的小样本表面缺陷检测[J]. 电子设计工程, 2021, 29(5): 180-189.
ZHANG Q Q, SHI J F. Small sample surface defect detection based on semantic segmentation network[J]. Electronic Design Engineering, 2021, 29(5): 180-189.
[4] 段续庭, 周宇康, 田大新, 等. 深度学习在自动驾驶领域应用综述[J]. 无人系统技术, 2021, 4(6): 1-27.
DUAN X T, ZHOU Y K, TIAN D X, et al. A review of the application of deep learning in the field of autonomous driving[J]. Unmanned System Technology, 2021, 4(6): 1-27.
[5] 周涛, 董雅丽, 霍兵强, 等. U-Net网络医学图像分割应用综述[J]. 中国图象图形学报, 2021, 26(9): 2058-2077.
ZHOU T, DONG Y L, HUO B Q, et al. Review of U-Net network medical image segmentation applications[J]. Chinese Journal of Image Graphics, 2021, 26(9): 2058-2077.
[6] KRIZHEVSKY A, SUTSKEVER I, HINTON G. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the NIPS, 2012.
[7] 胡潇菡. 基于生成对抗网络的半监督图像语义分割方法研究[D]. 西安: 西安电子科技大学, 2020.
HU X H. Research on semi supervised image semantic segmentation method based on generative adversarial network [D]. Xi’an: Xidian University, 2020
[8] KOLESNIKOV A, LAMPERT C H. Seed, expand and constrain: three principles for weakly-supervised image segmentation[C]//Proceedings of the European Conference on Computer Vision, 2016.
[9] WEI Y, LIANG X, CHEN Y, et al. STC: a simple to complex framework for weakly-supervised semantic segmentation[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(11): 2314-2320.
[10] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Advances in Neural Information Processing Systems, 2014, 27.
[11] 孙天鹏. 基于GAN的局部写实感漫画图像风格迁移方法研究[D]. 南京: 南京邮电大学, 2022.
SUN T P. Research on GAN based local realistic cartoon image style transfer method[D]. Nanjing: Nanjing University of Posts and Telecommunications, 2022.
[12] 盛煜炜. 基于条件生成对抗网络的数据集扩充在右心室分割中的应用研究[D]. 苏州: 苏州大学, 2021.
SHENG Y W. Research on the application of data set expansion based on conditional generative adversarial network in right ventricle segmentation[D]. Suzhou: Suzhou University, 2021.
[13] WANG P, BAI X. Thermal infrared pedestrian segmentation based on conditional GAN[J]. IEEE Transactions on Image Processing, 2019, 28(12): 6007-6021.
[14] HUNG W C, TSAI Y H, LIOU Y T, et al. Adversarial learning for semi-supervised semantic segmentation[C]//Proceedings of the British Machine Vision Conference (BMVC), 2018.
[15] RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks[J]. arXiv:1511.06434, 2015.
[16] BROCK A, DONAHUE J, SIMONYAN K. Large scale GAN training for high fidelity natural image synthesis[J]. arXiv:1809.11096, 2018.
[17] SHRIVASTAVA A, PFISTER T, TUZEL O, et al. Learning from simulated and unsupervised images through adversarial training[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2107-2116.
[18] ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE International Conference on Computer Vision, 2017: 2223-2232.
[19] 苏晨, 任志俊, 范彪, 等. 基于注意力机制与ResNet的残余奥氏体评级研究[J]. 轻工机械, 2023, 41(2): 78-84.
SU C, REN Z J, FAN B. Research on residual austenite rating based on attention mechanism and ResNet[J]. Light Industry Machinery, 2023, 41(2): 78-84.
[20] HOWARD A G, ZHU M, CHEN B, et al.MobileNets: efficient convolutional neural networks for mobile vision applications[J]. arXiv:1704.04861,2017.
[21] 王士斌, 高梓雕, 刘栋. 一种基于有限数据的改进DCGAN图像生成方法[J]. 河南师范大学学报(自然科学版), 2023,51(6): 39-46.
WANG S B, GAO Z D, LIU D. An improved DCGAN image generation method based on limited data[J]. Journal of Henan Normal University(Natural Science Edition), 2023,51(6): 39-46.
[22] 王可, 沈川贵, 罗孟华. 基于深度学习的图像语义分割方法综述[J]. 信息技术与信息化, 2022(4): 23-30.
WANG K, SHEN C G, LUO M H. Overview of image semantic segmentation methods based on deep learning[J]. Information Technology and Informationization, 2022(4): 23-30.
[23] AHN J, KWAK S. Learning pixel-level semantic affinity with image-level supervision for weakly supervised semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 4981-4990.
[24] WEI Y, FENG J, LIANG X, et al. Object region mining with adversarial erasing: a simple classification to semantic segmentation approach[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 1568-1576.