Research on Road Scene Semantic Segmentation Based on Improved CycleGAN

doi:10.3778/j.issn.1002-8331.2012-0245

Abstract

Abstract: Semantic segmentation in road scenes is a key technology in unmanned driving, and it is also an important field in computer vision, but traditional semantic segmentation methods require pixel-level labeling of training data, which has extremely high data requirements. To address this problem, an improved cycle-consistent adversarial network is used for semantic segmentation, the network avoids a large number of pixel-level annotations and does not require paired data sets, which reduces the requirements of data sets. Replace the objective function of the original network with least square loss and Smooth L1 norm, which increases the stability of network training and improves the quality of the generated image, the introduction of identity loss ensures the preservation of image features, making the generated image more realistic. The cityscapes dataset commonly used in road segmentation is used to conduct experiments, and the effectiveness of the method is verified by performance evaluation indicators commonly used in segmentation, the experimental results show that each performance has a certain improvement compared with the original network.

Key words: semantic segmentation, cycle-consistent adversarial networks, loss function, image generate

摘要： 道路场景下的语义分割是无人驾驶中关键的技术，也是计算机视觉中重要的一个领域，而传统的语义分割方法需要对训练数据进行像素级的标注，对数据的要求极高。针对这一问题，将改进的循环生成对抗网络（cycle-consistent adversarial networks，CycleGAN）用于道路场景语义分割，该网络避免了大量的像素级标注且不需要成对的数据集，降低了数据集的要求。将原网络的目标函数用最小二乘损失和Smooth L1范数替代，增加了网络训练的稳定性且提高了生成图像的质量，并引入特征损失保证图像特征的保留，使得生成图像更加真实。使用道路场景分割中常用的Cityscapes数据集进行实验，并用语义分割领域常用的性能评价指标验证了方法的有效性，实验结果表明相较于原网络各性能都有一定提升。

关键词: 语义分割, 循环生成对抗网络, 损失函数, 图像生成

ZHANG Rutao, HUANG Shan, WANG Honghao. Research on Road Scene Semantic Segmentation Based on Improved CycleGAN[J]. Computer Engineering and Applications, 2022, 58(15): 278-284.

张如涛, 黄山, 汪鸿浩. 基于改进CycleGAN的道路场景语义分割研究[J]. 计算机工程与应用, 2022, 58(15): 278-284.

References

[1] CHENG G，ZHENG J Y，KILICARSLAN M.Semantic segmentation of road profiles for efficient sensing in autonomous driving[C]//2019 IEEE Intelligent Vehicles Symposium（IV），Paris，France，2019：564-569.
[2] 吴玉超，林岚，王婧璇，等.基于卷积神经网络的语义分割在医学图像中的应用[J].生物医学工程学杂志，2020，37（3）：533-540.
WU Y C，LIN L，WANG J X，et al.Application of semantic segmentation based on convolutional neural network in medical images[J].Journal of Biomedical Engineering，2020，37（3）：533-540.
[3] LI W，HE C，FANG J，et al.Semantic segmentation-based building footprint extraction using very high-resolution satellite images and multi-source GIS data[J].Remote Sensing，2019，11（4）：403.
[4] LONG J，SHELHAMER E，DARRELL T.Fully convolutional networks for semantic segmentation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2015：3431-3440.
[5] MEI J，CHEN J，YAO W，et al.Supervised learning for semantic segmentation of 3D lidardata[C]//2019 IEEE Intelligent Vehicles Symposium（IV），Paris，France，2019：1491-1498.
[6] KIM M，JO H，RA M，et al.Weakly-supervised defect segmentation on periodic textures using cyclegan[J].IEEE Access，2020，8：176202-176216.
[7] GOODFELLOW I，POUGET-ABADIE J，MIRZA M，et al.Generative adversarial nets[C]//Advances in Neural Information Processing Systems，2014：2672-2680.
[8] ZHOU X，BAI T，HAN Y.Vision-based robot navigation through combining unsupervised learning and hierarchical reinforcement learning[J].Sensors，2019，19（7）：1576.
[9] LUC P，COUPRIE C，CHINTALA S，et al.Semantic segmentation using adversarial networks[J].arXiv：1611.08408，2016.
[10] METZEN J H，KUMAR M C，BROX T，et al.Universal adversarial perturbations against semantic image segmentation[C]//Proceedings of the IEEE International Conference on Computer Vision（ICCV）.Venice，Italy：IEEE，2017.2755-2764.
[11] ZHU J Y，PARK T，ISOLA P，et al.Unpaired image-to-imagetranslation using cycle-consistentadversarial networks[C]//Proceedings of the IEEE International Conference on Computer Vision（ICCV），2017：2223-2232.
[12] MAO X，LI Q，XIE H，et al.Least squares generative adversarial networks[C]//Proceedings of the IEEE International Conference on Computer Vision（ICCV），2017：2794-2802.
[13] HE K，GKIOXARI G，DOLLAR P，et al.Mask R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision（ICCV），2017：2961-2969.
[14] TAIGMAN Y，POLYAK A，WOLF L.Unsupervised cross-domain image generation[J].arXiv：1611.02200，2016.
[15] PATHAK D，KPAHENBUHL P，DONAHUE J，et al.Contex encoders：feature learning by impainting[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR）.Piscataway：IEEE，2016：2536-2544.
[16] SUN R Y.Optimization for deep learning：theory and algorithms[J].arXiv：1912.08957，2019.
[17] WU Y X，HE K M.Group normalization[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：3-19.
[18] CORDTS M，OMRAN M，RAMOS S，et al.The cityscapes dataset for semantic urban scene understanding[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition，2016：3213-3223.
[19] GARCIA-GARCIA A，ORTS-ESCOLANO S，OPREA S，et al.A review on deep learning techniques applied to semantic segmentation[J].arXiv：1704.06857，2017.