Research on New Style Transfer Network for Sample Expansion

doi:10.3778/j.issn.1002-8331.2107-0516

Abstract

Abstract: The training of fully supervised semantic segmentation network needs a lot of manpower and time cost to label samples. Therefore, reducing the time of manually labeling samples and improving the effect of semantic segmentation are of great significance for the rapid deployment and application promotion of deep learning network. A sample expansion method based on improved image style transfer network（CycleGAN-AD） is proposed. Based on CycleGAN, the attention mechanism is introduced into the generator, and the depth residual network is changed into a densely connected convolution network. The computer is used to batch generate the simulation samples with their own labels, and the CycleGAN-AD network is used to transfer the simulation sample style to the real sample style（the label remains unchanged）, which is used to expand the training samples. The experimental results of semantic segmentation of steel seal characters on graphite electrode show that the segmentation effect is significantly improved after sample expansion with CycleGAN-AD network, and the MIoU value is up to 0.826?0. It can be seen that the sample expansion method proposed is hopeful to obtain high-quality training samples while significantly reducing the workload of manual annotation.

Key words: semantic segmentation, sample expansion, CycleGAN, style transfer

摘要： 全监督语义分割网络在训练时需要耗费大量的人力与时间成本来标注样本。所以减少人工标注样本的时间，同时提升语义分割效果，对于深度学习网络的快速部署和应用推广具有重要意义。提出一种基于改进图像风格迁移网络（CycleGAN-AD）的样本扩充方法。以CycleGAN为基础，在生成器中引入注意力机制并将深度残差网络改为密集连接卷积网络。利用计算机批量产生自带标签的模拟样本，使用CycleGAN-AD网络将模拟样本风格迁移成为真实样本风格（标签不变），并用于扩充训练样本。对石墨电极的钢印字符进行语义分割的实验结果表明，采用CycleGAN-AD网络进行样本扩充后，其分割效果得到显著提升，MIoU值最高升至0.826?0。可见，提出的样本扩充方法有希望在显著减少人工标注工作量的同时，获得高质量的训练样本。

关键词: 语义分割, 样本扩充, CycleGAN, 风格迁移

TIAN Min, LIU Mingguo, CHEN Lijia, HAN Zonghuan, LAN Tianxiang, LIANG Qian. Research on New Style Transfer Network for Sample Expansion[J]. Computer Engineering and Applications, 2023, 59(1): 228-235.

田敏, 刘名果, 陈立家, 韩宗桓, 兰天翔, 梁倩. 面向样本扩充的新型风格迁移网络研究[J]. 计算机工程与应用, 2023, 59(1): 228-235.

References

[1] LONG J，SHELHAMER E，DARRELL T.Fully convolutional networks for semantic segmentation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition，2015：3431-3440.
[2] RONNEBERGER O，FISCHER P，BROX T.U-Net：Convolutional networks for biomedical image Segmentation[J]. arXiv：1505.04597，2015.
[3] BADRINARAYANAN V，KENDALL A，CIPOLIA R.SegNet：A deep convolutional encoder-decoder architecture for scene segmentation[J].IEEE Transactions on Pattern Analysis & Machine Intelligence，2017，39（12）：2481-2495.
[4] NOH H，HONG S，HAN B.Learning deconvolution network for semantic segmentation[C]//Proceedings of 2015 IEEE International Conference on Computer Vision（ICCV），2015：234-241.
[5] CHEN L C，PAPANDREOU G，KOKKINOS I，et al.Deeplab：Semantic image segmentation with deep convolutional nets，atrous convolution，and fully connected CRFS[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2018，40（4）：834-848.
[6] CHEN L C，PAPANDREOU G，SCHROFF F，et al.Rethinking atrous convolution for semantic image segmentation[J].arXiv：1706.05587，2017.
[7] CHEN L C，ZHU Y K，PAPANDREOU G，et al.Encoder-decoder with atrous separable convolution for semantic image segmentation[J].arXiv：1802.02611，2018.
[8] 潘今一，王亚蒙，王伟，等.基于风格迁移和薄板样条的扩充汉字样本方法[J].浙江工业大学学报，2020，48（1）：25-29.
PAN J Y，WANG Y M，WANG W，et al.Extended Chinese character sample method based on style transfer and thin plate spline[J].Journal of Zhejiang University of Technology，2020，48（1）：25-29.
[9] 陈淮源，张广驰，陈高，等.基于深度学习的图像风格迁移研究进展[J].计算机工程与应用，2021，57（11）：37-45.
CHEN H Y，ZHANG G C，CHEN G，et al.Research progress of image style transfer based on deep learning[J].Computer Engineering and Applications，2021，57（11）：37-45.
[10] ZHU J Y，PARK T，ISOLA P，et al.Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision，2017：2223-2232.
[11] MIRZA M，OSINDERO S.Conditional generative adversarialnets[J].arXiv：1411.1784，2014.
[12] WOO S，PARK J，LEE J Y，et al.CBAM：Convolutional block attention module[C]//Proceedings of European Conference on Computer Vision，2018.
[13] HUANG G，LIU Z，MAATEN L，et al.Densely connected convolutional networks[C]//Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition，2017：4700-4798.
[14] ZHANG Z C，LIANG X K，DONG X，et al.A sparse-view CT reconstruction method based on combination of densenet and deconvotion[J].IEEE Transactions on Medical Imaging，2018，37（6）：1407-1417.
[15] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[16] 刘璟，宋海川，黄建设，等.通道与空间注意力图像超分辨率网络[J].计算机工程与应用，2021，57（2）：209-216.
LIU J，SONG H C，HUANG J S，et al.Channel and spatial attention image super resolution network[J].Computer Engineering and Applications，2021，57（2）：209-216.
[17] 夏国恩，唐琪，张显全.改进的多层感知机在客户流失预测中的应用[J].计算机工程与应用，2020，56（14）：257-263.
XIA G E，TANG Q，ZHANG X Q.Improved multi-layer perceptron applied to customer churn prediction[J].Computer Engineering and Applications，2020，56（14）：257-263.
[18] ISOLA P，ZHU J Y，ZHOU T，et al.Image-to-image translation with conditional adversarial networks[C]//Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition，2017：1125-1134.