Unsupervised Segmentation Algorithm Based on Multi-Scale Feature Fusion and Novel Discriminator

doi:10.3778/j.issn.1002-8331.2111-0459

Abstract

Abstract: In the process of intelligent upgrading of factories, there are many application scenarios that need to use semantic segmentation algorithms. However, the use of fully supervised semantic segmentation methods requires a lot of labor cost for sample labeling, so it is necessary to study unsupervised semantic segmentation methods. Aiming at the semantic segmentation of characters imprinted by graphite electrodes in a local carbon factory, an unsupervised semantic segmentation method CycleGAN-Seg is proposed. Combining the idea of cross-layer connection and atrous spatial pooling pyramid（ASPP）, a novel multi-scale feature fusion generator is constructed, and an improved attention module is added to improve the network performance. At the same time, a new U-shaped discriminator is proposed to discriminate the reconstructed images. In the semantic segmentation experiment of the imprinted character dataset on the graphite electrode surface, the MIoU value can reach 70.81%. The semantic segmentation effect basically meets the recognition needs, and it is expected to replace the fully supervised learning method in this industrial scenario to save the cost of manual annotation and achieve the purpose of rapid training and deployment.

Key words: multi-scale feature fusion, attentional module, unsupervised segmentation, surface imprint characters

摘要： 工厂在智能化升级过程中，有很多应用场景需要用到语义分割。然而使用全监督语义分割方法需要耗费大量人力成本进行样本标注，所以研究无监督语义分割方法很有必要。针对本地某碳素厂石墨电极压印字符的语义分割问题，提出了一种无监督语义分割方法CycleGAN-Seg。结合跨层连接和空洞空间池化金字塔（ASPP）的思想，构建了新型多尺度特征融合生成器，加入了改进的注意力模块以提升网络性能。同时提出一种新的U形判别器对重构图像进行判别。在石墨电极表面压印字符数据集语义分割实验中，MIoU值可达70.81%，分割效果基本满足识别需要，有望在该工业场景中替代全监督学习方法，以节省人工标注成本，达到快速训练和部署的目的。

关键词: 多尺度特征融合, 注意力模块, 无监督分割, 表面压印字符

HAN Zonghuan, LIU Mingguo, LI Shen, CHEN Lijia, TIAN Min, LAN Tianxiang, LIANG Qian. Unsupervised Segmentation Algorithm Based on Multi-Scale Feature Fusion and Novel Discriminator[J]. Computer Engineering and Applications, 2023, 59(7): 152-162.

韩宗桓, 刘名果, 李珅, 陈立家, 田敏, 兰天翔, 梁倩. 多尺度特征融合与新型判别器的无监督分割[J]. 计算机工程与应用, 2023, 59(7): 152-162.

References

[1] LONG J，SHELHAMER E，DARRELL T.Fully convolu-tional networks for semantic segmentation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Boston：IEEE Press，2015：3431-3440.
[2] RONNEBERGER O，FISCHER P，BROX T.U-Net：convolutional networks for biomedical image segmentation[J].Medical Image Computing and Computer-Assisted Intervention，2015：1520-1528.
[3] CHEN L C，PAPANDREOU G，KOKKINOS I，et al.Semantic image segmentation with deep convolutional nets and fully connected CRFS[C]//Proceedings of International Conference in Learning Representations，2015.
[4] CHEN L C，PAPANDREOU G，KOKKINOS I，et al.DeepLab：semantic image segmentation with deep convolutional nets，atrous convolution，and fully connected crfs[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2018，40（4）：834-848.
[5] CHEN L C，ZHU Y，PAPANDREOU G，et al.Rethinking atrous convolution for semantic image segmentation[J].arXiv：1706.05587，2017.
[6] CHEN L C，PAPANDREOU G，et al.Encoder-decoder with atrous separable convolution for semantic image segmentation[J].arXiv：1802.02611，2018.
[7] BADRINARAYANAN V，KENDALL A，CIPOLLA R.SegNet：a deep convolutional encoder-decoder architecture for image segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，39（12）：2481-2495.
[8] PASZKE A，CHAURASIA A，KIM S，et al.ENet：a deep neural network architecture for real-time semantic segmentation[C]//Proceedings of the International Conference on Learning Representations，2017.
[9] ZHAO H，ZHANG Y，LIU S，et al.PSANet：point-wise spatial attention network for scene parsing[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：267-283.
[10] WANG X，GIRSHICK R，GUPTA A，et al.Non-local neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：7794-7803.
[11] VISIN F，CICCONE M，ROMERO A，et al.ReSeg：a recurrent neural network-based model for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops，2016：41-48.
[12] LI Z，GAN Y，LIANG X，et al.LSTM-CF：unifying context modeling and fusion with lstms for RGB-D scene labeling[C]//Proceedings of European Conference on Computer Vision，2016：541-557.
[13] 马冬梅，贺三三，杨彩锋，等.特征融合型卷积神经网络的语义分割[J].计算机工程与应用，2020，56（10）：193-198.
MA D M，HE S S，YANG C F，et al.Semantic segmentation based on convolutional neural networks with feature fusion[J].Computer Engineering and Applications，2020，56（10）：193-198.
[14] 宋廷强，刘童心，宗达，等.改进U-Net网络的遥感影像道路提取方法研究[J].计算机工程与应用，2021，57（14）：209-216.
SONG T Q，LIU T X，ZONG D，et al.Research on road extraction method from remote sensing images based on improved U-net network[J].Computer Engineering and Applications，2021，57（14）：209-216.
[15] GOODFELLOW I J，POUGET-ABADIE J，MIRZA M，et al.Generative adversarial networks[J].arXiv：1406.2661，2014.
[16] NASH J.Non-cooperative games[C]//Proceedings of Annals of Mathematics，1951：286-295.
[17] 杨培健，吴晓富，张索非，等.基于空洞卷积鉴别器的语义分割迁移算法[J].计算机科学，2020，47（11）：174-178.
YANG P J，WU X F，ZHANG S F，et al.Semantic segmentation transfer algorithm based on atrous convolution discriminator[J].Computer Science，2020，47（11）：174-178.
[18] PEI Z，CAO Z，LONG M，et al.Multi-adversarial domain adaptation[C]//Proceedings of the 32nd Conference on Artificial Intelligence，New Orleans，Louisiana，2018：3934-3941.
[19] LUO Y，ZHENG L，GUAN T，et al.Taking a closer look at domain shift：category-level adversaries for semantics consistent domain adaptation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：2507-2516.
[20] WANG S，YU L，YANG X，et al.Patch-based output space adversarial learning for joint optic disc and cup segmentation[J].IEEE Transactions on Medical Imaging，2019，38（11）：2485-2495.
[21] WANG S，YU L，LI K，et al.Boundary and entropy-driven adversarial learning for fundus image segmentation[C]//Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention，2019：102-110.
[22] ARASLANOV N，ROTH S.Self-supervised augmentation consistency for adapting semantic segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：15384-15394.
[23] ZHU J，PARK T，ISOLA P，et al.Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision，2017：2242-2251.
[24] WOO S，PARK J，LEE J Y，et al.CBAM：convolutional block attention module[C]//Proceedings of the 15th European Conference on Computer Vision，Munich，Sep 8-14，2018：3-19.
[25] OTSU N.A threshold selection method from gray-level histograms[J].IEEE Transactions on Systems，Man，and Cybernetics，1979，9（1）：62-66.
[26] ISOLA P，ZHU J Y，ZHOU T，et al.Image-to-image translation with conditional adversarial networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：1125-1134.