Research on Real-Time Segmentation Network Based on Attention and Edge Extraction Tasks

doi:10.3778/j.issn.1002-8331.2206-0255

Abstract

Abstract: Aiming at the problems of low segmentation accuracy of small object and blurring segmentation boundary, BiSeNet V3 is improved, and a real-time segmentation network based on global attention and edge extraction tasks is proposed. Firstly, combined with asymmetric convolution, the short-term dense concatenate module is designed for lightweight, and the global correlation between deep-layer features is enhanced through global attention module. Secondly, edge branching can effectively filter out the boundary independent information in semantic features through edge feature fuse module, and recover the detailed information in the decoding stage. Finally, the improved loss function ensures that the network can update parameters toward the direction conducive to small object segmentation. Comparative experiments on datasets and real environment prediction results show that the proposed network improves the boundary definition and small object segmentation accuracy, and still has high robustness in real scenes.

Key words: small object, boundary blur, global attention, edge extraction task

摘要： 针对小目标分割精度不高且分割边界模糊等问题，对BiSeNet V3进行改进，提出了一种基于全局注意力及边缘提取任务的实时分割网络。结合非对称卷积，对短期密集拼接模块做了轻量化处理，并通过全局注意力增强了特征的全局相关性。边缘分支可通过边缘特征融合模块有效滤除语义特征中与边界无关的信息，并在解码阶段恢复损失的细节信息。改进的损失函数保证了网络能够向着利于小目标分割的方向更新参数。在数据集上的结果及真实环境预测表明，所提网络改善了边界清晰度及小目标分割精度，并在真实场景下仍具有较高鲁棒性。

关键词: 小目标, 边界模糊, 全局注意力, 边缘提取任务

WEN Kai, YANG Yipeng, XIONG Junchen, WEI Shengnan. Research on Real-Time Segmentation Network Based on Attention and Edge Extraction Tasks[J]. Computer Engineering and Applications, 2023, 59(22): 213-222.

文凯, 杨一鹏, 熊俊臣, 韦胜男. 基于注意力及边缘提取任务的实时分割网络[J]. 计算机工程与应用, 2023, 59(22): 213-222.

References

[1] LIN G，MILAN A，SHEN C，et al.RefineNet：multi-path refinement networks for high-resolution semantic segmentation[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition，2017：1925-1934.
[2] CHEN L C，PAPANDREOU G，KOKKINOS I，et al.Semantic image segmentation with deep convolutional nets and fully connected crfs[J].arXiv：1412.7062，2014.
[3] LI X T，LI X，ZHANG L，et al.Improving semantic segmentation via decoupled body and edge supervision[C]//Proceedings of the 16th European Conference on Computer Vision.Cham：Springer，2020：435-452.
[4] TAKIKAWA T，ACUNA D，JAMPANI V，et al.Gated-SCNN：gated shape CNNs for semantic segmentation[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision，2019：5229-5238.
[5] HU X G，GONG Y.Lightweight asymmetric dilation network for real-time semantic segmentation[J].IEEE Access，2021，9：55630-55643.
[6] YANG Q H，CHEN T，FAN J Y，et al.EADNet：efficient asymmetric dilated network for semantic segmentation[C]//Proceedings of the 2021 IEEE International Conference on Acoustics，Speech and Signal Processing，2021：2315-2319.
[7] FAN M Y，LAI S Q，HUANG J S，et al.Rethinking BiSeNet for real-time semantic segmentation[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：9711-9720.
[8] FU J，LIU J，TIAN H J，et al.Dual attention network for scene segmentation[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：3146-3154.
[9] GAO G W，XU G A，YU Y，et al.MSCFNet：a lightweight network with multi-scale context fusion for real-time semantic segmentation[J].IEEE Transactions on Intelligent Transportation Systems，2022，23（12）：25489-25499.
[10] ZHAO Y Z，LIU Z G，HU Z H.Focal learning on stranger for imbalance image segmentation[J].IET Image Processing，2022，16（5）：1305-1323.
[11] ZHAO Y Z，LIU Z G，HU Z H.Dynamically balancing class losses in imbalanced deep learning[J].Electronics Letters，2022，58（5）：203-206.
[12] WANG X L，GIRSHICK R，GUPTA A，et al.Non-local neural networks[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition，Salt Lake City，2018：7794-7803.
[13] HU J，SHEN L，SUN G.Squeeze-and-excitation networks[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition，Salt Lake City，2018：7132-7141.
[14] CHEN L C，PAPANDREOU G，KOKKINOS I，et al.DeepLab：semantic image segmentation with deep convolutional nets，atrous convolution，and fully connected CRFs[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，40（4）：834-848.
[15] XIE S N，TU Z W.Holistically-nested edge detection[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision，Santiago，2015：1395-1403.
[16] LIN T Y，GOYAL P，GIRSHICK R，et al.Focal loss for dense object detection[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision，Venice，2017：2980-2988.
[17] PASZKE A，CHAURASIA A，KIM S，et al.ENet：a deep neural network architecture for real-time semantic segmentation[J].arXiv：1606.02147，2016.
[18] ROMERA E，ALVAREZ J M，BERGASA L M，et al.ERFNet：efficient residual factorized convnet for real-time semantic segmentation[J].IEEE Transactions on Intelligent Transportation Systems，2017，19（1）：263-272.
[19] MEHTA S，RASTEGARI M，CASPI A，et al.ESPNet：efficient spatial pyramid of dilated convolutions for semantic segmentation[C]//Proceedings of the 15th European Conference on Computer Vision，Munich，2018：552-568.
[20] LYU H，FU H Y，HU X J，et al.ESNet：edge-based segmentation network for real-time semantic segmentation in traffic scenes[C]//Proceedings of the 2019 IEEE International Conference on Image Processing，2019：1855-1859.
[21] LI G，YUN I，KIM J，et al.DABNet：depth-wise asymmetric bottleneck for real-time semantic segmentation[J].arXiv：1907.11357，2019.
[22] WANG Y，ZHOU Q，LIU J，et al.LEDNet：a lightweight encoder-decoder network for real-time semantic segmentation[C]//Proceedings of the 2019 IEEE International Conference on Image Processing，2019：1860-1864.
[23] CHEN P R，HANG H M，CHAN S W，et al.DSNet：an efficient CNN for road scene segmentation[J].APSIPA Transactions on Signal and Information Processing，2020，9：1-14.
[24] BADRINARAYANAN V，KENDALL A，CIPOLLA R.SEGNet：a deep convolutional encoder-decoder architecture for image segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，39（12）：2481-2495.
[25] LI H C，XIONG P F，FAN H Q，et al.DFANet：deep feature aggregation for real-time semantic segmentation[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition，Long Beach，2019：9522-9531.