Image Semantic Segmentation Network Fusing Multi-Scale and Boundary Optimization

doi:10.3778/j.issn.1002-8331.2103-0586

Abstract

Abstract: About the problem that multiple convolution layers used on convolutional neural networks can cause the loss of small-scale targets in the image and the blurring of category boundaries. A stepped image semantic segmentation network structure based on multi-scale feature fusion and boundary optimization is proposed. The network aims to improve the accuracy of the network model, the spatial pooling pyramid module in the Deeplab V3+ network is optimized, and using the new activation function Funnel ReLU（FReLU） for the vision task to replace the original nonlinear activation function, which obtains accuracy compensation. Adding the optimization branch to build a ladder network and accurately predicting the boundaries of each category are used to improve the overall image segmentation accuracy and reduce the problem of the misrecognition within the category and the loss of small-scale targets in the prediction results. The experimental result on the Cityscapes dataset shows that the mean intersection over the improved network union has been significantly improved.

Key words: semantic segmentation, convolutional neural network, boundary optimization, Deeplab V3+, accuracy compensation

摘要： 针对卷积神经网络在多卷积层叠加造成的图像内小尺度目标丢失和类别边界模糊问题，提出一种基于多尺度特征融合和边界优化的阶梯型图像语义分割网络结构。该网络以提升网络模型的准确率为目标，对Deeplab V3+网络中空间池化金字塔模块进行优化，使用针对视觉任务的新激活函数Funnel ReLU（FReLU）替换原有非线性激活函数获取精度补偿，增添优化分支构建阶梯型网络，通过对各类别边界的精确预测提升整体图像分割准确率，减少预测结果中类内误识别和小尺度目标丢失问题。在Cityscapes数据集上的实验结果表明，改进后的网络各类别平均交并比指标均取得明显提升。

关键词: 语义分割, 卷积神经网络, 边界优化, Deeplab V3+, 精度补偿

LI Xin, ZHANG Hongying, LIU Hanyu. Image Semantic Segmentation Network Fusing Multi-Scale and Boundary Optimization[J]. Computer Engineering and Applications, 2022, 58(21): 250-257.

李鑫, 张红英, 刘汉玉. 融合多尺度和边界优化的图像语义分割网络[J]. 计算机工程与应用, 2022, 58(21): 250-257.

References

[1] CSURKA G，PERRONNION F.An efficient approach to semantic segmentation[J].International Journal of Computer Vision，2011，95（2）：198-212.
[2] 袁铭阳，黄宏博，周长胜.全监督学习的图像语义分割方法研究进展[J].计算机工程与应用，2021，57（4）：43-54.
YUAN M Y，HUANG H B，ZHOU C S.Research progress of image semantic segmentation based on fully supervised learning[J].Computer Engineering and Applications，2021，57（4）：43-54.
[3] HINTON G E，SALAKHUTDINOV R R.Reducing the dimensionality of data with neural networks[J].Science，2006，313（5786）：504-507.
[4] LONG J，SHELHAMER E，DARRELL T.Fully convolutional networks for semantic segmentation[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），Boston，MA，USA，2015：3431-3440.
[5] SUN W，WANG R.Fully convolutiona networks for semantic segmentation of very high resolution remotely sensed images combined with DSM[J].IEEE Geoscience and Remote Sensing Letters，2018，15（3）：474-478.
[6] RONNEBERGER O，FISCHER P，BROX T.U-Net：convolutional networks for biomedical image segmentation[C]//Medical Image Computing and Computer-Assisted Intervention，2015：234-241.
[7] CHEN L C，PAPANDREOU G，KOKKINOS I，et al.Semantic image segmentation with deep convolutional nets and fully connected CRFs[EB/OL].[2020-05-11].https：//arxiv.org/abs/1412.7062.
[8] CHEN L C，PAPANDREOU G，KOKKINOS I，et al.DeepLab：semantic image segmentation with deep convolutional nets，atrous convolution，and fully connected CRFs[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2018，40（4）：834-848.
[9] HE K，ZHANG X，REN S，et al.Spatial pyramid pooling in deep convolutional networks for visual recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，37（9）：1904-1916.
[10] CHEN L C，PAPANDREOU G，KOKKINOS I，et al.DeepLab：semantic image segmentation with deep convolutional nets，atrous convolution，and fully connected CRFs[J].IEEE Transactions on Pattern Analysis & Machine Intelligence，2016，40（4）：834-848.
[11] CHEN L C，ZHU Y K，PAPANDREOU G，et al.Enco-derdecoder with atrous separable convolution for semantic image segmentation[C]//European Conference on Computer Vision，2018：833-851.
[12] SUN K，XIAO B，LIU D，et al.Deep high-resolution re-presentation learning for human pose estimation[J].
arXiv：1902.09212，2019.
[13] MA N，ZHANG X，SUN J.Funnel activation for visual recognition[C]//Proceedings of the European Conference on Computer Vision（ECCV），2020：351-368.
[14] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），June 27-30，2016，Las Vegas，NV，USA.New York：IEEE，2016：770-778.
[15] YUAN Y H，XIE J Y，CHEN X L，et al.SegFix：model-agnostic boundary refiement for segmentation[C]//Proceedings of the European Conference on Computer Vision（ECCV），2020：489-506.
[16] CORDTS M，OMRAN M，RAMOS S，et al.The cityscapes dataset for semantic urban scene understanding[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition，2016：3213-3223.
[17] HARIHARAN B，ARBELAEZ P，BOURDEV L，et al.Semantic contours from inverse detectors[C]//2011 IEEE International Conference on Computer Vision（ICCV）.November 6-13，2011，Barcelona，Spain.New York：IEEE，2011：991-998.
[18] EVERINGHAM M，ESLAMI S M A，VAN GOOL L，et al.The pascal visual object classes challenge：a retrospective[J].International Journal of Computer Vision，2015，111（1）：98-136.
[19] BADRINARAYANAN V，HANDA A，CIPOLLA R.SegNet：a deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，39（12）：2481-2496.
[20] ZHAO H，SHI J，QI X，et al.Pyramid scene parsing network[C]//IEEE Conference on Computer Vision and Pattern Recognition，Hawaii，2017：2881-2890.
[21] XIAO T，LIU Y，ZHOU B，et al.Unified perceptual parsing for scene understanding[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：418-434.