基于边缘增强的遥感图像弱监督语义分割方法

doi:10.3778/j.issn.1002-8331.2203-0122

摘要/Abstract

摘要： 随着弱监督学习被应用于遥感图像语义分割，大大降低了模型训练的数据成本。然而，由于监督信息不足，类激活图难以准确激活出遥感图像中不同尺度大小的目标，这使得基于类激活图获得的伪分割掩码边缘粗糙，从而导致最终的分割结果不准确。此外，大部分的弱监督语义分割方法都是基于可视化的两阶段方法，模型复杂繁琐。针对上述问题，设计了一种基于边缘增强的端到端弱监督语义分割网络。在特征空间边缘增强模块中，以自监督方式引导网络学习遥感图像中尺寸不一的目标，并且，细化伪分割掩码的边缘；在输出空间边缘增强模块中，通过端到端训练提升分割精度，同时降低模型训练的繁琐度。在ISPRS 2D数据集上的实验结果表明，该方法在仅使用图像级标签的情况下MIoU分别为57.72%和59.45%，与其他方法相比，效果较好。

关键词: 弱监督学习, 遥感图像语义分割, 自监督学习, 边缘增强, 端到端

Abstract: With the application of weakly supervised learning in remote sensing image semantic segmentation, the data cost of model training is greatly reduced. However, due to the lack of supervision information, the class activation map is difficult to accurately activate the targets with different scales in the remote sensing image, which makes the edge of the pseudo segmentation mask obtained based on the class activation map rough, resulting in the inaccurate final segmentation result. In addition, most of the weakly supervised semantic segmentation methods are two-stage methods based on visualization, and the model is complex and cumbersome. To solve the above problems, this paper designs an end-to-end weakly supervised semantic segmentation network based on edge enhancement. Firstly, this paper proposes a feature space edge enhancement module, which guides the network to learn the targets with different sizes in the remote sensing image in a self-monitoring way, and refines the edge of the pseudo segmentation mask. Secondly, this paper designs the output space edge enhancement module. Through end-to-end training, it not only improves the segmentation accuracy, but also reduces the complexity of model training. The experimental results on the ISPRS 2D dataset show that the MIoU of this method is 57.72% and 59.45% respectively when only image level labels are used, which is better than other methods.

Key words: weakly supervised learning, semantic segmentation of remote sensing image, self-supervised learning, edge enhancement, end-to-end

栾晓梅, 刘恩海, 武鹏飞, 张军. 基于边缘增强的遥感图像弱监督语义分割方法[J]. 计算机工程与应用, 2022, 58(20): 188-196.

LUAN Xiaomei, LIU Enhai, WU Pengfei, ZHANG Jun. Weakly-Supervised Semantic Segmentation Method of Remote Sensing Images Based on Edge Enhancement[J]. Computer Engineering and Applications, 2022, 58(20): 188-196.

参考文献

[1] LONG J，SHELHAMER E，DARRELL T.Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2015：3431-3440.
[2] RONNEBERGER O，FISCHER P，BROX T.U-net：convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham：Springer，2015.
[3] BADRINARAYANAN V，KENDALL A，CIPOLLA R.Segnet：a deep convolutional encoder-decoder architecture for image segmentation[J].IEEE Transactions on Pattern Analy-
sis and Machine Intelligence，2017，39（12）：2481-2495.
[4] CHEN L C，PAPANDREOU G，KOKKINOS I，et al.Deep-
lab：semantic image segmentation with deep convolutional nets，atrous convolution，and fully connected crfs[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，40（4）：834-848.
[5] 徐辉，祝玉华，甄彤，等.深度神经网络图像语义分割方法综述[J].计算机科学与探索，2021，15（1）：47-59.
XU Hui，ZHU Yuhua，ZHEN Tong，et al.Survey of image semantic segmentation methods based on deep neural network[J].Journal of Frontiers of Computer Science and Technology，2021，15（1）：47-59.
[6] 马宇，张丽果，杜慧敏，等.卷积神经网络的交通标志语义分割[J].计算机科学与探索，2021，15（6）：1114-1121.
MA Yu，ZHANG Guoli，DU Huimin，et al.Traffic sign semantic segmentation based on convolutional neural network[J].Journal of Frontiers of Computer Science and Technology，2021，15（6）：1114-1121.
[7] 王鑫，张昊宇，凌诚.基于U-Net优化的SAR遥感图像语义分割[J].计算机科学，2021，48（S2）：376-381.
WANG Xin，ZHANG Haoyu，LING Cheng.Semantic segmentation of SAR remote sensing image based on U-Net optimization[J].Computer Science，2021，48（S2）：376-381.
[8] YAN M，WANG J，LI J，et al.Traffic scene semantic segmentation using self-attention mechanism and bi-directional GRU to correlate context[J].Neurocomputing，2020，386：293-304.
[9] ZHANG W，TANG P，CORPETTI T，et al.WTS：a weakly towards strongly supervised learning framework for remote sensing land cover classification using segmentation models[J].Remote Sensing，2021，13（3）：394.
[10] ZHOU B，KHOSLA A，LAPEDRIZA A，et al.Learning deep features for discriminative localization[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：2921-2929.
[11] FU K，LU W，DIAO W，et al.WSF-NET：weakly supervised feature-fusion network for binary segmentation in remote sensing image[J].Remote Sensing，2018，10（12）：1970.
[12] 赵斐，张文凯，闫志远，等.基于多特征图金字塔融合深度网络的遥感图像语义分割[J].电子与信息学报，2019，41（10）：2525-2531.
ZHAO Fei，ZHANG Wenkai，YAN Zhiyuan，et al.Multi-feature map pyramid fusion deep network for semantic segmentation on remote sensing data[J].Journal of Electronics and Information Technology，2019，41（10）：2525-2531.
[13] 张静，靳淇兆，王洪振，等.多尺度信息融合的遥感图像语义分割模型[J].计算机辅助设计与图形学学报，2019，31（9）：1509-1517.
ZHANG Jing，JIN Qizhao，WANG Hongzhen，et al.Semantic segmentation on remote sensing images with multi-scale feature fusion[J].Journal of Computer-Aided Design & Computer Graphics，2019，31（9）：1509-1517.
[14] MA F，GAO F，SUN J，et al.Weakly supervised segmentation of SAR imagery using superpixel and hierarchically adversarial CRF[J].Remote Sensing，2019，11（5）：512.
[15] CHEN J，HE F，ZHANG Y，et al.SPMF-Net：weakly supervised building segmentation by combining superpixel pooling and multi-scale feature fusion[J].Remote Sensing，2020，12（6）：1049.
[16] KR?HENBüHL P，KOLTUN V.Efficient inference in fully connected crfs with Gaussian edge potentials[C]//Advances in Neural Information Processing Systems，2011：109-117.
[17] 左宗成，张文，张东映.融合可变形卷积与条件随机场的遥感影像语义分割方法[J].测绘学报，2019，48（6）：718-726.
ZUO Zongcheng，ZHANG Wen，ZHANG Dongying.A remote sensing image semantic segmentation method by combining deformable convolution with conditional random fields[J].Acta Geodaetica et Cartographica Sinica，2019，48（6）：718-726.
[18] 熊昌镇，智慧.融合多尺度信息的弱监督语义分割及优化[J].通信学报，2019，40（1）：163-171.
XIONG Changzhen，ZHI Hui.Weakly supervised semantic segmentation and optimization algorithm based on multi-scale feature model[J].Journal on Communications，2019，40（1）：163-171.
[19] 陈琴，朱磊，吕燧栋，等.高分辨率遥感影像的边缘损失增强地物分割[J].中国图象图形学报，2021，26（3）：674-685.
CHEN Qin，ZHU Lei，LV Suidong，et al.Segmentation of high-resolution remote sensing image by collaborating with edge loss enhancement[J].Journal of Image and Graphics，2021，26（3）：674-685.
[20] NIVAGGIOLI A，RANDRIANARIVO H.Weakly supervised semantic segmentation of satellite images[C]//2019 Joint Urban Remote Sensing Event（JURSE），2019：1-4.
[21] AHN J，KWAK S.Learning pixel-level semantic affinity with image-level supervision for weakly supervised semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：4981-4990.
[22] JING L，TIAN Y.Self-supervised visual feature learning with deep neural networks：a survey[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2021，43（11）：4037-4058.
[23] WANG Y，ZHANG J，KAN M，et al.Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：12275-12284.
[24] SHIMODA W，YANAI K.Self-supervised difference detection for weakly-supervised semantic segmentation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：5208-5217.
[25] 刘雨溪，张铂，王斌.基于生成式对抗网络的遥感图像半监督语义分割[J].红外与毫米波学报，2020，39（4）：473-482.
LIU Yuxi，ZHANG Bo，WANG Bin.Semi-supervised semantic segmentation based on generative adversarial networks for remote sensing images[J].Journal of Infrared and Millimeter Waves，2020，39（4）：473-482.
[26] GERKE M.Use of the stair vision library within the ISPRS 2D semantic labeling benchmark（Vaihingen）[Z].2014.
[27] WU Z，SHEN C，VAN DEN HENGEL A.Wider or deeper：revisiting the resnet model for visual recognition[J].Pattern Recognition，2019，90：119-133.