遥感图像语义分割中的弱监督域自适应算法

doi:10.3778/j.issn.1002-8331.2104-0259

摘要/Abstract

摘要： 近年来，遥感图像的语义分割得到广泛应用。虽然基于深度学习的方法极大程度地提高了遥感图像的语义分割精度，但由于遥感图像的多样性（不同的地理位置、地形和天气条件）以及像素级标签的缺乏，该算法难以适用于跨多个域的任务。而若重新训练新的域则需要消耗大量人力资源来收集相应的像素级标签。为了解决这一跨域问题，基于对抗学习提出了一种目标域图像级标签已知的弱监督域自适应方法：利用图像级标签，提出多类域判别器，使目标域各个类别自适应地对齐到源域；为保持训练过程的平稳，一种基于熵值产生目标域像素级伪标签的方法被提出以监督特征提取器，提升模型表现。在ISPRS Vaihingen与ISPRS Potsdam数据集上的大量实验表明，改进后的算法优于其他遥感图像语义分割中的基于对抗学习域自适应算法。

关键词: 语义分割, 遥感图像, 地物分类, 弱监督域自适应

Abstract: In recent years, semantic segmentation of remote sensing images has been widely used. Although the method based on deep learning has greatly improved the semantic segmentation accuracy of remote sensing image, due to the diversity of remote sensing images（different geographic locations, terrain and weather conditions） and lack of pixel-level labels, the algorithm is difficult to apply across domains task. Retraining a new domain needs to collect the corresponding pixel-level labels, which consumes lots of human resources. So based on adversarial learning, a weakly-supervised domain adaptative algorithm with known image-level labels of the target domain is proposed：Firstly, using image-level labels, a multi-class domain discriminator is proposed to align each category of the target domain to the source domain adaptively. Secondly, to maintain the stability of the training process, a method to generate pixel-level pseudo-labels of the target domain based on the entropy value is proposed to supervise the feature extractor and improve the performance of model. Numerous experiments on ISPRS Vaihingen and ISPRS Potsdam datasets show that the improved algorithm is superior to other adaptive algorithms based on the adversarial learning domain in the semantic segmentation of remote sensing images.

Key words: semantic segmentation, remote sensing image, classification of ground objects, weakly-supervised domain adaptation

丁一鹏, 赵璐. 遥感图像语义分割中的弱监督域自适应算法[J]. 计算机工程与应用, 2022, 58(22): 195-202.

DING Yipeng, ZHAO Lu. Weakly-Supervised Domain Adaptive Algorithm for Remote Sensing Image Semantic Segmentation[J]. Computer Engineering and Applications, 2022, 58(22): 195-202.

参考文献

[1] MICHAE K，ARNT B.Semantic segmentation of small objects and modeling of uncertainty in urban remote sensing images using deep convolutional neural networks[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition，2016：680-688.
[2] SUN W W，WANG R S.Fully convolutional networks for semantic segmentation of very high resolution remotely sensed images combined with DSM[J].IEEE Geoscience and Remote Sensing Letters，2018，15（3）：474-478.
[3] MOU L C，HUA Y S，ZHU X.A relation-augmented fully convolutional network for semantic segmentation in aerial scenes[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：12408-12417.
[4] 徐辉，祝玉华，甄彤，等.深度神经网络图像语义分割方法综述[J].计算机科学与探索，2021，15（1）：47-59.
XU H，ZHU Y H，ZHEN T，et al.Survey of image semantic segmentation methods based on deep neural network[J].Journal of Frontiers of Computer Science and Technology，2021，15（1）：47-59.
[5] SIMONYAN K，ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].arXiv：1409.
1556，2014.
[6] HE K，ZHANG X，Ren S，et al.Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[7] JONATHAN L，EVAN S，TREVOR D.Fully convolutional networks for semantic segmentation[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition，2015：3431-3440.
[8] 张哲晗，方薇，杜丽丽，等.基于编码-解码卷积神经网络的遥感图像语义分割[J].光学学报，2020，40（3）：46-55.
ZHANG Z H，FANG W，DU L L，et al.Semantic segmentation of remote sensing images based on encoding-decoding convolutional neural network[J].Acta Optica Sinica，2020，40（3）：46-55.
[9] CHEN L C，PAPANDREOU G，KOKKINOS I，et al.Deep-
Lab：semantic image segmentation with deep convolutional nets，atrous convolution，and fully connected CRFs[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2018，40（4）：834-848.
[10] CHEN L C，et al.Rethinking atrous convolution for semantic image segmentation[J].arXiv：1706.05587，2017.
[11] ZHAO H，SHI J，QI X，et al.Pyramid scene parsing network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition，2017：6230-6239.
[12] RONNEBERGER O，FISCHER P，BROX T.U-Net：convolutional networks for biomedical image segmentation[C]//18th International Conference on Medical Image Computing and Computer-Assisted Intervention，2015：234-241.
[13] 苏健民，杨岚心，景维鹏.基于U-Net的高分辨率遥感图像语义分割方法[J].计算机工程与应用，2019，55（7）：207-213.
SU J M，YANG L X，JING W P.Semantic segmentation method of high-resolution remote sensing image based on U-Net[J].Computer Engineering and Applications，2019，55（7）：207-213.
[14] WANG X，GIRSHICK R，GUPTA A，et al.Non-local neural networks[C]//2018 IEEE Conference on Computer Vision and Pattern Recognition，2018：7794-7803.
[15] FU J，LIU J，TIAN H，et al.Dual attention network for scene segmentation[C]//2019 IEEE Conference on Computer Vision and Pattern Recognition，2019：3141-3149.
[16] HUANG Z，WANG X，HUANG L，et al.CCNet：criss-cross attention for semantic segmentation[C]//2019 International Conference on Computer Vision，2019：603-612.
[17] IQBAL J，ALI M.Weakly-supervised domain adaptation for built-up region segmentation in aerial and satellite imagery[J].ISPRS Journal of Photogrammetry and Remote Sensing，2020，167：263-275.
[18] HOFFMAN J，WANG D Q，YU F，et al.FCNs in the wild：pixel-level adversarial and constraint-based adaptation[J].arXiv：1612.02649，2016.
[19] TSAI Y H，HUNG W C，SCHULTER S，et al.Learning to adapt structured output space for semantic segmentation[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2018：7472-7481.
[20] LUO Y W，ZHENG L，GUAN T，et al.Taking a closer look at domain shift：category-level adversaries for semantics consistent domain adaptation[C]//2019 IEEE Conference on Computer Vision and Pattern Recognition，2019：250.
[21] LUO Y W，LIU P，GUAN T，et al.Significance-aware information bottleneck for domain adaptive semantic segmentation[C]//2019 IEEE/CVF International Conference on Computer Vision，2019：677.
[22] TASAR O，HAPPY S L，TARABALKA Y，et al.ColorMapGAN：unsupervised domain adaptation for semantic segmentation using color mapping generative adversarial networks[J].IEEE Transactions on Geoscience and Remote Sensing，2020，58（10）：7178-7193.
[23] ZHU J Y，PARK T，ISOLA P，et al.Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//2017 IEEE International Conference on Computer Vision，2017：2223-2232.
[24] YANG Y C，SOATTO S.FDA：Fourier domain adaptation for semantic segmentation[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：4084-4094.
[25] LIU M Y，BREUEL T，KAUTZ J.Unsupervised image-to-image translation networks[C]//Advances in Neural Information Processing Systems 30，2017：700-708.
[26] HUANG H S，HUANG Q X，KR?HENBüHL P.Domain transfer through deep activation matching[C]//15th European Conference on Computer Vision，2018：611-626.
[27] VU T H，JAIN H，BUCHER M，et al.ADVENT：adversarial entropy minimization for domain adaptation in semantic segmentation[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2018：2517-2526.
[28] ZOU Y，YU Z，VIJAYA K B，et al.Unsupervised domain adaptation for semantic segmentation via class-balanced self-training[C]//15th European Conference on Computer Vision，2018：289-305.
[29] DU L，TAN J，YANG H，et al.SSF-DAN：separated semantic feature based domain adaptation network for semantic segmentation[C]//2019 IEEE International Conference on Computer Vision，2019：982-991.
[30] SUBHAIN M N，ALI M.Learning from scale-invariant examples for domain adaptation in semantic segmentation[J].arXiv：2007.14449，2020.
[31] ZHANG Y，QIU Z，YAO T，et al.Fully convolutional adaptation networks for semantic segmentation[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2018：6810-6818.
[32] WANG Z H，YU M，WEI Y，et al.Differential treatment for stuff and things：a simple unsupervised domain adaptation method for semantic segmentation[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：12632-12641.
[33] PASZKE A，GROSS S，CHINTALA S，et al.Automatic differentiation in PyTorch[C]//NIPS 2017 Workshop，2017.
[34] 田萱，王亮，丁琪.基于深度学习的图像语义分割方法综述[J].软件学报，2019，30（2）：440-468.
TIAN X，WANG L，DING Q.A review of image semantic segmentation methods based on deep learning[J].Journal of Software，2019，30（2）：440-468.