融合动态权重与语义筛选的双分支遥感图像语义分割

doi:10.3778/j.issn.1002-8331.2409-0070

摘要/Abstract

摘要： 针对遥感图像语义分割中存在的类内特征异质性导致错误分类和大目标分割碎片化的问题，提出一种融合动态权重与语义筛选的双分支遥感图像语义分割网络，所构建网络能够以逐层交互的方式建模Transformer和CNN的全局局部信息。构建基于动态权重调节机制的动态感知融合模块，通过聚合双分支的全局局部建模优势，缓解目标类内特征不一致导致的错误分类问题。提出语义筛选注意力用于交叉特征筛选模块，挖掘重要语义信息的同时过滤无效特征，以增强模型对大目标的连续分割能力。设计浅层空间-语义信息耦合模块，通过双路耦合注意力结构，缩小跳跃连接中深层语义信息和浅层空间细节之间的特征差异。此外，构建山西太原城区高分辨率卫星遥感数据集，并进行了九类地物目标的分割实验。在自建数据集和公开数据集ISPRS上的实验结果表明，所提算法能够在缓解同类型目标错误分类问题的同时增强对大目标的连续分割能力。

关键词: 语义分割, 深度学习, Transformer, 注意力机制, 遥感图像

Abstract: A dual-branch semantic segmentation network for remote sensing imagery is proposed to address the issues of fragmentation in large target segmentation and misclassification caused by intra-class feature heterogeneity. The network merges global and local modeling capabilities of Transformers and CNN through layer-by-layer interaction. Initially, a dynamic perception fusion module based on a dynamic weight adjustment mechanism is constructed. This module aggregates the advantages of global and local modeling from both branches, alleviating misclassification issues caused by inconsistencies in target class features. Subsequently, a semantic filtering attention mechanism is introduced in the cross-feature selection module. This mechanism enhances the model’s capability for continuous segmentation of large targets by mining important semantic information and filtering out ineffective features. Lastly, a shallow space-semantic information coupling module is designed, featuring a dual-path coupling attention structure that reduces the disparity between deep semantic information and shallow spatial details in skip connections. Furthermore, a high-resolution satellite remote sensing dataset of the Taiyuan urban area in Shanxi is constructed, and segmentation experiments on nine types of ground objects are conducted. The proposed algorithm outperforms comparative algorithms in experiments conducted on the custom dataset and the public ISPRS dataset, effectively enhancing continuous segmentation capabilities for large targets while mitigating misclassification issues within the same type of targets.

Key words: semantic segmentation, deep learning, Transformer, attention mechanisms, remote sensing images

房梁, 谢刚, 续欣莹, 曲铁座, 谢新林. 融合动态权重与语义筛选的双分支遥感图像语义分割[J]. 计算机工程与应用, 2025, 61(20): 295-305.

FANG Liang, XIE Gang, XU Xinying, QU Tiezuo, XIE Xinlin. Dual-Branch Network for Remote Sensing Image Semantic Segmentation with Dynamic Weights and Semantic Filtering[J]. Computer Engineering and Applications, 2025, 61(20): 295-305.

参考文献

[1] 吴强强, 王帅, 王彪, 等. 空间信息感知语义分割模型的高分辨率遥感影像道路提取[J]. 遥感学报, 2022, 26(9): 1872-1885.
WU Q Q, WANG S, WANG B, et al. Road extraction method of high-resolution remote sensing image on the basis of the spatial information perception semantic segmentation model[J]. National Remote Sensing Bulletin, 2022, 26(9): 1872-1885.
[2] ZHAO Q, LIU J H, LI Y W, et al. Semantic segmentation with attention mechanism for remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 99: 1-13.
[3] 胡宇翔, 余长宏, 高明. 多模态融合的遥感图像语义分割网络[J]. 计算机工程与应用, 2024, 60(15): 234-242.
HU Y X, YU C H, GAO M. Remote sensing image semantic segmentation network based on multimodal fusion[J]. Computer Engineering and Applications, 2024, 60(15): 234-242.
[4] 林云浩, 王艳军, 李少春, 等. 一种耦合DeepLab与Transformer的农作物种植类型遥感精细分类方法[J]. 测绘学报, 2024, 53(2): 353-366.
LIN Y H, WANG Y J, LI S C, et al. A coupled DeepLab and Transformer approach for fine classification of crop cultivation types in remote sensing[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(2): 353-366.
[5] GAO Y, ZHANG S, ZUO D, et al. TMNet: a two-branch multi-scale semantic segmentation network for remote sensing images[J]. Sensors (Basel), 2023, 23(13): 5909.
[6] 董张玉, 李金徽, 张晋, 等. 边缘增强的BECU-Net模型高分辨率遥感影像耕地提取[J]. 遥感学报, 2023, 27(12): 2847-2859.
DONG Z Y, LI J H, ZHANG J, et al. Cultivated land extra-ction from high-resolution remote sensing images based on BECU-Net model with edge enhancement[J]. National Remote Sensing Bulletin, 2023, 27(12): 2847-2859.
[7] 何佳佳, 徐杨, 张永丹. 改进U-net++的遥感图像语义分割方法[J]. 计算机工程与应用, 2024, 60(13): 255-265.
HE J J, XU Y, ZHANG Y D. Improved U-net++ semantic segmentation method for remote sensing images[J]. Computer Engineering and Applications, 2024, 60(13): 255-265.
[8] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recogn-ition. Piscataway: IEEE, 2015: 3431-3440.
[9] RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proceedings of the Medical Image Computing and Computer-Assisted Intervention. Cham: Springer International Publishing, 2015: 234-241.
[10] BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.
[11] YANG M Y, KUMAAR S, LYU Y, et al. Real-time semantic segmentation with context aggregation network[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 178: 124-134.
[12] ZHAO H S, SHI J P, QI X J, et al. Pyramid scene parsing network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 6230-6239.
[13] DIAKOGIANNIS F I, WALDNER F, CACCETTA P, et al. ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 162: 94-114.
[14] MA A L, WANG J J, ZHONG Y F, et al. FactSeg: foreground activation-driven small object semantic segment-ation in large-scale remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-16.
[15] LI R, ZHENG S Y, ZHANG C, et al. ABCNet: attentive bilateral contextual network for efficient semantic segmentation of Fine-Resolution remotely sensed imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 181: 84-98.
[16] HOU J L, GUO Z, WU Y M, et al. BSNet: dynamic hybrid gradient convolution based boundary-sensitive network for remote sensing image segmentation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-22.
[17] HAN K, WANG Y, CHEN H, et al. A survey on vision transformer[J]. IEEE Transactions on Pattern Analysis Machine Intelligence, 2023, 45(1): 87-110.
[18] STRUDEL R, GARCIA R, LAPTEV I, et al. Segmenter: transformer for semantic segmentation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2021: 7242-7252.
[19] CHEN J, XIA M, WANG D H, et al. Double branch parallel network for segmentation of buildings and waters in remote sensing images[J]. Remote Sensing, 2023, 15(6): 1536.
[20] LIU Z, LIN Y T, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2021: 9992-10002.
[21] ZHANG Q, YANG Y B. Rest: an efficient transformer for visual recognition[J]. Advances in Neural Information Processing Systems, 2021, 34: 15475-15485.
[22] DONG X Y, BAO J M, CHEN D D, et al. CSWin transformer: a general vision transformer backbone with cross-shaped windows[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 12114-12124.
[23] ZHANG G, LEI T, CUI Y, et al. A dual-path and lightweight convolutional neural network for high-resolution aerial image segmentation[J]. ISPRS International Journal of Geo-Information, 2019, 8(12): 582.
[24] JIANG S, LI J J, HUA Z. DPCFN: dual path cross fusion network for medical image segmentation[J]. Engineering Applications of Artificial Intelligence, 2022, 116: 105420.
[25] HE X, ZHOU Y, ZHAO J Q, et al. Swin transformer embedding UNet for remote sensing image semantic segmentation[J]. IEEE Transactions on Geoscience and Remote Sen-sing, 2022, 60: 1-15.
[26] CHEN Y H, LIU P Y, ZHAO J C, et al. Shallow-guided transformer for semantic segmentation of hyperspectral remote sensing imagery[J]. Remote Sensing, 2023, 15(13): 3366.
[27] WENG L G, PANG K, XIA M, et al. Sgformer: a local and global features coupling network for semantic segmentation of land cover[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2023, 16: 6812-6824.
[28] GENG J, SONG S, JIANG W. Dual-path feature aware network for remote sensing image semantic segmentation[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2024, 34(5): 3674-3686.
[29] HE K M, ZHANG X Y, REN S Q, et al. Deep residual lear-ning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770-778.
[30] ROTTENSTEINER F, SOHN G, GERKE M, et al. Results of the ISPRS benchmark on urban object detection and 3D building reconstruction[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2014, 93: 256-271.