聚合全局重构语义的航空遥感多目标分割模型

doi:10.3778/j.issn.1002-8331.2408-0077

摘要/Abstract

摘要： 为了解决航空遥感图像存在目标尺度多且语义信息不足和特征边界不清晰等问题，设计了一种聚合全局信息再对特征分类后重构语义的分割模型。将Swin-Transformer作为编码结构，利用其对上下文信息的理解来提取特征，再通过设计的深浅语义重构模块和通道残差重构模块将提取到的特征按信息量进行分类后重构，最后通过设计的区域上采样及下采样连接，将重构后的特征与编码器提取的特征融合成全面的特征聚合块后进行输出。对多目标下重构目标特征做到精细化并生成对应的分割图，以此提高分割精度，实现了高质量的逐像素回归。在ISPRS Vaihingen和ISPRS Potsdam两个数据集上的平均交并比（mean intersection over union，mIoU）分数达到了87.2%和82.9%，整体精准度（overall accuracy，OA）分数达到了91.4%和91.2%。

关键词: 航空遥感图像, 语义分割, 深浅语义重构卷积组, 通道残差重构卷积组, 区域上采样, 特征融合模块

Abstract: To address the challenges of multiple target scales, insufficient semantic information, and blurred feature boundaries in aerial remote sensing images, a segmentation model that aggregates global information and reconstructs semantic representations after feature classification is proposed. Swin-Transformer is employed as the encoder to capture contextual information and extract deep features. A designed deep shallow semantic reconstruction module and a channel residual reconstruction module classify and reconstruct these features based on their information contens. Subsequently, a regional upsampling and downsampling connection strategy is introduced to fuse the reconstructed features with the encoder features into a comprehensive feature aggregation block for final output. This approach enables the fine-grained reconstruction of multi-target features and the generation of accurate segmentation maps, thereby enhancing segmentation precision and achieving high-quality pixel-wise regression. Experimental results show that the model achieves mean intersection over union (mIoU) scores of 87.2% and 82.9%, and overall accuracy (OA) scores of 91.4% and 91.2% on the ISPRS Vaihingen and ISPRS Potsdam datasets, respectively.

Key words: aerial remote sensing image, semantic segmentation, depth and shallow semantic reconstruction convolution group, channel residual reconstruction convolution group, region upsampling, feature fusion module

吴小所, 乔煜栋, 贺成龙, 刘小明, 闫浩文. 聚合全局重构语义的航空遥感多目标分割模型[J]. 计算机工程与应用, 2025, 61(22): 215-225.

WU Xiaosuo, QIAO Yudong, HE Chenglong, LIU Xiaoming, YAN Haowen. Multi-Target Segmentation Model for Aerial Remote Sensing Based on Global Reconstruction Semantics Aggregation[J]. Computer Engineering and Applications, 2025, 61(22): 215-225.

参考文献

[1] 王曦, 于鸣, 任洪娥. UNET与FPN相结合的遥感图像语义分割[J]. 液晶与显示, 2021, 36(3): 475-483.
WANG X, YU M, REN H E. Remote sensing image semantic segmentation combining UNET and FPN[J]. Chinese Journal of Liquid Crystals and Displays, 2021, 36(3): 475-483.
[2] 胡宇翔, 余长宏, 高明. 多模态融合的遥感图像语义分割网络[J]. 计算机工程与应用, 2024, 60(15): 234-242.
HU Y X, YU C H, GAO M. Remote sensing image semantic segmentation network based on multimodal fusion[J]. Computer Engineering and Applications, 2024, 60(15): 234-242.
[3] MA J, ZHANG Y, LI X. Deep learning for hyperspectral image analysis: a review[J]. IEEE Geoscience and Remote Sensing Magazine, 2020, 8(1): 55-76.
[4] SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651.
[5] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834-848.
[6] CHEN L C, ZHU Y K, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the 15th European Conference on Computer Vision. Cham: Springer, 2018: 833-851.
[7] FU J, LIU J, TIAN H J, et al. Dual attention network for scene segmentation[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 3141-3149.
[8] YU C Q, WANG J B, PENG C, et al. BiSeNet: bilateral segmentation network for real-time semantic segmentation[C]//Proceedings of the 15th European Conference on Computer Vision. Cham: Springer, 2018: 334-349.
[9] LIU Z, LIN Y T, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2022: 9992-10002.
[10] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the 15th European Conference on Computer Vision. Cham: Springer, 2018: 3-19.
[11] HUANG Z L, WANG X G, WEI Y C, et al. CCNet: criss-cross attention for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(6): 6896-6908.
[12] WU Y X, HE K M. Group normalization[C]//Proceedings of the 15th European Conference on Computer Vision. Cham: Springer, 2018: 3-19.
[13] CHOLLET F. Xception: deep learning with depthwise separable convolutions[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 1800-1807.
[14] CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[J]. arXiv: 1706.05587, 2017.
[15] WANG L B, LI R, WANG D Z, et al. Transformer meets convolution: a bilateral awareness network for semantic segmentation of very fine resolution urban scene images[J]. Remote Sensing, 2021, 13(16): 3065.
[16] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
[17] JADON S. A survey of loss functions for semantic segmentation[C]//Proceedings of the 2020 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology. Piscataway: IEEE, 2020: 1-7.
[18] HUA B S, TRAN M K, YEUNG S K. Pointwise convolutional neural networks[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 984-993.
[19] LI R, ZHENG S Y, ZHANG C, et al. ABCNet: attentive bilateral contextual network for efficient semantic segmentation of fine-resolution remotely sensed imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 181: 84-98.
[20] WANG G, LI W, ZOU L, et al. MaNet: a robust and efficient convolutional neural network for medical image analysis[J]. Computers in Biology and Medicine, 2018, 102: 52-67.
[21] WANG L B, LI R, ZHANG C, et al. UNetFormer: a UNet-like transformer for efficient semantic segmentation of remote sensing urban scene imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 190: 196-214.
[22] LI R, ZHENG S Y, DUAN C X, et al. Multistage attention ResU-net for semantic segmentation of fine-resolution remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 8009205.
[23] STRUDEL R, GARCIA R, LAPTEV I, et al. Segmenter: transformer for semantic segmentation[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2022: 7242-7252.
[24] DIAKOGIANNIS F I, FURBY S, CACCETTA P, et al. SSG2: a new modelling paradigm for semantic segmentation[J]. arXiv:2310.08671, 2023.
[25] LUO Z, YANG W, YUAN Y, et al. Semantic segmentation of agricultural images: a survey[J]. Information Processing in Agriculture, 2024, 11(2): 172-186.
[26] LI L, ZHOU T, WANG W, et al. Deep hierarchical semantic segmentation[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 1246-1257.
[27] GU J, KWON H, WANG D, et al. Multi-scale high-resolution vision transformer for semantic segmentation[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 12094-12103.
[28] 李俊杰, 易诗, 何润华, 等. 基于窗口注意力聚合Swin Transformer的无人机影像语义分割方法[J]. 计算机工程与应用, 2024, 60(15): 198-210.
LI J J, YI S, HE R H, et al. Semantic segmentation method of UAV image based on window attention aggregation swin transformer[J]. Computer Engineering and Applications, 2024, 60(15): 198-210.