双分支特征融合的遥感建筑物检测模型

doi:10.3778/j.issn.1002-8331.2307-0270

摘要/Abstract

摘要： 针对遥感建筑物图像中建筑物大小不一、边缘模糊导致精度不高的问题，提出一种双分支并行融合注意力机制的网络模型TC-UNet++。针对卷积神经网络擅长提取局部特征，难以捕获全局信息的特点，引入Transformer结构以解决全局信息丢失的问题。对于两种结构的特征维度和通道数不匹配的问题，设计一种TC（Transformer to CNN）模块以交互的方式融合不同分辨率下局部与全局特征。引入坐标注意力机制，根据像素在图像中的位置信息，定位和识别建筑物。实验结果表明，TC-UNet++在WHU数据集上交互比、准确率、总精度分别达到了93.1%、95.9%、98.8%，在不显著增加参数的情况下，展现出良好的有效性。

关键词: TC-UNet++, 遥感建筑物图像, 双分支, 坐标注意力机制, 特征融合

Abstract: In order to solve the problem of low accuracy caused by different building sizes and fuzzy edges in remote sensing building images, a dual-branch parallel fusion attention mechanism network model TC-UNet++ is proposed. Firstly, considering that convolutional neural networks are good at extracting local features and difficult to capture global information, Transformer structure is introduced to solve the problem of global information loss. Secondly, to solve the problem of mismatch between the feature dimension and channel number of the two structures, a TC (Transformer to CNN) module is designed to interactively integrate local and global features at different resolutions. Finally, the coordinate attention mechanism is introduced to locate and identify buildings according to the position information of pixels in the image. Experimental results show that the interaction ratio, accuracy and total accuracy of TC-UNet++ on WHU dataset reach 93.1%, 95.9% and 98.8% respectively, showing good effectiveness without significantly increasing parameters.

Key words: TC-UNet++, remote sensing building images, dual-branch, coordinate attention mechanism, feature fusion

成嘉伟, 郭荣佐, 吴建成, 张浩. 双分支特征融合的遥感建筑物检测模型[J]. 计算机工程与应用, 2024, 60(22): 145-153.

CHENG Jiawei, GUO Rongzuo, WU Jiancheng, ZHANG Hao. Dual-Branch Feature Fusion Remote Sensing Building Detection Model[J]. Computer Engineering and Applications, 2024, 60(22): 145-153.

参考文献

[1] 聂光涛, 黄华. 光学遥感图像目标检测算法综述[J]. 自动化学报, 2021, 47(8): 1749-1768.
NIE G T, HUANG H. A survey of object detection in optical remote sensing images[J]. Acta Automatica Sinica, 2021, 47(8): 1749-1768.
[2] 王燕, 吕艳萍. 混合深度CNN联合注意力的高光谱图像分类[J]. 计算机科学与探索, 2023, 17(2): 385-395.
WANG Y, LYU Y P. Hybrid deep CNN-attention for hyperspectral image classification[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(2): 385-395.
[3] DONG H, YU B, WU W, et al. Enhanced lightweight end-to-end semantic segmentation for high-resolution remote sensing images[J]. IEEE Access, 2022, 10: 70947-70954.
[4] MA J, WU L, TANG X, et al. Building extraction of aerial images by a global and multi-scale encoder-decoder network[J]. Remote Sensing, 2020, 12(15): 2350.
[5] 叶应辉. 基于深度学习的卫星遥感图像边缘检测方法[J]. 计算机测量与控制, 2022, 30(10): 39-44.
YE Y H. Edge detection method for satellite remote sensing images based on deep learning[J]. Computer Measurement and Control, 2022, 30(10): 39-44.
[6] CHENG B, LI Z, XU B, et al. Target detection in remote sensing image based on object-and-scene context constrained CNN[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 1-5.
[7] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7132-7141.
[8] HU J, SHEN L, ALBANIE S, et al. Gather-excite: exploiting feature context in convolutional neural networks[C]//Advances in Neural Information Processing Systems 31, 2018.
[9] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the 15th European Conference on Computer Vision, 2018: 3-19.
[10] HU H, GU J, ZHANG Z, et al. Relation networks for object detection[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, 2018: 3588-3597.
[11] ZHOU Z, RAHMAN SIDDIQUEE M M, TAJBAKHSH N, et al. UNet++: a nested U-Net architecture for medical image segmentation[M]//Deep learning in medical image analysis and multimodal learning for clinical decision support. Cham: Springer, 2018: 3-11.
[12] 任鸿杰, 刘萍, 岱超, 等. 改进DeepLabV3+网络的遥感影像农作物分割方法[J]. 计算机工程与应用, 2022, 58(11): 215-223.
REN H J, LIU P, DAI C, et al. Crop segmentation method of remote sensing image based on improved DeepLabV3+ network[J]. Computer Engineering and Applications, 2022, 58(11): 215-223.
[13] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, 2015: 3431-3440.
[14] PENG D, ZHANG Y, GUAN H. End-to-end change detection for high resolution satellite images using improved UNet++[J]. Remote Sensing, 2019, 11(11): 1382.
[15] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems 30, 2017.
[16] BEAL J, KIM E, TZENG E, et al. Toward transformer-based object detection[J]. arXiv:2012.09958, 2020.
[17] 岱超, 刘萍, 史俊才, 等. 利用U型网络的遥感影像建筑物规则化提取[J]. 计算机工程与应用, 2023, 59(8): 105-116.
DAI C, LIU P, SHI J C, et al. Regularized extraction of remotely sensed image buildings using U-shaped networks[J]. Computer Engineering and Applications, 2023, 59(8): 105-116.
[18] RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C]//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2015: 234-241.
[19] 贾天豪, 彭力, 戴菲菲. 引入残差学习与多尺度特征增强的目标检测器[J]. 计算机科学与探索, 2023, 17(5): 1102-1111.
JIA T H, PENG L, DAI F F. Object detector with residual learning and multi-scale feature enhancement[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(5): 1102-1111.
[20] 蔡肖, 陈志华, 盛斌. 基于移位窗口金字塔Transformer的遥感图像目标检测[J]. 计算机科学, 2023, 50(1): 105-113.
CAI X, CEHN Z H, SHENG B. ?SPT: swin pyramid transformer for object detection of remote sensing[J]. Computer Science, 2023, 50(1): 105-113.
[21] ZHENG S, LU J, ZHAO H, et al. Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 6881-6890.
[22] CHEN H, WANG Y, GUO T, et al. Pre-trained image processing transformer[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 12299-12310.
[23] JIANG Y, CHANG S, WANG Z. TransGAN: two transformers can make one strong GAN[J]. arXiv:2102.07074, 2021.
[24] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: transformers for image recognition at scale[J]. arXiv:2010.11929, 2020.
[25] HOU Q, ZHOU D, FENG J. Coordinate attention for efficient mobile network design[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 13713-13722.
[26] 刘剑峰, 潘晨. 增强特征金字塔结构的显著目标检测算法[J]. 计算机工程与应用, 2022, 58(12): 226-233.
LIU J F, PAN C. Salient object detection for enhanced feature pyramid structure[J]. Computer Engineering and Applications, 2022, 58(12): 226-233.
[27] ISLAM M A, KOWAL M, JIA S, et al. Position, padding and predictions: a deeper look at position information in CNNs[J]. arXiv:2101.12322, 2021.
[28] BA J L, KIROS J R, HINTON G E. Layer normalization[J]. arXiv:1607.06450, 2016.
[29] IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]//Proceedings of the 32nd International Conference on Machine Learning, 2015: 448-456.
[30] CAO H, WANG Y, CHEN J, et al. Swin-Unet: Unet-like pure transformer for medical image segmentation[J]. arXiv:2105.
05537, 2021.
[31] 季顺平, 魏世清. 遥感影像建筑物提取的卷积神经元网络与开源数据集方法[J]. 测绘学报, 2019, 48(4): 448-459.
JI S P, WEI S Q. Building extraction via convolutional neural networks from an open remote sensing building dataset[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(4): 448-459.
[32] WU G, SHAO X, GUO Z, et al. Automatic building segmentation of aerial imagery using multi-constraint fully convolutional networks[J]. Remote Sensing, 2018, 10(3): 407.
[33] 吕少云, 李佳田, 阿晓荟, 等. Res_ASPP_UNet++: 结合分离卷积与空洞金字塔的遥感影像建筑物提取网络[J]. 遥感学报, 2023, 27(2): 502-519.
LYU S Y, LI J T, A X H, et al. Res_ASPP_UNet++: building an extraction network from remote sensing imagery combining depthwise separable convolution with atrous spatial pyramid pooling[J]. National Remote Sensing Bulletin, 2023, 27(2): 502-519.