Improved Feature Fusion Network for Small Object Detection in Remote Sensing Images

doi:10.3778/j.issn.1002-8331.2205-0058

Abstract

Abstract: Small scale and dense arrangement of objects in remote sensing images present significant challenges for feature extraction and object detection. To solve the challenges above, a network based on feature fusion and attention mechanism is proposed. Firstly, to enhance detailed features which are useful for small scale object, a scale attention module is proposed to filter invalid semantic features and generate attention masks with different scales after the backbone network. Moreover, an edge refinement module is introduced to reduce false detection by suppressing feature misalignment in the dense area. Experimental results show that, compared with the baseline model Faster R-CNN, the method improves the detection accuracy AP50 and APS by 10 and 11.1 percentage points, respectively.

Key words: remote sensing image, small object detection, attention mechanism, feature fusion

摘要： 遥感图像中目标尺寸小且密集排布的特点给特征提取和目标检测带来了困难。针对上述问题，提出一种基于注意力机制的特征融合网络ABFN（attention based feature fusion network）。通过尺度注意力模块在骨干网络提取的特征图上生成不同尺度的注意力Mask过滤无效的语义信息，以此增强小目标的细节信息；引入边缘细化模块抑制目标密集区域的特征错位，以此降低目标的误检率。实验结果表明，相比于基准模型Faster R-CNN，该方法在遥感数据集xView上的检测精度AP50和APS分别提高了10和11.1个百分点。

关键词: 遥感图像, 小目标检测, 注意力机制, 特征融合

LI Chao, WANG Kai, DING Caichang, ZHANG Jinyue, LI Jiabao. Improved Feature Fusion Network for Small Object Detection in Remote Sensing Images[J]. Computer Engineering and Applications, 2023, 59(17): 232-241.

李超, 王凯, 丁才昌, 张金玥, 李贾宝. 改进特征融合网络的遥感图像小目标检测[J]. 计算机工程与应用, 2023, 59(17): 232-241.

References

[1] 高新波，莫梦竟成，汪海涛，等.小目标检测研究进展[J].数据采集与处理，2021，36（3）：391-417.
GAO X B，MO M J C，WANG H T，et al.Recent advances in small object detection[J].Journal of Data Acquisition and Processing，2021，36（3）：391-417.
[2] GIRSHICK R，DONAHUE J，DARRELL T，et al.Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2014：580-587.
[3] HE K，ZHANG X，REN S，et al.Spatial pyramid pooling in deep convolutional networks for visual recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，37（9）：1904-1916.
[4] GIRSHICK R.Fast R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision，2015：1440-1448.
[5] REN S，HE K，GIRSHICK R，et al.Faster R-CNN：towards real-time object detection with region proposal networks[C]//Advances in Neural Information Processing Systems，2015.
[6] EVERINGHAM M，ESLAMI S M，VAN GOOL L，et al.The pascal visual object classes challenge：a retrospective[J].International Journal of Computer Vision，2015，111（1）：98-136.
[7] LIN T Y，MAIRE M，BELONGIE S，et al.Microsoft COCO：common objects in context[C]//European Conference on Computer Vision.Cham：Springer，2014：740-755.
[8] LIN T Y，DOLLáR P，GIRSHICK R，et al.Feature pyramid networks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：2117-2125.
[9] YU X，GONG Y，JIANG N，et al.Scale match for tiny person detection[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision，2020：1257-1265.
[10] ZOPH B，CUBUK E D，GHIASI G，et al.Learning data augmentation strategies for object detection[C]//European Conference on Computer Vision.Cham：Springer，2020：566-583.
[11] CHEN Z M，JIN X，ZHAO B，et al.Hierarchical context embedding for region-based object detection[C]//European Conference on Computer Vision.Cham：Springer，2020：633-648.
[12] HONG M，LI S，YANG Y，et al.SSPNet：scale selection pyramid network for tiny person detection from UAV images[J].IEEE Geoscience and Remote Sensing Letters，2021，19：1-5.
[13] NAYAN A A，SAHA J，MOZUMDER A N，et al.Real time multi-class object detection and recognition using vision augmentation algorithm[J].arXiv：2003.07442，2020.
[14] DENG C，WANG M，LIU L，et al.Extended feature pyramid network for small object detection[J].IEEE Transactions on Multimedia，2021，24：1968-1979.
[15] 郭璠，张泳祥，唐琎，等.YOLOv3-A：基于注意力机制的交通标志检测网络[J].通信学报，2021，42（1）：87-99.
GUO F，ZHANG Y X，TANG J，et al.YOLOv3-A：a traffic sign detection network based on attention mechanism[J].Journal on Communications，2021，42（1）：87-99.
[16] WANG X，GIRSHICK R，GUPTA A，et al.Non-local neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：7794-7803.
[17] HUANG Z，WANG X，HUANG L，et al.CCNet：criss-cross attention for semantic segmentation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：603-612.
[18] DOSOVITSKIY A，BEYER L，KOLESNIKOV A，et al.An image is worth 16x16 words：transformers for image recognition at scale[J].arXiv：2010.11929，2020.
[19] 徐辉，祝玉华，甄彤，等.深度神经网络图像语义分割方法综述[J].计算机科学与探索，2021，15（1）：47-59.
XU H，ZHU Y H，ZHEN T，et al.Survey of image semantic segmentation methods based on deep neural network[J].Journal of Frontiers of Computer Science and Technology，2021，15（1）：47-59
[20] YAN M，WANG J，LI J，et al.Traffic scene semantic segmentation using self-attention mechanism and bi-directional GRU to correlate context[J].Neurocomputing，2019，386：293-304.
[21] 马宇，张丽果，杜慧敏，等.卷积神经网络的交通标志语义分割[J].计算机科学与探索，2021，15（6）：1114-1121.
MA Y，ZHANG L G，DU H M，et al.Traffic sign semantic segmentation based on convolutional neural network[J].Journal of Frontiers of Computer Science and Technology，2021，15（6）：1114-1121.
[22] CHEN L C，ZHU Y，PAPANDREOU G，et al.Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：801-818.
[23] PINHEIRO P O，COLLOBERT R.From image-level to pixel-level labeling with convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2015：1713-1721.
[24] 张颖，李小霞，李永龙，等.结合密集注意力和并行上采样的遥感图像道路分割[J].小型微型计算机系统，2021，42（11）：2356-2361.
ZHANG Y，LI X X，LI Y L，et al.Remote sensing image road segmentation combining intensive attention and parallel upsampling[J].Journal of Chinese Computer Systems，2021，42（11）：2356-2361.
[25] LIU Z，LIN Y，CAO Y，et al.Swin transformer：hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2021：10012-10022.
[26] WOO S，PARK J，LEE J Y，et al.CBAM：convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：3-19.
[27] CHEN K，WANG J，PANG J，et al.MMDetection：open MMLab detection toolbox and benchmark[J].arXiv：1906.07155，2019.
[28] CAI Z，VASCONCELOS N.Cascade R-CNN：delving into high quality object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：6154-6162.
[29] ZHANG H，CHANG H，MA B，et al.Dynamic R-CNN：towards high quality object detection via dynamic training[C]//European Conference on Computer Vision.Cham：Springer，2020：260-275.