Adaptive Feature Refinement for Oriented Object Detection in Remote Sensing Images

doi:10.3778/j.issn.1002-8331.2207-0077

Abstract

Abstract: Object detection is an important and challenging task in remote sensing research. Remote sensing images are mostly taken from a top-down perspective. Due to their complex background and arbitrary orientation, object detection algorithms in natural scenes face some challenges when directly applied to remote sensing. Aiming at the above problems, this paper proposes an adaptive feature refinement network AFR-Net to generate directed candidate boxes with high matching degree to objects. Firstly, the feature enhancement module is designed to increase the feature representation with discriminative power, so as to improve the ability of capturing spatial details in complex background. Secondly, in order to obtain the directed candidate box adapted to the object direction, an adaptive feature alignment module is proposed to alleviate the spatial misalignment problem between convolution feature and directed objects, and the rotation-invariant feature is obtained. Finally, the rotation sensitive features are obtained by decoupling detection head module and accurate bounding box regression is refined. The proposed network achieves 66.71% and 97.12% accuracy in the publicly available remote sensing object detection datasets DIOR-R and HRSC2016, which are 2.3 and 0.9 percentage points higher than the original algorithm, respectively. At the same time, compared with some mainstream object detection algorithms, the proposed network has certain advantages.

Key words: convolutional neural network, remote sensing image, object detection, feature refinement, feature alignment

摘要： 目标检测是遥感研究中重要且具有挑战性的任务。遥感图像大多通过俯视视角拍摄，其背景复杂、方向任意的特点，使得自然场景中的目标检测算法直接应用于遥感领域会面临一些挑战。针对上述问题，提出一种自适应特征细化网络AFR-Net，生成与物体之间具有高匹配度的有向候选框。设计特征增强模块增加具有判别力的特征表示，提升复杂背景下空间细节的捕捉能力；为得到适应于物体方向的有向候选框，提出自适应特征对齐模块缓解卷积特征与有向目标的空间错位问题，得到旋转不变特征；通过解耦检测头模块获取旋转敏感特征并细化精确的边界盒回归。提出的网络在公开的遥感目标检测数据集DIOR-R和HRSC2016达到了66.71%和97.12%的准确率，相比原始算法分别提高了2.3和0.9个百分点的检测精度，同时与一些主流的目标检测算法相比，该算法具有一定的优越性。

关键词: 卷积神经网络, 遥感图像, 目标检测, 特征细化, 特征对齐

LIU Enhai, XU Jiayin, LI Yan, FAN Shiyan. Adaptive Feature Refinement for Oriented Object Detection in Remote Sensing Images[J]. Computer Engineering and Applications, 2023, 59(24): 155-164.

刘恩海, 许佳音, 李妍, 樊世燕. 自适应特征细化的遥感图像有向目标检测[J]. 计算机工程与应用, 2023, 59(24): 155-164.

References

[1] CHEN C，GONG W，CHEN Y，et al.Object detection in remote sensing images based on a scene-contextual feature pyramid network[J].Remote Sensing，2019，11（3）：339.
[2] CUI Z，LENG J，LIU Y，et al.SKNet：detecting rotated ships as keypoints in optical remote sensing images[J].IEEE Transactions on Geoscience and Remote Sensing，2021，59（10）：8826-8840.
[3] CAO D，ZHU X，HUANG X，et al.Domain balancing：face recognition on long-tailed domains[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：5671-5679.
[4] 秦伟伟，宋泰年，刘洁瑜，等.基于轻量化YOLOv3的遥感军事目标检测算法[J].计算机工程与应用，2021，57（21）：263-269.
QIN W W，SONG T N，LIU J Y，et al.Remote sensing military target detection algorithm based on lightweight YOLOv3[J].Computer Engineering and Applications，2021，57（21）：263-269.
[5] 聂光涛，黄华.光学遥感图像目标检测算法综述[J].自动化学报，2021，47（8）：1749-1768.
NIE G T，HUANG H.A survey of object detection in optical remote sensing images[J].Acta Automatica Sinica，2021，47（8）：1749-1768.
[6] LIU Z，HU J，WENG L，et al.Rotated region based CNN for ship detection[C]//2017 IEEE International Conference on Image Processing（ICIP），2017：900-904.
[7] 徐昌贵，张波，高建威，等.FCOSR：一种无锚框的SAR图像任意朝向船舶目标检测网络[J].雷达学报，2022，11（3）：335-346.
XU C G，ZHANG B，GAO J B，et al.FCOSR：an anchor-free method for arbitrary-oriented ship detection in SAR images[J].Journal of Radars，2022，11（3）：335-346.
[8] 陈少波，雷泽人.基于GA-RoI Transformer的遥感图像任意方向目标检测[J].中南民族大学学报，2022，41（3）：312-318.
CHEN S B，LEI Z R.Oriented object detection in remote sensing image based on GA-RoI Transformer[J].Journal of South-Central Minzu University，2022，41（3）：312-318.
[9] ZHANG J，HUANG R，LI Y，et al.Oriented ship detection based on intersecting circle and deformable RoI in remote sensing images[J].Remote Sensing，2022，14（19）：4749.
[10] LIU Z，WANG H，WENG L，et al.Ship rotated bounding box space for ship extraction from high-resolution optical satellite images with complex backgrounds[J].IEEE Geoscience and Remote Sensing Letters，2016，13（8）：1074-1078.
[11] XIAO Z，QIAN L，SHAO W，et al.Axis learning for orientated objects detection in aerial images[J].Remote Sensing，2020，12（6）：908.
[12] LIU L，PAN Z，LEI B.Learning a rotation invariant detector with rotatable bounding box[J].arXiv：1711.09405，2017.
[13] JIANG Y，ZHU X，WANG X，et al.R2CNN：rotational region CNN for orientation robust scene text detection[J].arXiv：1706.09579，2017.
[14] 王明阳，王江涛，刘琛.基于关键点的遥感图像旋转目标检测[J].电子测量与仪器学报，2021，35（6）：102-108.
WANG M Y，WANG J T，LIU C.Remote sensing image rotation object detection based on key points[J].Journal of Electronic Measurement and Instrument，2021，35（6）：102-108.
[15] YANG X，YAN J.Arbitrary-oriented object detection with circular smooth label[C]//European Conference on Computer Vision.Cham：Springer，2020：677-694.
[16] YANG X，HOU L，ZHOU Y，et al.Dense label encoding for boundary discontinuity free rotation detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：15819-15829.
[17] MA J，SHAO W，YE H，et al.Arbitrary-oriented scene text detection via rotation proposals[J].IEEE Transactions on Multimedia，2018，20（11）：3111-3122.
[18] DING J，XUE N，LONG Y，et al.Learning RoI transformer for oriented object detection in aerial images[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：2849-2858.
[19] 朱煜，方观寿，郑兵兵，等.基于旋转框精细定位的遥感目标检测方法研究[J].自动化学报，2023，49（2）：415-424.
ZHU Y，FANG G S，ZHENG B B，et al.Research on detection method of refined rotated boxes in remote sensing[J].Acta Automatica Sinica，2023，49（2）：415-424.
[20] YANG X，YANG J，YAN J，et al.Scrdet：towards more robust detection for small，cluttered and rotated objects[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：8232-8241.
[21] ZHANG G，LU S，ZHANG W.CAD-Net：a context-aware detection network for objects in remote sensing imagery[J].IEEE Transactions on Geoscience and Remote Sensing，2019，57（12）：10015-10024.
[22] GUO Z，ZHANG X，LIU C，et al.Convex-hull feature adaptation for oriented and densely packed object detection[J].IEEE Transactions on Circuits and Systems for Video Technology，2022，32（8）：5252-5265.
[23] 刘万涛，聂云峰，陈兴峰，等.复杂背景下深度学习方法的夜光船舶目标检测[J].航天返回与遥感，2022，43（3）：124-137.
LIU W T，NIE Y F，CHEN X F，et al.Deep learning method in complex scenes luminous ship target detection[J].Spacecraft Recovery & Remote Sensing，2022，43（3）：124-137.
[24] PAN X，REN Y，SHENG K，et al.Dynamic refinement network for oriented and densely packed object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：11207-11216.
[25] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[26] LI J，FANG F，MEI K，et al.Multi-scale residual network for image super-resolution[C]//Proceedings of the European Conference on Computer Vision，2018：517-532.
[27] LI J Q，LI J C，FANG F M，et al.Luminance-aware pyramid network for low-light image enhancement[J].IEEE Transactions on Multimedia，2020，23：3153-3165.
[28] DING X，GUO Y，DING G，et al.Acnet：strengthening the kernel skeletons for powerful cnn via asymmetric convolution blocks[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：1911-1920.
[29] DAI J，QI H，XIONG Y，et al.Deformable convolutional networks[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：764-773.
[30] WU Y，CHEN Y，YUAN L，et al.Rethinking classification and localization for object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：10186-10195.
[31] LIU Z，YUAN L，WENG L，et al.A high resolution optical satellite image dataset for ship recognition and some new baselines[C]//International Conference on Pattern Recognition Applications and Methods，2017：324-331.
[32] LI K，WAN G，CHENG G，et al.Object detection in optical remote sensing images：a survey and a new benchmark[J].ISPRS Journal of Photogrammetry and Remote Sensing，2020，159：296-307.
[33] RUSSAKOVSKY O，DENG J，SU H，et al.Imagenet large scale visual recognition challenge[J].International Journal of Computer Vision，2015，115（3）：211-252.
[34] GIRSHICK R.Fast R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision，2015：1440-1448.
[35] LIN T Y，GOYAL P，GIRSHICK R，et al.Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：2980-2988.
[36] XIE X，CHENG G，WANG J，et al.Oriented R-CNN for object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2021：3520-3529.
[37] TIAN Z，SHEN C，CHEN H，et al.Fcos：fully convolutional one-stage object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：9627-9636.
[38] HAN J，DING J，LI J，et al.Align deep features for oriented object detection[J].IEEE Transactions on Geoscience and Remote Sensing，2021，60：1-11.
[39] CHENG G，WANG J，LI K，et al.Anchor-free oriented proposal generator for object detection[J].IEEE Transactions on Geoscience and Remote Sensing，2022，60：5625411.
[40] XU Y，FU M，WANG Q，et al.Gliding vertex on the horizontal bounding box for multi-oriented object detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2020，43（4）：1452-1459.