多尺度特征优化的实时Transformer在无人机航拍中的研究

doi:10.3778/j.issn.1002-8331.2409-0049

摘要/Abstract

摘要： 针对无人机目标检测场景中的目标尺度小、遮挡严重、样本分布不均匀等问题，提出了一种改进实时检测Transformer（real-time detection Transformer，RT-DETR）的MSM-DETR检测器。在颈部网络中设计并引入了DSSF特征融合结构，通过结合维度感知选择性整合（DASI）模块以及尺度序列特征融合（SSFF）模块丰富特征融合阶段小目标信息，提高检测精度。针对遮挡严重和样本分布不均匀的问题，提出多核并行尺度间融合（multi-core parallel scale fusion，MCPSF）模块，通过利用尺度间融合思想改进多核分组卷积带来的尺度间信息不平衡问题，为模型提供多尺度感受野，同时利用EMA注意力进一步增强组内上下文信息，提升检测精度。将Inner思想融入原损失函数中，通过引入不同尺度的辅助边框计算损失，加速收敛。实验结果表明，改进后的模型在VisDrone2019数据集中的验证集和测试集的mAP为49.5%、38.9%，较原模型分别提升2.5、2.4个百分点。

关键词: RT-DETR, 航拍图像, 多尺度, 感受野

Abstract: In response to the challenges of small target size, severe occlusion, and uneven sample distribution in UAV target detection scenarios, this paper proposes an enhanced version of the real-time detection Transformer (RT-DETR), termed MSM-DETR. The DSSF feature fusion architecture is designed and incorporated into the neck network, integrating the dimension-aware selective integration (DASI) module and the scale sequence feature fusion (SSFF) module to enrich small target information during the feature fusion stage, thereby enhancing detection accuracy. In addressing the issues of severe occlusion and sample imbalance, the multi-core parallel scale fusion (MCPSF) module is introduced. This module addresses the imbalance in scale-level information caused by multi-core grouped convolutions through scale fusion, allowing the model to capture multi-scale receptive fields. Furthermore, the integration of EMA attention strengthens intra-group contextual information, resulting in further improvements in detection performance. The inner concept is incorporated into the original loss function, facilitating the computation of auxiliary bounding box losses at different scales, thus accelerating convergence. Experimental results demonstrate that the proposed model achieves mAP scores of 49.5% on the validation set and 38.9% on the test set of the VisDrone2019 dataset, representing improvements of 2.5 and 2.4 percentage points, respectively, over the original model.

Key words: real-time detection Transformer (RT-DETR), aerial images, multi-scale, receptive field

向毅伟, 蒋瑜, 王琪凯, 罗熔熔. 多尺度特征优化的实时Transformer在无人机航拍中的研究[J]. 计算机工程与应用, 2025, 61(9): 221-229.

XIANG Yiwei, JIANG Yu, WANG Qikai, LUO Rongrong. Research on Real-Time Transformer for Multi-Scale Feature Optimization in Drone Aerial Imaging[J]. Computer Engineering and Applications, 2025, 61(9): 221-229.

参考文献

[1] 吴靖,韩禄欣,沈英,等. 基于改进YOLOv4-tiny的无人机航拍目标检测[J]. 电光与控制, 2022, 29(12):112-117.
WU J, HAN L X, SHEN Y, et al. UAV aerial target detection based on improved YOLOv4-tiny[J]. Electronics Optics & Control, 2022, 29(12): 112-117.
[2] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems, 2012.
[3] GIRSHICK R. Fast R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision, 2015: 1440-1448.
[4] REN S, HE K, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[C]//Advances in Neural Information Processing Systems, 2015.
[5] REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 779-788.
[6] LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector[C]//Proceedings of the 14th European Conference on Computer Vision(ECCV 2016), Amsterdam, The Netherlands, October 11-14, 2016: 21-37.
[7] SAHIN O, OZER S. YOLODrone: improved YOLO architecture for object detection in drone images[C]//Proceedings of the 2021 44th International Conference on Telecommunications and Signal Processing (TSP), 2021: 361-365.
[8] TAN L, LV X, LIAN X, et al. YOLOv4_Drone: UAV image target detection based on an improved YOLOv4 algorithm[J]. Computers & Electrical Engineering, 2021, 93: 107261.
[9] 李成豪,张静,胡莉,等. 基于多尺度感受野融合的小目标检测算法[J]. 计算机工程与应用, 2022, 58(12): 177-182.
LI C H, ZHANG J, HU L, et al. Small object detection algorithm based on multiscale receptive field fusion[J]. Computer Engineering and Applications, 2022, 58(12): 177-182.
[10] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems, 2017.
[11] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: Transformers for image recognition at scale[J]. arXiv:2010.11929, 2020.
[12] CARION N, MASSA F, SYNNAEVE G, et al. End-to-end object detection with transformers[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer International Publishing, 2020: 213-229.
[13] ZHU X, SU W, LU L, et al. Deformable DETR: deformable transformers for end-to-end object detection[J]. arXiv:2010. 04159, 2020.
[14] LI F, ZHANG H, LIU S L, et al. DN-DETR: accelerate detr training by introducing query denoising[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 13619-13627.
[15] CHEN Q, CHEN X, WANG J, et al. Group DETR: fast detr training with group-wise one-to-many assignment[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023: 6633-6642.
[16] ZHANG H, LI F, LIU S, et al. DINO: DETR with improved denoising anchor boxes for end-to-end object detection[J]. arXiv:2203.03605, 2022.
[17] ZHAO Y, LV W, XU S, et al. Detrs beat YOLOs on real-time object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024: 16965-16974.
[18] 曹紫绚,刘刚,张文波,等. 改进回归损失的深度学习单阶段红外飞机检测[J]. 电光与控制, 2023, 30(4): 28-33.
CAO Z X, LIU G, ZHANG W B, et al. Deep learning single-stage infrared aircraft detection based on improved regression loss[J]. Electronics Optics & Control, 2023, 30(4): 28-33.
[19] ZHENG Z, WANG P, LIU W, et al. Distance-IoU loss: faster and better learning for bounding box regression[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2020: 12993-13000.
[20] XU S, ZHENG S C, XU W, et al. HCF-Net: hierarchical context fusion network for infrared small object detection[J]. arXiv:2403.10778, 2024.
[21] KANG M, TING C M, TING F F, et al. ASF-YOLO: a novel YOLO model with attentional scale sequence fusion for cell instance segmentation[J]. Image and Vision Computing, 2024, 147: 105057.
[22] OUYANG D, HE S, ZHANG G, et al. Efficient multi-scale attention module with cross-spatial learning[C]//Proceedings of the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023: 1-5.
[23] ZHANG H, XU C, ZHANG S. Inner-IoU: more effective intersection over union loss with auxiliary bounding box[J]. arXiv:2311.02877, 2023.
[24] DING X, ZHANG X, HAN J, et al. Scaling up your kernels to 31x31: revisiting large kernel design in CNNs[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 11963-11975.
[25] GUO M H, LU C Z, LIU Z N, et al. Visual attention network[J]. Computational Visual Media, 2023, 9(4): 733-752.
[26] CAI X, LAI Q, WANG Y, et al. Poly kernel inception network for remote sensing detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024: 27706-27716.
[27] 吕伏,傅宇恒,贺丽娜,等. 三维多层次特征协同的无人机遥感目标检测算法[J]. 计算机科学与探索, 2024, 18(5): 1301-1317.
LYU F, FU Y H, HE L N, et al. UAV remote sensing object detection based on 3D multi-layer feature collaboration[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(5): 1301-1317.
[28] GAO S H, CHENG M M, ZHAO K, et al. Res2Net: a new multi-scale backbone architecture[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 43(2): 652-662.
[29] CHOLLET F. Xception: deep learning with depthwise separable convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 1251-1258.
[30] DU D, ZHU P, WEN L, et al. VisDrone-DET2019: the vision meets drone object detection in image challenge results[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, 2019.
[31] ZHANG H, ZHANG S. Shape-IoU: more accurate metric considering bounding box shape and scale[J]. arXiv:2312. 17663, 2023.
[32] LIU Z, LIN Y, CAO Y, et al. Swin Transformer: hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 10012-10022.
[33] TAN M, PANG R, LE Q V. EfficientDet: scalable and efficient object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 10781-10790.
[34] CHEN Y, ZHANG C, CHEN B, et al. Accurate leukocyte detection based on deformable-DETR and multi-level feature fusion for aiding diagnosis of blood diseases[J]. Computers in Biology and Medicine, 2024, 170: 107917.