基于高阶空间特征提取的无人机航拍小目标检测算法

doi:10.3778/j.issn.1002-8331.2407-0230

摘要/Abstract

摘要： 针对视觉算法在检测航拍图像中密集小目标时容易受到目标重叠、遮挡等情况干扰的现象，提出了一种基于高阶空间特征（目标形状、位置等信息的高级表示）提取的Transformer检测头HSF-TPH（Transformer prediction head with high-order spatial feature extraction）。所提检测头中将自注意力机制中的二阶交互扩展到三阶以生成高阶空间特征，提取更有区分度的空间关系，突出每一个小目标在空间上的语义信息。同时，为了缓解骨干网络过度下采样对小目标信息的压缩，设计了一种高分辨率特征图生成机制，增加头部网络的输入特征分辨率，以提升HSF-TPH检测密集小目标的效果。设计了新的损失函数USIoU，降低算法位置偏差敏感性。在VisDrone2019数据集上开展实验证明，所提算法在面积最小、密度最高的人类目标的检测任务中实现了mAP50指标10个百分点以上的性能提升。

关键词: 无人机航拍, 小目标检测, 高阶空间特征提取, 注意力机制, 损失函数

Abstract: In response to the challenges that vision algorithms are easily interfered by target overlapping and occlusion when detecting dense and small targets in aerial images, a novel Transformer prediction head based on high-order spatial feature (high-level representations of the target’s shape, position, and other information) extraction (HSF-TPH) is proposed. This prediction head enhances the self-attention mechanism by extending the interactions from second-order to third-order, thereby generating higher-order spatial features. This advancement allows for the extraction of more discriminative spatial relationships and emphasizes the spatial semantic information of each small target. Simultaneously, to alleviate the compression of small target information due to excessive down-sampling in the backbone network, a high-resolution feature map generation mechanism is designed. The mechanism increases the input feature resolution of the head network to enhance the effect of HSF-TPH in detecting dense small targets. Finally, a new loss function, USIoU, is designed to reduce the sensitivity of algorithm to positional deviations. Experiments conducted on the VisDrone2019 dataset demonstrate that the proposed algorithm achieves a significant performance improvement of over 10 percentage points in the mAP50 metric in the detection task for human targets, which are characterized by the smallest area and highest density.

Key words: UAV aerial photography, small target detection, high-order spatial feature extraction, attention mechanism, loss function

张轩宇, 周思航, 黄健, 王冬. 基于高阶空间特征提取的无人机航拍小目标检测算法[J]. 计算机工程与应用, 2025, 61(12): 210-221.

ZHANG Xuanyu, ZHOU Sihang, HUANG Jian, WANG Dong. High-Order Spatial Feature Extraction Based Small Target Detection for UAV Aerial Photographs[J]. Computer Engineering and Applications, 2025, 61(12): 210-221.

参考文献

[1] 徐艺博, 于清华, 王炎娟, 等. 基于多源信息融合的巡飞弹对地目标识别与毁伤评估[J]. 系统仿真学报, 2024, 36(2): 511-521.
XU Y B, YU Q H, WANG Y J, et al. Ground target recognition and damage assessment of patrol missiles based on multi-source information fusion[J]. Journal of System Simulation, 2024, 36(2): 511-521.
[2] 朱学岩, 张新伟, 顾梦梦, 等. 基于无人机可见光图像的云杉计数方法[J]. 林业工程学报, 2021, 6(4): 140-146.
ZHU X Y, ZHANG X W, GU M M, et al. Spruce counting method based on UAV visible images[J]. Journal of Forestry Engineering, 2021, 6(4): 140-146.
[3] 江波, 屈若锟, 李彦冬, 等. 基于深度学习的无人机航拍目标检测研究综述[J]. 航空学报, 2021, 42(4): 524519.
JIANG B, QU R K, LI Y D, et al. Object detection in UAV imagery based on deep learning: review[J]. Acta Aeronautica et Astronautica Sinica, 2021, 42(4): 524519.
[4] 张艳, 张明路, 吕晓玲, 等. 深度学习小目标检测算法研究综述[J]. 计算机工程与应用, 2022, 58(15): 1-17.
ZHANG Y, ZHANG M L, LYU X L, et al. Review of research on small target detection based on deep learning[J]. Computer Engineering and Applications, 2022, 58(15): 1-17.
[5] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2014: 580-587.
[6] REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149.
[7] HE K M, ZHANG X Y, REN S Q, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904-1916.
[8] REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 779-788.
[9] WANG C Y, BOCHKOVSKIY A, LIAO H M. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[C]//Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2023: 7464-7475.
[10] LI Z X, YANG L, ZHOU F Q. FSSD: feature fusion single shot multibox detector[J]. arXiv:1712.00960, 2017.
[11] 刘颖, 刘红燕, 范九伦, 等. 基于深度学习的小目标检测研究与应用综述[J]. 电子学报, 2020, 48(3): 590-601.
LIU Y, LIU H Y, FAN J L, et al. A survey of research and application of small object detection based on deep learning[J]. Acta Electronica Sinica, 2020, 48(3): 590-601.
[12] SIMARD P Y, STEINKRAUS D, PLATT J C. Best practices for convolutional neural networks applied to visual document analysis[C]//Proceedings of the Seventh International Conference on Document Analysis and Recognition. Piscataway: IEEE, 2003: 958-963.
[13] WANG S H. An augmentation small object detection method based on NAS-FPN[C]//Proceedings of the 2020 7th International Conference on Information Science and Control Engineering. Piscataway: IEEE, 2020: 213-218.
[14] YAEGER L, LYON R, WEBB B. Effective training of a neural network character classifier for word recognition[C]//Proceedings of the 9th International Conference on Neural Information Processing Systems, 1996.
[15] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
[16] LI W J, TAN X F, WANG Z J. Small object detection of table tennis based on deep learning network[C]//Proceedings of the 2020 International Conference on Computer Science and Management Technology. Piscataway: IEEE, 2020: 149-152.
[17] CORES D, BREA V M, MUCIENTES M, et al. Downsampling GAN for small object data augmentation[C]//Proceedings of the 20th International Conference on Computer Analysis of Images and Patterns. Cham: Springer, 2023: 89-98.
[18] KIM J H, HWANG Y. GAN-based synthetic data augmentation for infrared small target detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5002512.
[19] ZHOU Z Q, QI L, YANG X, et al. Generalizable cross-modality medical image segmentation via style augmentation and dual normalization[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 20824-20833.
[20] SREENIVAS M, BISWAS S. Similar class style augmentation for efficient cross-domain few-shot learning[C]//Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Piscataway: IEEE, 2023: 4590-4598.
[21] PHILIP T J, AMIR A A, STEPHEN B, et al. Style augmentation: data augmentation via style randomization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Piscataway: IEEE, 2019: 83-92.
[22] WANG J D, SUN K, CHENG T H, et al. Deep high-resolution representation learning for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(10): 3349-3364.
[23] LIU Z M, GAO G Y, SUN L, et al. HRDNet: high-resolution detection network for small objects[C]//Proceedings of the 2021 IEEE International Conference on Multimedia and Expo. Piscataway: IEEE, 2021: 1-6.
[24] DENG C F, WANG M M, LIU L, et al. Extended feature pyramid network for small object detection[J]. IEEE Transactions on Multimedia, 2021, 24: 1968-1979.
[25] LIN T Y, DOLLáR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 936-944.
[26] QIAO Z T, SHI D X, YI X D, et al. UEFPN: unified and enhanced feature pyramid networks for small object detection[J]. ACM Transactions on Multimedia Computing, Communications, and Applications, 2023, 19(2s): 1-21.
[27] ZHENG L, PENG Y P, YE Z C, et al. Infrared small UAV target detection algorithm based on enhanced adaptive feature pyramid networks[J]. IEEE Access, 2022, 10: 115988-115995.
[28] 蒋伟, 王万虎, 杨俊杰. AEM-YOLOv8s: 无人机航拍图像的小目标检测[J]. 计算机工程与应用, 2024, 60(17): 191-202.
JIANG W, WANG W H, YANG J J. AEM-YOLOv8s: small target detection algorithm for UAV aerial images[J]. Computer Engineering and Applications, 2024, 60(17): 191-202.
[29] ZHOU J T, XU Q, ZHAO X R, et al. CA2Det: cascaded adaptive fusion pyramid network based on attention mechanism for small object detection[J]. IEEE Access, 2024, 12: 56924-56935.
[30] ZHONG S S, ZHOU H B, ZHENG Z X, et al. Hierarchical attention-guided multiscale aggregation network for infrared small target detection[J]. Neural Networks, 2024, 171: 485-496.
[31] XU S K, GU J N, HUA Y N, et al. DKTNet: dual-key Transformer network for small object detection[J]. Neurocomputing, 2023, 525: 29-41.
[32] WANG L, SHI Y, MAO G J, et al. Consumer-centric insights into resilient small object detection: sciou loss and recursive transformer network[J]. IEEE Transactions on Consumer Electronics, 2024, 70(1): 2178-2187.
[33] ZHU X K, LYU S C, WANG X, et al. TPH-YOLOv5: improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision Workshops. Piscataway: IEEE, 2021: 2778-2788.
[34] CHEN G, WANG H T, CHEN K, et al. A survey of the four pillars for small object detection: multiscale representation, contextual information, super-resolution, and region proposal[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2022, 52(2): 936-953.
[35] 张稀柳, 张晓玲, 何敏军. 基于改进YOLOX-s的车辆检测方法研究[J]. 系统仿真学报, 2024, 36(2): 487-496.
ZHANG X L, ZHANG X L, HE M J. Research on vehicle detection method based on improved YOLOX-s[J]. Journal of System Simulation, 2024, 36(2): 487-496.
[36] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: Transformers for image recognition at scale[J]. arXiv:2010.11929, 2020.
[37] RAO Y M, ZHAO W L, TANG Y S, et al. HorNet: efficient high-order spatial interactions with recursive gated convolutions[J]. arXiv:2207.14284, 2022.
[38] ZHU P F, DU D W, WEN L Y, et al. VisDrone-VID2019: the vision meets drone object detection in video challenge results[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshop. Piscataway: IEEE, 2019: 227-235.
[39] TAN M X, PANG R M, LE Q V. EfficientDet: scalable and efficient object detection[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 10778-10787.
[40] MA X, DAI X Y, BAI Y, et al. Rewrite the stars[C]//Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2024: 5694-5703.
[41] HOWARD A, SANDLER M, CHEN B, et al. Searching for MobileNetV3[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2019: 1314-1324.
[42] QIN D F, LEICHNER C, DELAKIS M, et al. MobileNetV4: universal models for the mobile ecosystem[J]. arXiv:2404. 10518, 2024.
[43] CHEN J R, KAO S H, HE H, et al. Run, don’t walk: chasing higher FLOPS for faster neural networks[C]//Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2023: 12021-12031.
[44] YANG C, HUANG Z H, WANG N Y. QueryDet: cascaded sparse query for accelerating high-resolution small object detection[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 13658-13667.
[45] WANG J J, YU J, HE Z. ARFP: a novel adaptive recursive feature pyramid for object detection in aerial images[J]. Applied Intelligence, 2022, 52(11): 12844-12859.
[46] ZHANG Z X. Drone-YOLO: an efficient neural network method for target detection in drone images[J]. Drones, 2023, 7(8): 526.
[47] YE T, ZHAO Z Y, ZHANG J, et al. Low-altitude small-sized object detection using lightweight feature-enhanced convolutional neural network[J]. Journal of Systems Engineering and Electronics, 2021, 32(4): 841-853.
[48] XIAO H, XIE X M, ZHAO K, et al. YOLO-Q: drone aerial target detection[C]//Proceedings of the 2023 IEEE 4th International Conference on Pattern Recognition and Machine Learning. Piscataway: IEEE, 2023: 13-20.
[49] ZHANG H Y, CHEN E Y. Bi-AFN++CA: bi-directional adaptive fusion network combining context augmentation for small object detection[J]. Applied Intelligence, 2024, 54(1): 614-628.
[50] LI S X, LIU C, TANG K W, et al. Improved YOLOv5s algorithm for small target detection in UAV aerial photography[J]. IEEE Access, 2024, 12: 9784-9791.
[51] SHI T, DING Y, ZHU W X. YOLOv5s_2E: improved YOLOv5s for aerial small target detection[J]. IEEE Access, 2023, 11: 80479-80490.