Improved DDETR UAV Target Detection Algorithm Incorporating Occlusion Information

doi:10.3778/j.issn.1002-8331.2208-0025

Abstract

Abstract: For the problem of complex target scenes, many small targets and severe occlusion in UAV aerial images, an improved DDETR (deformable DETR) UAV target detection algorithm that incorporates target occlusion information is proposed. The proposed model replaces the residual network in the DDETR model with Swin Transformer to obtain richer multi-level semantic features; increases the use of low-level features in the DDETR model to improve the detection of small and medium-sized targets; and uses the proposed occlusion degree estimation module to assist the model in solving the occlusion problem, so that the model can better detect the heavily occluded targets. The mean average precision (AP) of 32.3% is achieved on the VisDrone dataset, which is 3.3 percentage points higher than the AP value of the standard DDETR model, and reaches the current advanced level compared with the mainstream UAV aerial image target detection methods.

Key words: unmanned aerial vehicle (UAV) target detection, deep learning, cross attention, deformable convolution

摘要： 针对无人机航拍图像中目标场景复杂、小目标多、遮挡严重的问题，提出了一种融合目标遮挡信息的改进DDETR（deformable DETR）的无人机目标检测算法。提出模型用Swin Transformer代替DDETR模型中残差网络来获得更丰富的多层次语义特征；增加DDETR模型对低层次特征的使用来提高对中小目标的检测效果；利用提出的遮挡程度估计模块来辅助模型解决遮挡问题，使模型能更好地检测出遮挡严重的目标。在VisDrone数据集上达到32.3%的平均准确度均值（mean average precision，AP），比标准DDETR模型AP值提高了3.3个百分点，与主流无人机航拍图像目标检测方法相比，达到了当前先进水平。

关键词: 无人机目标检测, 深度学习, 交叉注意力, 可变形卷积

ZHOU Jianting, XUAN Shibin, WANG Ting. Improved DDETR UAV Target Detection Algorithm Incorporating Occlusion Information[J]. Computer Engineering and Applications, 2024, 60(1): 236-244.

周建亭, 宣士斌, 王婷. 融合遮挡信息的改进DDETR无人机目标检测算法[J]. 计算机工程与应用, 2024, 60(1): 236-244.

References

[1] SRIVASTAVA S, NARAYAN S, MITTAL S. A survey of deep learning techniques for vehicle detection from UAV images[J]. Journal of Systems Architecture, 2021, 117: 102152.
[2] 江波, 屈若锟, 李彦冬, 等. 基于深度学习的无人机航拍目标检测研究综述[J]. 航空学报, 2021, 42(4): 131-145.
JIANG B, QU R K, LI Y D, et al. Object detection in UAV imagery based on deep learning: review[J]. Acta Aeronautica et Astronautica Sinica, 2021, 42(4): 131-145.
[3] VIOLA P, JONES M J. Robust real-time face detection[J]. International Journal of Computer Vision, 2004, 57(2): 137-154.
[4] REN S, HE K, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149.
[5] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 42(2): 318-327.
[6] YANG F, FAN H, CHU P, et al. Clustered object detection in aerial images[J]. arXiv:1904.08008, 2019.
[7] DENG S, LI S, XIE K, et al. A global-local self-adaptive network for drone-view object detection[J]. IEEE Transactions on Image Processing, 2021, 30: 1556-1569.
[8] 刘英杰, 杨风暴, 胡鹏. 基于Cascade R-CNN的并行特征金字塔网络无人机航拍图像目标检测算法[J]. 激光与光电子学进展, 2020, 57(20): 302-309.
LIU Y J, YANG F B, HU P. Parallel FPN algorithm based on cascade R-CNN for object detection from UAV aerial images[J]. Laser & Optoelectronics Progress, 2020, 57(20): 302-309.
[9] 周阳. 基于深度学习的小目标检测方法研究与应用[D]. 成都: 电子科技大学, 2021.
ZHOU Y. Research and application on small object detection method based on deep learning[D]. Chengdu: University of Electronic Science and Technology of China, 2021.
[10] 张胜虎, 马惠敏. 遮挡对于目标检测的影响分析[J]. 图学学报, 2020, 41(6): 891-896.
ZHANG S H, MA H M. An analysis of occlusion influence on object detection[J]. Journal of Graphics, 2020, 41(6): 891-896.
[11] ZHANG S, WEN L, BIAN X, et al. Occlusion-aware R-CNN: detecting pedestrians in a crowd[J]. arXiv:1807.08407, 2018.
[12] 张瑞倩, 邵振峰, PORTNOV A, 等. 多尺度空洞卷积的无人机影像目标检测方法[J]. 武汉大学学报(信息科学版), 2020, 45(6): 895-903.
ZHANG R Q, SHAO Z F, PORTNOV A, et al. Multi?scale dilated convolutional neural network for object detection in UAV images[J]. Geomatics and Information Science of Wuhan University, 2020, 45(6): 895-903.
[13] 田永林, 王雨桐, 王建功, 等. 视觉Transformer研究的关键问题: 现状及展望[J]. 自动化学报, 2022, 48(4): 957-979.
TIAN Y L, WANG Y T, WANG J G, et al. Key problems and progress of vision Transformers: the state of the art and prospects[J]. Acta Automatica Sinica, 2022, 48(4): 957-979.
[14] VASWANI A, SHAZEER N M, PARMAR N, et al. Attention is all you need[J]. arXiv:1706.03762, 2017.
[15] LIU Z, LIN Y, CAO Y, et al. Swin Transformer: hierarchical vision transformer using shifted windows[J]. arXiv:2103. 14030, 2021.
[16] ZHU X, SU W, LU L, et al. Deformable DETR: deformable transformers for end?to?end object detection[J]. arXiv:2010. 04159, 2021.
[17] DAI J, QI H, XIONG Y, et al. Deformable convolutional networks[J]. arXiv:1703.06211, 2017.
[18] CHEN K, WANG J Q, PANG J M, et al. MMDetection: open MMLab detection toolbox and benchmark[J]. arXiv:1906.07155, 2019.
[19] LAW H, DENG J. CornerNet: detecting objects as paired keypoints[C]//Computer Vision-ECCV 2018. Cham: Springer International Publishing, 2018: 765-781.
[20] RUKHOVICH D, SOFIIUK K, GALEEV D, et al. IterDet: iterative scheme for object detection in crowded environments[J]. arXiv:2005.05708, 2021.
[21] 徐坚, 谢正光, 李洪均. 特征平衡的无人机航拍图像目标检测算法[J]. 计算机工程与应用, 2023, 59(6): 196-203.
XU J, XIE Z G, LI H J. Feature-balanced UAV aerial image target detection algorithm[J]. Computer Engineering and Applications, 2023, 59(6): 196-203.
[22] ALBABA B M, OZER S. SyNet: an ensemble network for object detection in UAV images[C]//25th International Conference on Pattern Recognition, 2021.
[23] SUN W, DAI L, ZHANG X, et al. RSOD: real-time small object detection algorithm in UAV-based traffic monitoring[J]. Applied Intelligence, 2022, 52(8): 8448-8463.
[24] LI C, YANG T, ZHU S, et al. Density map guided object detection in aerial images[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW), Seattle, WA, USA, 2020: 737-746.
[25] YANG C, HUANG Z, WANG N. QueryDet: cascaded sparse query for accelerating high-resolution small object detection[J]. arXiv:2103.09136, 2021.
[26] LIU Y, YANG F, HU P. Small-object detection in UAV-captured images via multi-branch parallel feature pyramid networks[J]. IEEE Access, 2020, 8: 145740-145750.
[27] ZHANG X, IZQUIERDO E, CHANDRAMOULI K. Dense and small object detection in UAV vision based on cascade network[C]//2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Korea (South), 2019: 118-126.
[28] ZHANG J, HUANG J, CHEN X, et al. How to fully exploit the abilities of aerial image detectors[C]//2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Korea (South), 2019: 1-8.