基于多模态融合网络的无人机小目标检测方法

doi:10.3778/j.issn.1002-8331.2508-0255

计算机工程与应用 ›› 2026, Vol. 62 ›› Issue (4): 273-283.DOI: 10.3778/j.issn.1002-8331.2508-0255

基于多模态融合网络的无人机小目标检测方法

姚继林1，刘宏哲1+，张铖1，2，路璐3

1.北京联合大学北京市信息服务工程重点实验室，北京 100101
2.北京强强源起科技有限公司，北京 100101
3.北京安信创业信息科技发展有限公司，北京 100013
+ 通信作者 E-mail：liuhongzhe@buu.edu.cn

收稿日期:2025-08-24 修回日期:2025-10-25 在线发布日期:2026-02-15 出版日期:2026-02-15
基金资助:
国家自然科学基金（62171042，U24A20331）；国家语委重点项目（ZDI145-110）；北京市重点科技项目（KZ202211417048）；北京市属高等学校高水平科研创新团队建设支持计划（BPHR20220121）；北京市自然科学基金（4232026，4242020）；北京联合大学学术研究项目（ZK20202514）。

UAV Small Target Detection Method Based on Multi-Modal Fusion Network

YAO Jilin1, LIU Hongzhe1+, ZHANG Cheng1，2, LU Lu3

1.Beijing Key Laboratory of Information Service Engineering, Beijing Union University, Beijing 100101, China
2.Beijing Qiangqiang Yuanqi Technology Co., Ltd., Beijing 100101, China
3.Beijing Anxin Entrepreneurship Information Technology Development Co., Ltd., Beijing 100013, China
+ Corresponding author E-mail：liuhongzhe@buu.edu.cn

Received:2025-08-24 Revised:2025-10-25 Online:2026-02-15 Published:2026-02-15

摘要/Abstract

摘要： 针对无人机小目标检测在低光和复杂背景下精度低、误检率高的问题，单一模态检测方法难以取得理想效果。因此，提出了一种基于动态交互融合策略的小目标检测方法YOLOv8-DF。设计了双流特征提取网络，采用并行模式分别提取红外和可见光特征，并在Backbone部分引入感受野卷积（receptive field attention convolution，RFAConv），以增强模型的多尺度感知能力。为进一步捕捉全局上下文信息，提出了基于注意力机制的远程信息增强模块（remote information enhancement module，RIEM），并设计了动态特征交互模块（dynamic feature interaction module，DFIM），动态调整各模态优势信息权重，实现深度特征融合。实验结果表明，在DroneVehicle数据集上，YOLOv8-DF与单模态基准模型YOLOv8_RGB和YOLOv8_IR相比，mAP50分别提升了12.3和13.4个百分点，相较于PSFusion融合检测方法，mAP50提升了8.4个百分点，并在LLVIP公开数据集上进行了计算效率和泛化性实验，证明了所提出的方法具有较好的计算效率和泛化性能。

关键词: 无人机检测, 多模态, 小目标检测, 特征融合, YOLO

Abstract: To address the problem of low accuracy and high false detection rate in UAV small target detection under low-light and complex background conditions，where single-modality detection methods often fail to achieve satisfactory performance, this paper proposes a small target detection algorithm based on a dynamic interaction fusion strategy, named YOLOv8-DF. A dual-stream feature extraction network is designed, which extracts infrared and visible features in parallel. The receptive field attention convolution (RFAConv) is introduced into the backbone to enhance the multi-scale perception capability of the model. To further capture global contextual information, a remote information enhancement module (RIEM) based on attention mechanism is proposed. A dynamic feature interaction module (DFIM) is designed to dynamically adjust the weights of advantageous information across modalities, thereby achieving deep feature fusion. Experimental results show that, on the DroneVehicle dataset, YOLOv8-DF improves mAP50 by 12.3 and 13.4 percentage points compared with the single-modality baseline models YOLOv8_RGB and YOLOv8_IR, respectively. Compared with the PSFusion detection method, mAP50 is improved by 8.4 percentage points. Furthermore, computational efficiency and generalization experiments conducted on the LLVIP public dataset demonstrate that the proposed method achieves both competitive efficiency and strong generalization performance.

Key words: UAV detection, multimodal, small target detection, feature fusion, YOLO

姚继林, 刘宏哲, 张铖, 路璐. 基于多模态融合网络的无人机小目标检测方法[J]. 计算机工程与应用, 2026, 62(4): 273-283.

YAO Jilin, LIU Hongzhe, ZHANG Cheng, LU Lu. UAV Small Target Detection Method Based on Multi-Modal Fusion Network[J]. Computer Engineering and Applications, 2026, 62(4): 273-283.

参考文献

[1] MATTERN A，GERDES H，GRUNERT D，et al.A comparison of transformer and CNN-based object detection models for surface defects on Li-Ion Battery Electrodes[J].Journal of Energy Storage，2025，105：114378.
[2] 刘桂超，王怀光，任国全，等.基于深度学习的单目视觉目标检测综述[J].计算机工程与应用，2026，62（1）：1-19.
LIU G C，WANG H G，REN G Q，et al.Review of monocular vision object detection based on deep learning[J].Computer Engineering and Applications，2026，62（1）：1-19.
[3] MAGDY A，MOUSTAFA M S，EBIED H M，et al.Lightweight faster R-CNN for object detection in optical remote sensing images[J].Scientific Reports，2025，15：16163.
[4] 杭焕云，刘俊.结合解耦与广义微调的少样本目标检测[J].计算机工程与应用，2026，62（4）：293-301.
HANG H Y，LIU J.Few-shot object detection combining decoupling and generalized fine-tuning[J].Computer Engineering and Applications，2026，62（4）：293-301.
[5] FAN Q S，LI Y T，DEVECI M，et al.LUD-YOLO：a novel lightweight object detection network for unmanned aerial vehicle[J].Information Sciences，2025，686：121366.
[6] TAN L，WU H，XU Z F，et al.Multi-object garbage image detection algorithm based on SP-SSD[J].Expert Systems with Applications，2025，263：125773.
[7] ZHANG Q，FU Y G.Effective traffic density recognition based on ResNet-SSD with feature fusion and attention mechanism in normal intersection scenes[J].Expert Systems with Applications，2025，261：125508.
[8] YANG X X，QIAN Y Q，ZHU H J，et al.BAANet：learning bi-directional adaptive attention gates for multispectral pedestrian detection[C]//Proceedings of the 2022 International Conference on Robotics and Automation.Piscataway：IEEE，2022：2920-2926.
[9] ZHANG J Q，LEI J，XIE W Y，et al.SuperYOLO：super resolution assisted object detection in multimodal remote sensing imagery[J].IEEE Transactions on Geoscience and Remote Sensing，2023，61：5605415.
[10] MENG F Z，HONG A P，TANG H Y，et al.FQDNet：a fusion-enhanced quad-head network for RGB-infrared object detection[J].Remote Sensing，2025，17（6）：1095.
[11] 官敬超，刘家辉，赵庆辉，等.RFC-YOLOv8：可见光和红外图像自适应融合的目标检测[J].电光与控制，2025，32（9）：54-60.
GUAN J C，LIU J H，ZHAO Q H，et al.RFC-YOLOv8：adaptive fusion of visible and infrared images for target detection[J].Electronics Optics & Control，2025，32（9）：54-60.
[12] CAO Y，BIN J C，HAMARI J，et al.Multimodal object detection by channel switching and spatial attention[C]//Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.Piscataway：IEEE，2023：403-411.
[13] ZHU Y H，SUN X Y，WANG M，et al.Multi-modal feature pyramid transformer for RGB-infrared object detection[J].IEEE Transactions on Intelligent Transportation Systems，2023，24（9）：9984-9995.
[14] GUO J J，GAO C Q，LIU F C，et al.DAMSDet：dynamic adaptive multispectral detection transformer with competitive query selection and adaptive feature fusion[C]//Proceedings of the 18th European Conference on Computer Vision.Cham：Springer，2024：464-481.
[15] LIU K W，LI T，PENG D L.Aerial image object detection based on RGB-infrared multibranch progressive fusion[J].IEEE Transactions on Geoscience and Remote Sensing，2025，63：5402714.
[16] ZHANG X，LIU C，YANG D G，et al.RFAConv：innovating spatial attention and standard convolutional operation[EB/OL].[2025-08-12].https：//arxiv.org/abs/2304.03198.
[17] CAI X H，LAI Q X，WANG Y W，et al.Poly kernel inception network for remote sensing detection[C]//Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway：IEEE，2024：27706-27716.
[18] ISLAM S，ELMEKKI H，ELSEBAI A，et al.A comprehensive survey on applications of transformers for deep learning tasks[J].Expert Systems with Applications，2024，241：122666.
[19] SUN Y M，CAO B，ZHU P F，et al.Drone-based RGB-infrared cross-modality vehicle detection via uncertainty-aware learning[J].IEEE Transactions on Circuits and Systems for Video Technology，2022，32（10）：6700-6713.
[20] JIA X Y，ZHU C，LI M Z，et al.LLVIP：a visible-infrared paired dataset for low-light vision[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision Workshops.Piscataway：IEEE，2021：3489-3497.
[21] DONG W H，ZHU H D，LIN S H，et al.Fusion-Mamba for cross-modality object detection[J].IEEE Transactions on Multimedia，2025，27：7392-7406.
[22] QIAN J C，QIAO B Y，ZHANG Y K，et al.DACFusion：dual asymmetric cross-attention guided feature fusion for multispectral object detection[J].Neurocomputing，2025，635：129913.
[23] LI H，WU X J，DURRANI T.NestFuse：an infrared and visible image fusion architecture based on nest connection and spatial/channel attention models[J].IEEE Transactions on Instrumentation and Measurement，2020，69（12）：9645-9656.
[24] TANG L F，YUAN J T，ZHANG H，et al.PIAFusion：a progressive infrared and visible image fusion network based on illumination aware[J].Information Fusion，2022，83/84：79-92.
[25] MA J Y，TANG L F，FAN F，et al.SwinFusion：cross-domain long-range learning for general image fusion via swin transformer[J].IEEE/CAA Journal of Automatica Sinica，2022，9（7）：1200-1217.
[26] TANG L F，ZHANG H，XU H，et al.Rethinking the necessity of image fusion in high-level vision tasks：a practical infrared and visible image fusion network based on progressive semantic injection and scene fidelity[J].Information Fusion，2023，99：101870.
[27] FANG Q Y，HAN D P，WANG Z K.Cross-modality fusion transformer for multispectral object detection[EB/OL].[2025-08-16].https：//arxiv.org/abs/2111.00273.
[28] SHEN J F，CHEN Y F，LIU Y，et al.ICAFusion：iterative cross-attention guided feature fusion for multispectral object detection[J].Pattern Recognition，2024，145：109913.

基于多模态融合网络的无人机小目标检测方法

UAV Small Target Detection Method Based on Multi-Modal Fusion Network

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	薛光辉, 闫朝阳, 吴冕. PCSED-YOLO：复杂环境下跨尺度多目标穿戴检测算法研究[J]. 计算机工程与应用, 2026, 62(5): 88-105.
[2]	李俊林, 张雪松, 宋存利, 李光宇. 改进YOLOv11的低光照目标检测方法研究[J]. 计算机工程与应用, 2026, 62(5): 106-119.
[3]	余长同, 赖惠成, 汤静雯, 高古学. DBA-YOLO：双骨干轻量化无人机小目标检测算法[J]. 计算机工程与应用, 2026, 62(5): 120-136.
[4]	刘方轩, 李为相, 肖迪, 宋张民杰, 陈闯. 改进YOLOv11n的输电线路缺陷检测算法[J]. 计算机工程与应用, 2026, 62(5): 137-149.
[5]	夏嘉雨, 韩庆邦, 葛考. MSF-YOLO：基于声呐图像的下水道缺陷检测模型[J]. 计算机工程与应用, 2026, 62(5): 150-161.
[6]	董凯, 蔡国永, 邓天生, 王顺杰. 双图融合驱动下说话人感知增强的多模态会话情感分析[J]. 计算机工程与应用, 2026, 62(5): 218-229.
[7]	薛维刚, 谭迎春, 施小刚, 程跃红, 张晋东, 李德生, 宁纪锋. 基于时间信息建模和特征融合的多模态藏酋猴行为识别[J]. 计算机工程与应用, 2026, 62(5): 230-241.
[8]	周渴一, 崔竞松, 李娇娇, 齐浩. 多尺度卷积与自注意力融合下的蛋白质翻译调控[J]. 计算机工程与应用, 2026, 62(5): 364-373.
[9]	张红民, 冯雪, 陈清, 林强. SDA-YOLO：改进YOLOv11的无人机航拍图像小目标检测方法[J]. 计算机工程与应用, 2026, 62(4): 261-272.
[10]	计忠平, 林晨伟, 何志伟, 杜晨杰, 金冉, 柴本成. 集成全局局部感知与分层特征融合的带钢表面缺陷检测[J]. 计算机工程与应用, 2026, 62(4): 373-381.
[11]	王羽铁, 赵军阳, 祝慧鑫, 吕慎华, 喻涵. 面向自动驾驶语义分割的图卷积神经网络方法研究[J]. 计算机工程与应用, 2026, 62(4): 65-79.
[12]	邰徐伟, 孙雪莲. 多尺度边缘增强与空间记忆注意力驱动的头盔检测算法[J]. 计算机工程与应用, 2026, 62(4): 137-151.
[13]	苏易礌, 李卫军, 李贯峰, 刘雪洋, 丁建平, 刘世侠, 王子怡. 基于门控特征融合的双通道文本分类模型[J]. 计算机工程与应用, 2026, 62(4): 168-179.
[14]	李娜, 刘娜, 张翼龙, 冯伟昊. 基于亚像素卷积的多尺度加权融合船舶检测方法[J]. 计算机工程与应用, 2026, 62(3): 287-296.
[15]	王宁, 武芳宇, 赵宇轩, 张百灵, 庞超逸. 基于集成学习与多模态大语言模型的图文情感分析方法[J]. 计算机工程与应用, 2026, 62(3): 153-162.