基于特征融合增强的低光照目标检测算法

doi:10.3778/j.issn.1002-8331.2503-0108

摘要/Abstract

摘要： 针对可见光图像目标在低光照等不良条件下特征不足的问题，提出了一种融合可见光图像和红外图像以增强目标特征的目标检测算法。该算法以YOLOv11n为基线算法，通过构建双分支的图像多层级特征提取及融合网络，提取目标可见光图像、红外图像的浅层特征和深层特征，按照特征级融合的方法实现同层级的特征融合；引入了语义与细节注入模块，融合可见光图像、红外图像的不同尺度特征，实现两种图像的目标特征信息优势互补，提高了低光照条件下目标检测性能。在M3FD、LLVIP和FLIR-aligned数据集上对所提算法进行验证，实验结果表明，较基线算法分别检测可见光图像和红外图像，该算法在M3FD数据集上，mAP@0.5分别提升了10.2和12.4个百分点，mAP@0.5：0.95分别提升了7.1和6.0个百分点；在LLVIP数据集上，mAP@0.5分别提升了4.8和0.3个百分点，mAP@0.5：0.95分别提升了12.8和1.8个百分点；在FLIR-aligned数据集上，mAP@0.5分别提升了11.3和3.5个百分点，mAP@0.5：0.95分别提升了8.2和2.0个百分点。该算法对比其他两种基于Transformer的图像融合目标检测算法，总体检测性能更高，证明了该算法的先进性和有效性。

关键词: 低光照目标, 特征级融合, 可见光图像, 红外图像, YOLOv11n

Abstract: A target detection algorithm that combines visible light images and infrared images to enhance target features is proposed to address the problem of insufficient features of visible light image targets under adverse conditions such as low light. This algorithm uses YOLOv11n as the baseline algorithm, and constructs a dual branch image multi-level feature extraction and fusion network to extract shallow and deep features of the target visible light image and infrared image. The feature fusion at the same level is achieved according to the feature level fusion method. Semantic and detail injection modules are introduced to integrate the different scale features of visible light images and infrared images, achieving complementary advantages of target feature information between the two images, and improving the performance of target detection under low light conditions. The proposed algorithm is validated on the M3FD, LLVIP and FLIR aligned datasets, and the experimental results show that compared to the baseline algorithm for detecting visible light images and infrared images, the algorithm is effective on the M3FD dataset, mAP@0.5 is improved by 10.2 and 12.4 percentage points respectively, mAP@0.5：0.95 is improved by 7.1 and 6.0 percentage points respectively; On the LLVIP dataset, mAP@0.5 is improved by 4.8 and 0.3 percentage points respectively, mAP@0.5：0.95 is improved by 12.8 and 1.8 percentage points respectively; On the FLIR aligned dataset, mAP@0.5 is improved by 11.3 and 3.5 percentage points respectively, mAP@0.5：0.95 is improved by 8.2 and 2.0 percentage points respectively. Compared with other two Transformer based image fusion target detection algorithms, this algorithm has higher overall detection performance, which proves the progressiveness and effectiveness of the proposed algorithm.

Key words: low light target, feature level fusion, visible light image, infrared image, YOLOv11n

陈谢发, 李敏, 赵晶钰, 何玉杰, 杨爱涛. 基于特征融合增强的低光照目标检测算法[J]. 计算机工程与应用, 2025, 61(22): 235-244.

CHEN Xiefa, LI Min, ZHAO Jingyu, HE Yujie, YANG Aitao. Low Light Target Detection Algorithm Based on Feature Fusion Enhancement[J]. Computer Engineering and Applications, 2025, 61(22): 235-244.

参考文献

[1] 彭大鑫, 甄彤, 李智慧. 低光照图像增强研究方法综述[J]. 计算机工程与应用, 2023, 59(18): 14-27.
PENG D X, ZHEN T, LI Z H. Survey of research methods for low light image enhancement[J]. Computer Engineering and Applications, 2023, 59(18): 14-27.
[2] 李扬, 杨海涛, 孔卓, 等. 像素级红外与可见光图像融合方法综述[J]. 计算机工程与应用, 2022, 58(14): 40-50.
LI Y, YANG H T, KONG Z, et al. Review of pixel-level infrared and visible image fusion methods[J]. Computer Engineering and Applications, 2022, 58(14): 40-50.
[3] 张宏钢, 杨海涛, 郑逢杰, 等. 特征级红外与可见光图像融合方法综述[J]. 计算机工程与应用, 2024, 60(18): 17-31.
ZHANG H G, YANG H T, ZHENG F J, et al. Review of feature-level infrared and visible image fusion[J]. Computer Engineering and Applications, 2024, 60(18): 17-31.
[4] WANG W S, REN J X, SU C, et al. Ship detection in multispectral remote sensing images via saliency analysis[J]. Applied Ocean Research, 2021, 106: 102448.
[5] ZHANG H, FROMONT E, LEFEVRE S, et al. Guided attentive feature fusion for multispectral pedestrian detection[C]//Proceedings of the 2021 IEEE Winter Conference on Applications of Computer Vision. Piscataway: IEEE, 2021: 72-80.
[6] AN Z J, LIU C L, HAN Y Q. Effectiveness guided cross-modal information sharing for aligned RGB-T object detection[J]. IEEE Signal Processing Letters, 2022, 29: 2562-2566.
[7] 白玉, 侯志强, 刘晓义, 等. 基于可见光图像和红外图像决策级融合的目标检测算法[J]. 空军工程大学学报(自然科学版), 2020, 21(6): 53-59.
BAI Y, HOU Z Q, LIU X Y, et al. An object detection algorithm based on decision-level fusion of visible light image and infrared image[J]. Journal of Air Force Engineering University (Natural Science Edition), 2020, 21(6): 53-59.
[8] 山显英, 张琳, 李泽慧. 深度学习驱动下的目标检测研究进展综述[J]. 计算机工程与应用, 2025, 61(1): 24-41.
SHAN X Y, ZHANG L, LI Z H. Review of research progress in object detection driven by deep learning[J]. Computer Engineering and Applications, 2025, 61(1): 24-41.
[9] 张秀再, 邱野, 沈涛. 基于改进SSD算法的地铁场景小行人目标检测[J]. 计算机研究与发展, 2025, 62(2): 397-407.
ZHANG X Z, QIU Y, SHEN T. Small pedestrian target detection in subway scene based on improved SSD algorithm[J]. Journal of Computer Research and Development, 2025, 62(2): 397-407.
[10] 邵延华, 张铎, 楚红雨, 等. 基于深度学习的YOLO目标检测综述[J]. 电子与信息学报, 2022, 44(10): 3697-3708.
SHAO Y H, ZHANG D, CHU H Y, et al. A review of YOLO object detection based on deep learning[J]. Journal of Electronics & Information Technology, 2022, 44(10): 3697-3708.
[11] 范加利, 田少兵, 黄葵, 等. 基于Faster R-CNN的航母舰面多尺度目标检测算法[J]. 系统工程与电子技术, 2022, 44(1): 40-46.
FAN J L, TIAN S B, HUANG K, et al. Multi-scale object detection algorithm for aircraft carrier surface based on Faster R-CNN[J]. Systems Engineering and Electronics, 2022, 44(1): 40-46.
[12] LI L, HASSAN M A, YANG S R, et al. Development of image-based wheat spike counter through a Faster R-CNN algorithm and application for genetic studies[J]. The Crop Journal, 2022, 10(5): 1303-1311.
[13] 高海涛, 朱超涵, 张天棋, 等. 基于深度学习的无锚框目标检测算法综述[J]. 机床与液压, 2024, 52(1): 202-209.
GAO H T, ZHU C H, ZHANG T Q, et al. Overview of anchor-free object detection algorithms based on depth learning[J]. Machine Tool & Hydraulics, 2024, 52(1): 202-209.
[14] 李悦言, 程培涛, 杜淑幸. 一种改进CenterNet的轻量化目标检测算法[J]. 西安电子科技大学学报, 2022, 49(5): 137-144.
LI Y Y, CHENG P T, DU S X. Lightweight object detection algorithm based on the improved CenterNet[J]. Journal of Xidian University, 2022, 49(5): 137-144.
[15] 杜昌皓, 张智. 改进FCOS算法的车辆检测方法研究[J]. 计算机应用与软件, 2024, 41(6): 257-262.
DU C H, ZHANG Z. Improved FCOS algorithm for vehicle detection[J]. Computer Applications and Software, 2024, 41(6): 257-262.
[16] 刘鑫, 黄进, 杨涛, 等. 改进CenterNet的无人机小目标捕获检测方法[J]. 计算机工程与应用, 2022, 58(14): 96-104.
LIU X, HUANG J, YANG T, et al. Improved small object detection for UAV acquisition based on CenterNet[J]. Computer Engineering and Applications, 2022, 58(14): 96-104.
[17] 陈洛轩, 林成创, 郑招良, 等. Transformer在计算机视觉场景下的研究综述[J]. 计算机科学, 2023, 50(12): 130-147.
CHEN L X, LIN C C, ZHENG Z L, et al. Review of transformer in computer vision[J]. Computer Science, 2023, 50(12): 130-147.
[18] CARION N, MASSA F, SYNNAEVE G, et al. End-to-end object detection with transformers[C]//Proceedings of the 16th European Conference on Computer Vision. Cham: Springer, 2020: 213-229.
[19] ZHU X Z, SU W J, LU L W, et al. Deformable DETR: deformable transformers for end-to-end object detection[J]. arXiv: 2010.04159, 2020.
[20] KHANAM R, HUSSAIN M. YOLOv11: an overview of the key architectural enhancements[J]. arXiv:2410.17725, 2024.
[21] PENG Y P, CHEN D Z, SONKA M. U-Net V2: rethinking the skip connections of U-Net for medical image segmentation[C]//Proceedings of the 2025 IEEE 22nd International Symposium on Biomedical Imaging. Piscataway: IEEE, 2025: 1-5.
[22] LIU J Y, FAN X, HUANG Z B, et al. Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 5792-5801.
[23] JIA X Y, ZHU C, LI M Z, et al. LLVIP: a visible-infrared paired dataset for low-light vision[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision Workshops. Piscataway: IEEE, 2021: 3489-3497.
[24] ZHANG H, FROMONT E, LEFEVRE S, et al. Multispectral fusion for object detection with cyclic fuse-and-refine blocks[C]//Proceedings of the 2020 IEEE International Conference on Image Processing. Piscataway: IEEE, 2020: 276-280.
[25] RAO D Y, XU T Y, WU X J. TGFuse: an infrared and visible image fusion approach based on transformer and generative adversarial network[J]. IEEE Transactions on Image Processing, 2023. DOI: 10. 1109/TIP. 2023. 3273451.
[26] FANG Q Y, HAN D P, WANG Z K. Cross-modality fusion transformer for multispectral object detection[J]. arXiv:2111. 00273, 2021.