改进SAF-FCOS的雷视融合目标检测算法

doi:10.3778/j.issn.1002-8331.2303-0127

摘要/Abstract

摘要： 针对雷视特征融合目标检测网络难以有效利用雷达点云信息和图像特征，在恶劣天气环境下，仍然容易出现误检、漏检的问题，提出了一种改进SAF-FCOS雷视融合目标检测网络。对SAF-FCOS的骨干网络结构进行改进，在C3、C4特征层进行雷达特征信息的多尺度融合，使网络模型更充分地利用雷达信息；在检测层前使用改进的LNblock模块——LNblcok_GAM，能够以较低的计算成本提取图像特征的同时提高网络的检测性能；在回归损失方面，使用基于EIOU与GIOU改进的CEIOU替换原网络中的GIOU，提高了网络的检测精度，提升了模型的鲁棒性。在NuScenes数据集上，改进网络的mAP0.5:0.95达到了70.7%，AP50达到了90.5%，比原网络SAF-FCOS分别提高了1.7个百分点和0.9个百分点，漏检、误检的情况得到了有效改善，同时，该改进网络的总体检测效果要优于其他经典的纯视觉目标检测算法。

关键词: 目标检测, 雷视融合, SAF-FCOS网络, 多尺度融合, LNblock_GAM

Abstract: An improved SAF-FCOS radar-vision fusion target detection network is proposed to address the difficulty in effectively utilizing radar point cloud information and image features, as well as the issue of false or missed detections in harsh weather environments. The backbone network structure of SAF-FCOS is improved, and multi-scale fusion of radar feature information is carried out at C3 and C4 feature layers, so that the network model can make full use of radar information. Using improved LNblock module—LNblcok_GAM before detection layers can extract image features at a lower computational cost while improving the detection performance of the network. In terms of regression loss, the improved CEIOU based on EIOU and GIOU is used to replace the GIOU in the original network, improving the detection accuracy of the network and enhancing the robustness of the model. On the NuScenes dataset, the improved network achieves 70.7% mAP0.5:0.95 and 90.5% AP50, respectively, which are 1.7?percentage points and 0.9?percentage points higher than the original network SAF-FCOS. The missed and false detections are effectively reduced, and the overall detection performance of the improved network is better than other classic visual target detection algorithms.

Key words: target detection, radar-vision fusion, SAF-FCOS network, multi-scale fusion, LNblock_GAM

陈正浩, 邓月明, 谢竞, 何鑫. 改进SAF-FCOS的雷视融合目标检测算法[J]. 计算机工程与应用, 2024, 60(14): 209-218.

CHEN Zhenghao, DENG Yueming, XIE Jing, HE Xin. Improved SAF-FCOS Target Detection Algorithm Based on Radar-Vision Fusion[J]. Computer Engineering and Applications, 2024, 60(14): 209-218.

参考文献

[1] 邵延华, 张铎, 楚红雨, 等. 基于深度学习的YOLO目标检测综述[J]. 电子与信息学报, 2022, 44(10): 3697-3708.
SHAO Y H, ZHANG D, CHU H Y, et al. A review of YOLO object detection based on deep learning[J]. Journal of Electronics and Information, 2022, 44(10): 3697-3708.
[2] LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector[C]//Proceedings of the European Conference on Computer Vision, 2016: 21-37.
[3] TIAN Z, SHEN C, CHEN H, et al. FCOS: fully convolutional one-stage object detection[C]//Proceedings of the IEEE International Conference on Computer Vision, 2019: 9627-9636.
[4] GIRSHICK R. Fast R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision, 2015: 1440-1448.
[5] REN S, HE K, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6): 1137-1149.
[6] 刘春磊, 陈天恩, 王聪, 等. 小样本目标检测研究综述[J]. 计算机科学与探索, 2023, 17(1): 53-73.
LIU C L, CHEN T E, WANG C, et al. Survey of few-shot object detection[J]. Journal of Frontiers of Computer Science & Technology, 2023, 17(1): 53-73.
[7] 王若萱, 吴建平, 徐辉. 自动驾驶汽车感知系统仿真的研究及应用综述[J]. 系统仿真学报, 2022, 34(12): 2507-2521.
WANG R X, WU J P, XU H. Overview of research and application of autonomous vehicle sensing system simulation[J]. Journal of System Simulation, 2022, 34(12): 2507-2521.
[8] WEI Z, ZHANG F, CHANG S, et al. MmWave radar and vision fusion for object detection in autonomous driving: a review[J]. arXiv:2108.03004, 2021.
[9] KUMAWAT H, MUKHOPADHYAY S. Radar guided dynamic visual attention for resource-efficient RGB object detection[C]//Proceedings of the 2022 International Joint Conference on Neural Networks, 2022: 1-8.
[10] XIAO P, GUO J S, DU J, et al. Pedestrian detection based on fusion of millimeter wave radar and vision[C]//Proceedings of the 2018 International Conference on Artificial Intelligence and Pattern Recognition, 2018: 38-42.
[11] LONG N, WANG K, CHENG R, et al. Fusion of millimeter wave radar and RGB-Depth sensors for assisted navigation of the visually impaired[C]//Proceedings of the Conference on Millimetre Wave and Terahertz Sensors and Technology, 2018.
[12] LV P, WANG B, CHENG F, et al. Multi-objective association detection of farmland obstacles based on information fusion of millimeter wave radar and camera[J]. Sensors, 2023, 23(1): 230.
[13] SONG Y, XIE Z, WANG X, et al. MS-YOLO: object detection based on YOLOv5 optimized fusion millimeter-wave radar and machine vision[J]. IEEE Sensors Journal, 2022, 22(15): 15435-15447.
[14] NOBIS F, GEISSLINGE M, WEBE M, et al. A deep learning-based radar and camera sensor fusion architecture for object detection[J]. arXiv:2005.07431, 2020.
[15] CHANG S, ZHANG Y, ZHANG F, et al. Spatial attention fusion for obstacle detection using MmWave radar and vision sensor[J]. Sensors, 2020, 20(4): 956.
[16] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
[17] STACKER L, HEIDENREICH P, RAMBACH J, et al. Fusion point pruning for optimized 2D object detection with radar-camera fusion[C]//Proceedings of the 2022 IEEE/CVF Winter Conference on Applications of Computer Vision, 2022: 1275-1282.
[18] QI C, SONG C, ZHANG N, et al. Millimeter-wave radar and vision fusion target detection algorithm based on an extended network[J]. Machines, 2022, 10(8): 675.
[19] HWANG B, LEE S, HAN H. LNFCOS: efficient object detection through deep learning based on LNblock[J]. ?Electronics, 2022, 11(17): 2783.
[20] CHOLLET F. Xception: deep learning with depthwise separable convolutions[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 1800-1807.
[21] FEI W, JIANG M, CHEN Q, et al. Residual attention network for image classification[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 6450-6458.
[22] LIU Y, SHAO Z, HOFFMANN N. Global attention mechanism: retain information to enhance channel-spatial interactions[J]. arXiv:2112.05561, 2021.
[23] 王剑哲, 吴秦. 坐标注意力特征金字塔的显著性目标检测算法[J]. 计算机科学与探索, 2023, 17(1): 154-165.
WANG J Z, WU Q. Saliency target detection algorithm based on coordinate attention feature pyramid[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(1): 154-165.
[24] 陈欣, 万敏杰, 马超, 等. 采用多尺度特征融合SSD的遥感图像小目标检测[J]. 光学精密工程, 2021, 29(11): 2672-2682.
CHEN X, WAN M J, MA C, et al. Small target detection in remote sensing image using multi-scale feature fusion SSD[J]. Optics and Precision Engineering, 2021, 29(11): 2672-2682.
[25] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the IEEE European Conference on Computer Vision, 2018: 3-19.
[26] ZHANG Q L, YANG Y B. SA-Net: shuffle attention for deep convolutional neural networks[C]//Proceedings of the 2021 IEEE International Conference on Acoustics, Speech and Signal Processing, 2021: 2235-2239.
[27] WANG Q, WU B, ZHU P, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
[28] QIN X, LI N, WENG C, et al. Simple attention module based speaker verification with iterative noisy label detection[C]//Proceedings of the 2022 IEEE International Conference on Acoustics, Speech and Signal Processing, 2022: 6722-6726.
[29] 郑美俊, 田益民, 杨帅. 基于改进FCOS网络的遥感目标检测[J]. 航天返回与遥感, 2022, 43(5): 133-141.
ZHENG M J, TIAN Y M, YANG S. Remote sensing target detection based on improved FCOS network[J]. Space Return and Remote Sensing, 2022, 43(5): 133-141.
[30] REZATOFIGHI H, TSOI N, GWAK J Y, et al. Generalized intersection over union: a metric and a loss for bounding box regression[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 658-666.
[31] ZHANG Y F, REN W, ZHANG Z, et al. Focal and efficient IOU loss for accurate bounding box regression[J]. Neurocomputing, 2022, 506: 146-157.
[32] REDMON J, FARHADI A. YOLOv3: an incremental improvement[J]. arXiv:1804.02767, 2018.
[33] BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection[J]. arXiv:2004.10934, 2020.
[34] WANG C Y, YEH I H, LIAO H Y M. You only learn one representation: unified network for multiple tasks[J]. arXiv:2105.04206, 2021.
[35] LAW H, DENG J. CornerNet: detecting objects as paired keypoints[J]. International Journal of Computer Vision, 2020, 128(3): 642-656.