3D Object Detection Based on Strong Semantic Key Point Sampling

doi:10.3778/j.issn.1002-8331.2212-0239

Abstract

Abstract: Feature extraction of key information in the target detection algorithm is an important factor affecting the accuracy of the algorithm. Aiming at the problems of key point sampling difficulty and insufficient feature extraction in the current 3D target detection algorithm, used for reference of PV-RCNN 3D target detection network, a 3D target detection algorithm SSPS-RCNN (strong semantic point sampling RCNN) based on strong semantic key point sampling is proposed. In the key point sampling stage, the algorithm used the fusion method of semantic weighted point sampling and proposal area point filtering to obtain more characteristic representative key points and increase the proportion of foreground points in the sampling points. Without adding the network structure, the point by semantic information is reweighted to further refine the feature contribution of the key points to improve the algorithm accuracy. Experiments show that this algorithm can reduce the problem of missing detection and wrong detection than the existing mainstream algorithms, and show good stability and robustness. Experimental results on KITTI dataset show that the proposed algorithm has good stability and robustness compared with the existing mainstream algorithms, which can reduce the problem of missing detection and wrong detection and improve the overall detection accuracy.

Key words: deep learning, light detection and ranging (LiDAR), 3D object detection, feature fusion

摘要： 目标检测算法中关键信息的特征提取是影响算法精度的重要因素。针对当前三维目标检测算法中存在的关键点采样困难、特征提取不充分等问题，借鉴PV-RCNN三维目标检测网络，提出一种基于强语义关键点采样的三维目标检测方法SSPS-RCNN（strong semantic point sampling RCNN）。在关键点采样阶段，该算法采用语义加权点采样和提案区域点过滤相融合的方法，获得更具特征代表性的采样关键点，以提升采样点中前景点的比例；在不增加网络结构的基础上，将语义信息重新加权关键点特征，以进一步细化关键点的特征贡献提升算法精度。在KITTI数据集上的实验结果表明该算法与现有主流算法相比，对减少物体检测中的漏检与错检问题和整体检测精度提升，表现出良好的稳定性和鲁棒性。

关键词: 深度学习, 激光雷达, 三维目标检测, 特征融合

CHE Yunlong, YUAN Liang, SUN Lihui. 3D Object Detection Based on Strong Semantic Key Point Sampling[J]. Computer Engineering and Applications, 2024, 60(9): 254-260.

车运龙, 袁亮, 孙丽慧. 基于强语义关键点采样的三维目标检测方法[J]. 计算机工程与应用, 2024, 60(9): 254-260.

References

[1] GHASEMIEH A, KASHEF R. 3D object detection for autonomous driving: methods, models, sensors, data, and challenges[J]. Transportation Engineering, 2022, 8: 100115.
[2] QIAN R, LAI X, LI X. 3D object detection for autonomous driving: a survey[J]. Pattern Recognition, 2022, 130: 108796.
[3] 肖雨晴, 杨慧敏. 目标检测算法在交通场景中应用综述[J]. 计算机工程与应用, 2021, 57(6): 30-41.
XIAO Y Q, YANG H M. Research on application of object detection algorithm in traffic scene[J]. Computer Engineering and Applications, 2021, 57(6): 30-41.
[4] 李宇杰, 李煊鹏, 张为公. 基于视觉的三维目标检测算法研究综述[J]. 计算机工程与应用, 2020, 56(1): 11-24.
LI Y J, LI X P, ZHANG W G. Survey on vision-based 3D object detection methods[J]. Computer Engineering and Applications, 2020, 56(1): 11-24.
[5] ZHOU Y, TUZEL O. Voxelnet: end-to-end learning for point cloud based 3D object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 4490-4499.
[6] YAN Y, MAO Y, LI B. Second: sparsely embedded convolutional detection[J]. Sensors, 2018, 18(10): 3337.
[7] QI C R, SU H, MO K, et al. PointNet: deep learning on point sets for 3D classification and segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 652-660.
[8] QI C R, YI L, SU H, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017.
[9] SHI S, WANG X, LI H. PointRCNN: 3D object proposal generation and detection from point cloud[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 770-779.
[10] SHI S, GUO C, JIANG L, et al. PV-RCNN: point-voxel feature set abstraction for 3D object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 10529-10538.
[11] SHI S, JIANG L, DENG J, et al. PV-RCNN++: point-voxel feature set abstraction with local vector representation for 3D object detection[J]. International Journal of Computer Vision, 2023, 131: 531-551.
[12] CHEN C, CHEN Z, ZHANG J, et al. SASA: semantics-augmented set abstraction for point-based 3d object detection[C]//AAAI Conference on Artificial Intelligence, 2022.
[13] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision, 2017: 2980-2988.
[14] GEIGER A, LENZ P, STILLER C, et al. Vision meets robotics: the kitti dataset[J]. The International Journal of Robotics Research, 2013, 32(11): 1231-1237.
[15] KU J, MOZIFIAN M, LEE J, et al. Joint 3D proposal generation and object detection from view aggregation[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018: 1-8.
[16] QI C R, LIU W, WU C, et al. Frustum pointnets for 3D object detection from RGB-D data[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 918-927.
[17] WANG Z, JIA K. Frustum convnet: sliding frustums to aggregate local point-wise features for amodal 3D object detection[C]//2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019: 1742-1749.
[18] SHI S, WANG Z, WANG X, et al. Part-A2 net: 3D part-aware and aggregation neural network for object detection from point cloud[J]. arXiv:1907.03670, 2019.
[19] HE Q, WANG Z, ZENG H, et al. SVGA-Net: sparse voxel-graph attention network for 3d object detection from point clouds[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2022: 870-878.
[20] CHEN X, MA H, WAN J, et al. Multi-view 3D object detection network for autonomous driving[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 1907-1915.
[21] YOO J H, KIM Y, KIM J, et al. 3D-CVF: generating joint camera and lidar features using cross-view spatial feature fusion for 3D object detection[C]//European Conference on Computer Vision. Cham: Springer, 2020: 720-736.
[22] LI Y, QI X, CHEN Y, et al. Voxel field fusion for 3D object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 1120-1129.
[23] YANG Z, SUN Y, LIU S, et al. STD: sparse-to-dense 3D object detector for point cloud[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019: 1951-1960.
[24] HE C, ZENG H, HUANG J, et al. Structure aware single-stage 3D object detection from point cloud[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 11873-11882.
[25] YANG Z, SUN Y, LIU S, et al. 3DSSD: point-based 3D single stage object detector[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 11040-11048.
[26] DENG J, SHI S, LI P, et al. Voxel R-CNN: towards high performance voxel-based 3D object detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2021: 1201-1209.
[27] MAO J, XUE Y, NIU M, et al. Voxel transformer for 3D object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 3164-3173.
[28] SONG N, JIANG T, YAO J. JPV-Net: joint point-voxel representations for accurate 3D object detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2022: 2271-2279.