3D Object Detection Algorithm Based on Raw Point Clouds

doi:10.3778/j.issn.1002-8331.2109-0239

Abstract

Abstract: Aiming at the existing problems in 3D object detection, such as difficult data sampling, insufficient feature extraction, limited receptive field and low regression quality of candidate bounding box, based on 3DSSD 3D object detection algorithm, a single stage and anchor-free 3D object detection algorithm RPV-SSD（random point voxel single stage object detector） is proposed, which based on the raw point clouds. The algorithm is composed of five parts, namely, the random voxel sampling layer, the 3D sparse convolution layer, the feature aggregation layer, the candidate point generation layer, and the region proposal network layer. By aggregating the point-wise feature of the keypoints, the sparse convolution feature of voxel, and the BEV（bird eye view） feature, the category, 3D bounding box and orientation of the object can be predicted. Experiments on KITTI datasets show that the algorithm performs well on the whole. It can not only hit the target in the truth label and regress an accurate bounding box, but also infer the category and complete shape of the object from its incomplete point clouds, and improve the performance of object detection.

Key words: deep learning, raw point clouds, object detection, single stage, anchor free

摘要： 针对当前三维目标检测中存在的数据降采样难、特征提取不充分、感受野有限、候选包围盒回归质量不高等问题，基于3DSSD三维目标检测算法，提出了一种基于原始点云、单阶段、无锚框的三维目标检测算法RPV-SSD（random point voxel single stage object detector），该算法由随机体素采样层、3D稀疏卷积层、特征聚合层、候选点生成层、区域建议网络层共五个部分组成，主要通过聚合随机体素采样的关键点逐点特征、体素稀疏卷积特征、鸟瞰图特征，进而实现对物体类别、3D包围盒以及物体朝向的预测。在KITTI数据集上的实验表明，该算法整体表现良好，不仅能够命中真值标签中的目标并且回归较好的包围盒，还能够从物体的不完整点云推测出物体的类别及其完整形状，提高目标检测性能。

关键词: 深度学习, 原始点云, 目标检测, 单阶段, 无锚框

ZHANG Dongdong, GUO Jie, CHEN Yang. 3D Object Detection Algorithm Based on Raw Point Clouds[J]. Computer Engineering and Applications, 2023, 59(3): 209-217.

张冬冬, 郭杰, 陈阳. 基于原始点云的三维目标检测算法[J]. 计算机工程与应用, 2023, 59(3): 209-217.

References

[1] GUO Y，WANG H，HU Q，et al.Deep learning for 3D point clouds：a survey[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2021，43（12）：4338-4364.
[2] 彭育辉，郑玮鸿，张剑锋.基于深度学习的三维目标检测方法综述[J].汽车技术，2020（9）：1-7.
PENG Y H，ZHEN W H，ZHANG J F.Review on the 3D object detection based on deep learning[J].Automobile Technology，2020（9）：1-7.
[3] WANG Y，YE J.An overview of 3D object detection[J].arXiv：2010.15614，2020.
[4] 李宇杰，李煊鹏，张为公.基于视觉的三维目标检测算法研究综述[J].计算机工程与应用，2020，56（1）：11-24.
LI Y J，LI X P，ZHANG W G.Survey on vision-based 3D object detection methods[J].Computer Engineering and Applications，2020，56（1）：11-24.
[5] QI C R，WEI L，WU C，et al.Frustum pointNets for 3D object detection from RGB-D data[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition，Jun 18-22，2018，Salt Lake City，UT，USA.Washington：IEEE Computer Society，2018：918-927.
[6] XU D，ANGUELOV D，JAIN A.Point fusion：deep sensor fusion for 3D bounding box estimation[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition，Jun 18-22，2018，Salt Lake City，UT，USA.Washington：IEEE Computer Society，2018：244-253.
[7] ZHAO X，LIU Z，HU R，et al.3D object detection using scale invariant and feature reweighting networks[C]//Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence，Jan 27-Feb 1，2019，Honolulu，Hawaii，USA.Palo Alto：AAAI，2019：9267-9274.
[8] WANG Z，JIA K.Frustum ConvNet：sliding frustums to aggregate local point-wise features for amodal 3D object detection[J].arXiv：1903.01864，2019.
[9] ZHOU Y，TUZEL O.VoxelNet：end-to-end learning for point cloud based 3D object detection[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition，Jun 18-22，2018，Salt Lake City，UT，USA.Washington：IEEE Computer Society，2018：4490-4499.
[10] YAN Y，MAO Y，LI B.SECOND：sparsely embedded convolutional detection[J].Sensors，2018，18（10）：3337.
[11] ZHENG W，TANG W，CHEN S，et al.CIA-SSD：confident IoU-aware single-stage object detector from point cloud[C]//Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence.Palo Alto：AAAI，2021：3555-3562.
[12] ZHENG W，TANG W，JIANG L，et al.SE-SSD：self-ensembling single-stage object detector from point cloud[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington：IEEE Computer Society，2021：14494-14503.
[13] QI C R，LITANY O，HE K，et al.Deep houghvoting for 3D object detection in point clouds[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision，Oct 27- Nov 2，2019，Seoul，Korea（South）.Washington：IEEE Computer Society，2019：9276-9285.
[14] SHI S，WANG X，LI H.PointRCNN：3D object proposal generation and detection from point cloud[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition，Jun 16-20，2019，Long Beach，CA，USA.Washington：IEEE Computer Society，2019：770-779.
[15] YANG Z，SUN Y，LIU S，et al.STD：sparse-to-dense 3D object detector for point cloud[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision，Oct 27-Nov 2，2019，Seoul，Korea（South）.Washington：IEEE Computer Society，2019：1951-1960.
[16] YANG Z，SUN Y，LIU S，et al.3DSSD：point-based 3D single stage object detector[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition，Jun 13-19，2020，Seattle，WA，USA.Washington：IEEE Computer Society，2020：11037-11045.
[17] QI C R，YI L，SU H，et al.PointNet++：deep hierarchical feature learning on point sets in a metric space[C]//Proceedings of the Annual Conference on Neural Information Processing Systems，Dec 4-9，2017，Long Beach，CA，USA.Red Hook：Curran Associates，2017：5099-5108.
[18] GEIGER A，LENZ P，STILLER C，et al.Vision meets robotics：the KITTI dataset[J].International Journal of Robotics Research，2013，32（11）：1231-1237.
[19] QI C R，SU H，MO K，et al.PointNet：deep learning on point sets for 3D classification and segmentation[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision，Oct 22-29，2017，Venice，Italy.Washington：IEEE Computer Society，2017：77-85.
[20] TIAN Z，SHEN C，CHEN H，et al.FCOS：fully convolutional one-stage object detection[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision，Oct 27-Nov 2，2019，Seoul，Korea（South）.Washington：IEEE Computer Society，2019：9626-9635.
[21] CHEN X，KUNDU K，ZHU Y，et al.3D object proposals for accurate object class detection[C]//Proceedings of the Annual Conference on Neural Information Processing Systems，Dec 7-12，2015，Montreal，Quebec，Canada.Red Hook：Curran Associates，2015：424-432.