KPP3D：Key Point Painting for 3D Object Detection

doi:10.3778/j.issn.1002-8331.2205-0023

Abstract

Abstract: In order to fuse image information to improve the accuracy of the 3D object detection model using lidar sensor and reduce runtime while improving detection accuracy, a 3D target detection model KPP3D based on key point information fusion is proposed. KPP3D uses a lightweight key point generation module to replace the non-derivable sampling process, so that the entire model can be trained end-to-end to generate task-related key points. Aiming at the problem of high sampling repetition rate caused by the introduction of this module, a sampling loss function based on the evolution of Bernoulli distribution entropy is proposed to reduce the repetition rate and control the attention distribution of key points. Experiments in the autonomous driving dataset KITTI show that the fusion of image and point cloud can improve the accuracy of 3D object detection, and the use of the KPP3D model with 512 key points can reduce the time consumption by 38%. In addition, experiments verify that the newly proposed loss function can make the key point non-repetitive rate reach more than 98%, thereby reducing the redundancy of key points.

Key words: sensor fusion, autonomous driving, 3D object detection, lidar, sampling

摘要： 为了融合图像信息以提高单激光雷达传感器模型的3D目标检测准确率，使融合感知模型在提升检测精度的同时减少运行时间，提出了基于关键点信息融合的3D目标检测模型KPP3D。KPP3D使用轻量级关键点生成模块取代不可求导的采样过程，使整个模型可以进行端到端的训练从而生成任务相关的关键点。针对引入该模块产生的高采样重复率问题，提出了一种基于伯努利分布熵演变来的采样损失函数降低重复率并聚焦关键点的注意力分布。在自动驾驶数据集KITTI的实验表明，融合图像信息到点云能提升3D目标检测准确率且使用512个关键点的KPP3D模型能减少38%时间消耗。此外，实验验证了新提出的损失函数能使关键点的不重复率达到98%以上，从而减少了关键点的冗余。

关键词: 传感器融合, 自动驾驶, 3D目标检测, 激光雷达, 采样

WANG Mingming, CHEN Qingkui, FU Zhibing. KPP3D：Key Point Painting for 3D Object Detection[J]. Computer Engineering and Applications, 2023, 59(17): 195-204.

汪明明, 陈庆奎, 付直兵. KPP3D：基于关键点信息融合的3D目标检测模型[J]. 计算机工程与应用, 2023, 59(17): 195-204.

References

[1] ZHOU Y，TUZEL O.Voxelnet：end-to-end learning for point cloud based 3D object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：4490-4499.
[2] LANG A H，VORA S，CAESAR H，et al.Pointpillars：fast encoders for object detection from point clouds[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：12697-12705.
[3] YAN Y，MAO Y，LI B.Second：sparsely embedded convolutional detection[J].Sensors，2018，18（10）：3337.
[4] YANG Z，SUN Y，LIU S，et al.3DSSD：point-based 3D single stage object detector[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Re-cognition，2020：11040-11048.
[5] CHEN X，MA H，WAN J，et al.Multi-view 3D object detection network for autonomous driving[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：1907-1915.
[6] QI C R，LIU W，WU C，et al.Frustum pointnets for 3D object detection from RGB-D data[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Re-cognition，2018：918-927.
[7] KU J，MOZIFIAN M，LEE J，et al.Joint 3D proposal generation and object detection from view aggregation[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems（IROS），2018：1-8.
[8] GEIGER A，LENZ P，URTASUN R.Are we ready for autonomous driving? the KITTI vision benchmark suite[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition，2012：3354-3361.
[9] CAESAR H，BANKITI V，LANG A H，et al.nuScenes：a multimodal dataset for autonomous driving[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：11621-11631.
[10]   VORA S，LANG A H，HELOU B，et al.Pointpainting：sequential fusion for 3D object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：4604-4612.
[11]   SONG S，CHANDRAKER M.Joint sfm and detection cues for monocular 3D localization in road scenes[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2015：3734-3742.
[12]   CHEN X，KUNDU K，ZHANG Z，et al.Monocular 3D object detection for autonomous driving[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：2147-2156.
[13]   MOUSAVIAN A，ANGUELOV D，FLYNN J，et al.3D bounding box estimation using deep learning and geome-try[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：7074-7082.
[14] YANG B，LUO W，URTASUN R.Pixor：real-time 3D object detection from point clouds[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Re-cognition，2018：7652-7660.
[15]   SIMONY M，MILZY S，MENDEY K，et al.Complex-yolo：an euler-region-proposal for real-time 3D object detection on point clouds[C]//Proceedings of the European Conference on Computer Vision（ECCV） Workshops，2018.
[16]   YANG B，LIANG M，URTASUN R.HDNET：exploiting HD maps for 3D object detection[C]//Conference on Robot Learning，2018：146-155.
[17]   LI B，ZHANG T，XIA T.Vehicle detection from 3D lidar using fully convolutional network[J].arXiv：1608.07916，2016.
[18]   CHAI Y，SUN P，NGIAM J，et al.To the point：efficient 3D object detection in the range image with graph convolution kernels[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：16000-16009.
[19]   GRAHAM B，ENGELCKE M，VAN DER MAATEN L. 3D semantic segmentation with submanifold sparse convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：9224-9232.
[20]   QI C R，SU H，MO K，et al.PointNet：deep learning on point sets for 3D classification and segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：652-660.
[21]   QI C R，YI L，SU H，et al.PointNet++：deep hierarchical feature learning on point sets in a metric space[C]//Advances in Neural Information Processing Systems，2017.
[22]   SHI S，WANG X，LI H.Pointrcnn：3D object proposal generation and detection from point cloud[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：770-779.
[23]   QI C R，LITANY O，HE K，et al.Deep hough voting for 3D object detection in point clouds[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：9277-9286.
[24]   SHI S，GUO C，JIANG L，et al.PV-RCNN：point-voxel feature set abstraction for 3D object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：10529-10538.
[25]   LIU G，MILAN A，SHEN C，et al.Refinenet：multi-path refinement networks for high-resolution semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：1925-1934.
[26]   CHEN L C，PAPANDREOU G，SCHROFF F，et al.Rethinking atrous convolution for semantic image segmentation[J].arXiv：1706.05587，2017.
[27]   CHEN L C，ZHU Y，PAPANDREOU G，et al.Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：801-818.
[28]   ZHU Y，SAPRA K，REDA F A，et al.Improving semantic segmentation via video propagation and label relaxation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：8856-8865.
[29]   KRIZHEVSKY A，SUTSKEVER I，HINTO G E.Image-Net classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems，2012.
[30] KINGMA D P，BA J.Adam：a method for stochastic optimization[J].arXiv：1412.6980，2014.