Multi-Scale Transformer Lidar Point Cloud 3D Object Detection

doi:10.3778/j.issn.1002-8331.2109-0489

Abstract

Abstract: Point cloud 3D object detection has low detection accuracy for small objects such as pedestrians and bicycles, which is easy to miss detection and false detection. A 3D object detection method MSPT-RCNN（multi-scale point transformer-RCNN） based on multi-scale point cloud transformer is proposed to improve the detection accuracy of point cloud 3D objects. The method consists of two stages, the first stage（RPN） and the second stage（RCNN）. In RPN stage, point cloud features are extracted through multi-scale transformer network, which includes multi-scale neighborhood embedding module and jump connection offset attention module to obtain multi-scale neighborhood geometric information and different levels of global semantic information, and generate high-quality initial 3D bounding box. In the RCNN stage, the multi-scale neighborhood geometric information of point cloud in the bounding box is introduced to optimize the position, size, orientation and confidence of the bounding box. The experimental results show that this method（MSPT-RCNN） has high detection accuracy, especially for distant and small objects. MSPT-RCNN can effectively improve the accuracy of 3D object detection by effectively learning the multi-scale geometric information in point cloud data and extracting different levels of effective semantic information.

Key words: Transformer, multi-scale, offset-attention, point cloud, 3D object detection

摘要： 激光雷达点云3D物体检测，对于小物体如行人、自行车的检测精度较低，容易漏检误检，提出一种多尺度Transformer激光雷达点云3D物体检测方法MSPT-RCNN（multi-scale point transformer-RCNN），提高点云3D物体检测精度。该方法包含两个阶段，即第一阶段（RPN）和第二阶段（RCNN）。RPN阶段通过多尺度Transformer网络提取点云特征，该网络包含多尺度邻域嵌入模块和跳跃连接偏移注意力模块，获取多尺度邻域几何信息和不同层次全局语义信息，生成高质量初始3D包围盒；在RCNN阶段，引入包围盒内的点云多尺度邻域几何信息，优化了包围盒位置、尺寸、朝向和置信度等信息。实验结果表明，该方法（MSPT-RCNN）具有较高检测精度，特别是对于远处和较小物体，提升更高。MSPT-RCNN通过有效学习点云数据中的多尺度几何信息，提取不同层次有效的语义信息，能够有效提升3D物体检测精度。

关键词: Transformer, 多尺度, 偏移注意力, 点云, 3D物体检测

SUN Liujie, ZHAO Jin, WANG Wenju, ZHANG Yusen. Multi-Scale Transformer Lidar Point Cloud 3D Object Detection[J]. Computer Engineering and Applications, 2022, 58(8): 136-146.

孙刘杰, 赵进, 王文举, 张煜森. 多尺度Transformer激光雷达点云3D物体检测[J]. 计算机工程与应用, 2022, 58(8): 136-146.

References

[1] GOODFELLOW I，BENGIO Y，COURVILLE A.Deep learning[M].[S.l.]：MIT Press，2016.
[2] GEIGER A，LENZ P，URTASUN R.Are we ready for autonomous driving? the kitti vision benchmark suite[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition，2012：3354-3361.
[3] KUMAR K，NANDAN S，MISHRA A，et al.Voice-controlled object tracking smart robot[C]//2015 International Conference on Signal Processing，Computing and Control（ISPCC），2015：40-45.
[4] UCKELMANN D.A definition approach to smart logis-tics[C]//International Conference on Next Generation Wired/Wireless Networking.Berlin，Heidelberg：Springer，2008：273-284.
[5] ARNOLD E，AL-JARRAH O Y，DIANATI M，et al.A survey on 3d object detection methods for autonomous driving applications[J].IEEE Transactions on Intelligent Transportation Systems，2019，20（10）：3782-3795.
[6] 位超杰，闫仁鹏，李旭东，等.三维成像激光雷达应用的亚纳秒激光器研究进展[J].光学精密工程，2021，29（6）：1270-1280.
WEI C J，YAN R P，LI X D，et al.Research progress of sub-nanosecond lasers for 3D imaging lidar[J].Optics and Precision Engineering，2021，29（6）：1270-1280.
[7] 蔡泽宇，金诚谦.基于二维激光雷达点云的物体轮廓识别[J].应用激光，2020，40（3）：141-146.
CAI Z Y，JIN C Q.Object contour recognition based on 2D lidar point cloud[J].Applied Laser，2020，40（3）：141-146.
[8] GUO Y，WANG H，HU Q，et al.Deep learning for 3d point clouds：a survey[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2021，43（12）：4338-4364.
[9] QI C R，LIU W，WU C，et al.Frustum pointnets for 3d object detection from rgb-d data[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：918-927.
[10] QI C R，SU H，MO K，et al.Pointnet：deep learning on point sets for 3d classification and segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：652-660.
[11] QI C R，YI L，SU H，et al.Pointnet++：deep hierarchical feature learning on point sets in a metric space[J].arXiv：1706.02413，2017.
[12] SHI S，WANG X，LI H.Pointrcnn：3d object proposal generation and detection from point cloud[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：770-779.
[13] SHI S，WANG Z，SHI J，et al.From points to parts：3d object detection from point cloud with part-aware and part-aggregation network[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2021，43（8）：2647-2664.
[14] ZENG Y，HU Y，LIU S，et al.Rt3d：real-time 3-d vehicle detection in lidar point cloud for autonomous driving[J].IEEE Robotics and Automation Letters，2018，3（4）：3434-3440.
[15] LIANG Z，ZHANG M，ZHANG Z，et al.RangeRCNN：towards fast and accurate 3D object detection with range image representation[J].arXiv：2009.00206，2020.
[16] ZHOU Y，TUZEL O.Voxelnet：end-to-end learning for point cloud based 3d object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：4490-4499.
[17] HAN K，WANG Y，CHEN H，et al.A survey on visual transformer[J].arXiv：2012.12556，2020.
[18] 侯强，侯瑞丽.机器翻译方法研究与发展综述[J].计算机工程与应用，2019，55（10）：30-35.
HOU Q，HOU R L.Review of studies and developments on machine translation methodology[J].Computer Engineering and Applications，2019，55（10）：30-35.
[19] 张晓旭，马志强，刘志强，等.Transformer在语音识别任务中的研究现状与展望[J].计算机科学与探索，2021，15（9）：1578-1594.
ZHANG X X，MA Z Q，LIU Z Q，et al.Research status and prospect of Transformer in speech recognition[J].Journal of Frontiers of Computer Science and Technology，2021，15（9）：1578-1594.
[20] YIN W，KANN K，YU M，et al.Comparative study of CNN and RNN for natural language processing[J].arXiv：1702.01923，2017.
[21] DOSOVITSKIY A，BEYER L，KOLESNIKOV A，et al.An image is worth 16x16 words：transformers for image recognition at scale[J].arXiv：2010.11929，2020.
[22] CARION N，MASSA F，SYNNAEVE G，et al.End-to-end object detection with transformers[C]//European Conference on Computer Vision.Cham：Springer，2020：213-229.
[23] ZHENG M，GAO P，WANG X，et al.End-to-end object detection with adaptive clustering transformer[J].arXiv：2011.09315，2020.
[24] WANG Y，XU Z，WANG X，et al.End-to-end video instance segmentation with transformers[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：8741-8750.
[25] ZHAO H，JIANG L，JIA J，et al.Point transformer[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2021：16259-16268.
[26] GUO M H，CAI J X，LIU Z N，et al.PCT：point cloud transformer[J].Computational Visual Media，2021，7（2）：187-199.
[27] PAN X，XIA Z，SONG S，et al.3d object detection with pointformer[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：7463-7472.
[28] GUO G，WANG H，BELL D，et al.KNN model-based approach in classification[C]//OTM Confederated Inter-national Conferences “On the Move to Meaningful Internet Systems”.Berlin，Heidelberg：Springer，2003：986-996.
[29] YANG Z，SUN Y，LIU S，et al.Std：sparse-to-dense 3d object detector for point cloud[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：1951-1960.
[30] WANG Y，SUN Y，LIU Z，et al.Dynamic graph CNN for learning on point clouds[J].ACM Transactions on Graphics，2019，38（5）：1-12.
[31] 李经宇，杨静，孔斌，等.基于注意力机制的多尺度车辆行人检测算法[J].光学精密工程，2021，29（6）：1448-1458.
LI J Y，YANG J，KONG B，et al.Multi-scale vehicle and pedestrian detection algorithm based on attention mechanism[J].Optics and Precision Engineering，2021，29（6）：1448-1458.
[32] BRUNA J，ZAREMBA W，SZLAM A，et al.Spectral networks and locally connected networks on graphs[J].arXiv：1312.6203，2013.
[33] GEIGER A，LENZ P，STILLER C，et al.Vision meets robotics：the kitti dataset[J].The International Journal of Robotics Research，2013，32（11）：1231-1237.
[34] LANG A H，VORA S，CAESAR H，et al.Pointpillars：fast encoders for object detection from point clouds[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：12697-12705.
[35] LIU Z，ZHAO X，HUANG T，et al.Tanet：robust 3d object detection from point clouds with triple attention[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2020，34（7）：11677-11684.