Point Cloud Classification Segmentation Model Based on Self-Attention and Edge Convolution

doi:10.3778/j.issn.1002-8331.2206-0135

Abstract

Abstract: The disorderly, unstructured, and discrete characteristics of cloud data make point cloud classification still challenging. To address the problem that the local structure information between points and the spatial information between regions cannot be captured in point cloud feature extraction, a point cloud classification segmentation network with self-attentiveness and edge convolution—Self Attention DGCNN is proposed. The Self Attention DGCNN cloud classification segmentation network first combines single-layer edge convolution and self-attention mechanism to extract local features and contextual features of point cloud data respectively, and then fuses these two parts of features to pass to the next layer for feature extraction again, and adds the features obtained from each layer to the global feature representation to enhance the capture of overall features of objects. The point cloud classification segmentation experiments are conducted on ModelNet40 dataset and ShapeNet dataset respectively, and the experimental results show that the overall accuracy（OA） of Self Attention DGCNN network reaches 93.5% and the average accuracy（mAcc） reaches 90.8% on ModelNet40 dataset. The overall accuracy is 4.3, 2.8, and 0.6 percentage points higher than PointNet, PointNet++, and Dynamic graph CNN for learning on point clouds（DGCNN）, respectively. The average concurrency ratio（mIoU） on the ShapeNet dataset reaches 86.1%, which is 2.4, 1.0, and 0.9 percentage points higher than PointNet, PointNet++, and DGCNN networks, respectively. There are also different degrees of improvement compared to other deep learning networks.

Key words: point cloud classification, point cloud segmentation, self-attention, neural network, deep learning

摘要： 点云数据的无序性、非结构化、离散的特点使得点云分类仍具有挑战性，针对点云特征提取中无法捕获各点之间的局部结构信息和各区域之间的空间信息的问题，提出了一种自注意力与边卷积的点云分类分割网络——Self Attention DGCNN。Self Attention DGCNN云分类分割网络首先将单层边卷积和自注意力机制相结合，分别提取点云数据的局部特征和上下文特征，然后将这两部分特征进行融合传递到下一层再进行特征抽取，并将各层获取的特征加入到全局特征表示中，从而加强物体整体特征的捕获。在ModelNet40数据集和ShapeNet数据集上分别进行了点云分类，部件分割实验。实验结果表明，在ModelNet40数据集上，Self Attention DGCNN网络的总体精度（OA）达到了93.5%，平均精度（mAcc）达到了90.8%。在总体精度上，相较于PointNet、PointNet++、动态图卷积（dynamic graph CNN for learning on point clouds，DGCNN）分别高出4.3、2.8、0.6个百分点。在ShapeNet数据集上的平均并交比（mIoU）达到了86.1%，相较于PointNet、PointNet++、DGCNN网络分别高出2.4、1.0、0.9个百分点，相比其他深度学习网络也有不同程度的提高。

关键词: 点云分类, 点云分割, 自注意力, 神经网络, 深度学习

SHEN Lu, YANG Jiazhi, ZHOU Guoqing, HUO Jiaxin, CHEN Mengqiang, YU Guangwang, ZHANG Yuyang. Point Cloud Classification Segmentation Model Based on Self-Attention and Edge Convolution[J]. Computer Engineering and Applications, 2023, 59(19): 106-113.

沈露, 杨家志, 周国清, 霍佳欣, 陈梦强, 于广旺, 张玉阳. 集自注意力与边卷积的点云分类分割模型[J]. 计算机工程与应用, 2023, 59(19): 106-113.

References

[1] RUSU R B，MARTON Z C，BLODOW N，et al.Towards 3D Point cloud based object maps for household environments[J].Robotics and Autonomous Systems，2008，56（11）：927-941.
[2] LIANG M，YANG B，WANG S，et al.Deep continuous fusion for multi-sensor 3D object detection[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：641-656.
[3] RAHMAN M M，TAN Y，XUE J，et al.Recent advances in 3D object detection in the era of deep neural networks：a survey[J].IEEE Transactions on image Processing，2019，29：2947-2962.
[4] HE K M，ZHANG X Y，REN S Q，et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2016：770-778.
[5] MATURANA D，SCHERER S.VoxNet：a 3D convolutional neural network for real-time object recognition[C]//Proceedings of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems（IROS），2015：922-928.
[6] RIEGLER G，OSMAN ULUSOY A，GEIGER A.OctNet：learning deep 3D representations at high resolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：3577-3586.
[7] SU H，MAJI S，KALOGERAKIS E，et al.Multi-view convolutional neural networks for 3D shape recognition[C]//Proceedings of the IEEE International Conference on Computer Vision（ICCV），2015.
[8] WEI X，YU R，SUN J.View-GCN：view-based graph convolutional network for 3D shape analysis[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition，2020.
[9] QI C R，SU H，MO K，et al.PointNet：deep learning on point sets for 3D classification and segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：652-660.
[10] QI C R，YI L，SU H，Et al.PointNet++：deep hierarchical feature learning on point sets in a metric space[C]//Advances in Neural Information Processing Systems，2017.
[11] WANG Y，SUN Y B，LIU Z W，et al.Dynamic graph CNN for learning on point clouds[J].ACM Transactions on Graphics（TOG），2019，38（5）：1-12.
[12] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Advances in Neural Information Processing Systems，2017.
[13] WU Z，SONG S，KHOSLA A，et al.3D shapenets：a deep representation for volumetric shapes[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2015：1912-1920.
[14] YI L，KIM V G，CEYLAN D，et al.A scalable active framework for region annotation in 3D shape collections[J].ACM Transactions on Graphics（ToG），2016，35（6）：1-12.
[15] LIN Z H，HUANG S Y，WANG Y C F.Convolution in the cloud：learning deformable kernels in 3D graph convolution networks for point cloud analysis[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：1800-1809.
[16] LI Y，BU R，SUN M，et al.PointCNN：convolution on x-transformed points[C]//Advances in neural Information Processing Systems，2018.
[17] YAN X，ZHENG C，LI Z，et al.PointasNL：robust point clouds processing using nonlocal neural networks with adaptive sampling[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition，2020.
[18] GUO M H，CAI J X，LIU Z N，et al.PCT：point cloud transformer[J].Computational Visual Media，2021，7（2）：187-199.
[19] ZHAO H，JIANG L，FU C W，et al.PointWeb：enhancing local neighborhood features for point cloud processing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：5565-5573.
[20] 王宝乐，霍占强.基于MANet的三维点云特征提取方法研究[J].计算机工程与应用，2022，58（19）：267-275.
WANG B L，HUO Z Q.Research on feature extraction of 3D point cloud based on MANet[J].Computer Engineering and Applications，2022，58（19）：267-275.
[21] 兰红，陈浩，张蒲芬.集图卷积和三维方向卷积的点云分类分割模型[J].计算机工程与用，2023，59（8）：182-191.
LAN H，CHEN H，ZHANG P F.Point cloud classification and segmentation model based on graph convolution and 3D direction convolution[J].Computer Engineering and Applications，2023，59（8）：182-191.
[22] ATZMON M，MARON H，LIPMAN Y.Point convolutional neural networks by extension operators[J].arXiv：1803.10091，2018.
[23] KLOKOV R，LEMPITSKY V.Escape from cells：deep KD-networks for the recognition of 3D point cloud models[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：863-872.
[24] LIN H X，XIAO Z L，TAN Y，et al.Justlookup：one millisecond deep feature extraction for point clouds by lookup tables[C]//Proceedings of the IEEE International Conference on Multimedia and Expo（ICME），2019.
[25] LIU X，HAN Z，LIU Y S，et al.Point2sequence：learning the shape representation of 3D point clouds with an attention-based sequence to sequence network[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2019：8778-8785.
[26] WU W X，QI Z G，LI F X.PointConv：deep convolutional networks on 3D point clouds[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：9621-9630.
[27] GAO Y，LIU X，LI J，et al.LFT-Net：local feature transformer network for point clouds analysis[J].IEEE Transactions on Intelligent Transportation Systems，2022：1-11.
[28] YU J，ZHANG C，WANG H，et al.3D medical point transformer：introducing convolution to attention networks for medical point cloud analysis[J].arXiv：2112.04863，2021.
[29] HAN X F，JIN Y F，CHENG H X，et al.Dual transformer for point cloud analysis[J].arXiv：2104.13044，2021.
[30] WU P，CHEN C，YI J，et al.Point cloud processing via recurrent set encoding[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2019：5441-5449.
[31] KOMARICHEV A，ZHONG Z，HUA J.A-CNN：annularly convolutional neural networks on point clouds[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：7421-7430.