Point Cloud Classification and Segmentation Model Based on Graph Convolution and 3D Direction Convolution

doi:10.3778/j.issn.1002-8331.2112-0412

Abstract

Abstract: The existing deep learning methods often ignore the position relationship and direction information between nodes when extracting the local features of point cloud, resulting in the inability to effectively learn the local features of point cloud. To solve this problem, a GCN3D model for point cloud classification and segmentation based on graph convolution and 3D direction convolution is proposed. GCN3D model applies graph convolution neural network to the field of point cloud classification and segmentation. Firstly, the point cloud is regarded as a node on the graph, its [K]-nearest neighbor is obtained for each node, the edge between two nodes in the local [K]-nearest neighbor neighborhood is established, and the edge features are parameterized by graph convolution neural network to capture the local position relationship between nodes and update the central node features. Then, the neighborhood of the node is divided into fine-grained neighborhood blocks in eight directions by using the direction coding module, and the node features in the local neighborhood structure are mapped into different fine-grained neighborhood spaces according to the direction of the 3D spatial coordinate axis to extract the direction information between the nodes, and two direction coding modules are superimposed to increase the receptive field of the network, improve the robustness of the model to sparse point cloud data and obtain local neighborhood multi-scale features. The experiments of point cloud classification and point cloud partial segmentation are carried out on ModelNet40 dataset and ShapeNet dataset respectively. The results show that compared with PointNet without considering local feature information, the overall classification accuracy of GCN3D model on ModelNet40 data set is improved by 3.8 percentage points, and the average classification accuracy is improved by 4.3 percentage points. The average intersection ratio on ShapeNet dataset is improved by 1.5 percentage points. Compared with other deep learning models, the performance of the model is improved in varying degrees.

Key words: point cloud, classification and segmentation, graph convolution neural network, three dimensional direction convolution, fine grained neighborhood, multiscale

摘要： 现有的深度学习方法在提取点云的局部特征时往往忽略了节点间的位置关系和方向信息，导致不能有效地学习点云的局部特征。为解决这一问题，提出一种集图卷积和三维方向卷积的点云分类分割模型GCN3D。GCN3D模型将图卷积神经网络应用在点云分类分割领域。将点云视作图上的节点，对每个节点求其[K]近邻，建立局部[K]近邻邻域内两两节点之间的边，并通过图卷积神经网络参数化边特征以捕捉节点间局部位置关系并更新中心节点特征；使用方向编码模块将节点的邻域划分为八个方位的细粒度的邻域小块，并按照三维空间坐标轴的方向依次将局部邻域结构内的节点特征映射到不同细粒度邻域空间内以提取节点间的方向信息，并且叠加两个方向编码模块增大网络的感受野，提高模型对于稀疏点云数据的鲁棒性并获取局部邻域多尺度特征。在ModelNet40数据集和ShapeNet数据集上分别进行点云分类和点云部分分割的实验。结果表明，相比没有考虑局部特征信息的PointNet，GCN3D模型在ModelNet40数据集上的总体分类精度提高了3.8个百分点，平均分类精度提高了4.3个百分点；在ShapeNet数据集上的平均交并比提高了1.5个百分点。相比其他深度学习模型性能有不同程度的提高。

关键词: 点云, 分类分割, 图卷积神经网络, 三维方向卷积, 细粒度邻域, 多尺度

LAN Hong, CHEN Hao, ZHANG Pufen. Point Cloud Classification and Segmentation Model Based on Graph Convolution and 3D Direction Convolution[J]. Computer Engineering and Applications, 2023, 59(8): 182-191.

兰红, 陈浩, 张蒲芬. 集图卷积和三维方向卷积的点云分类分割模型[J]. 计算机工程与应用, 2023, 59(8): 182-191.

References

[1] 王文曦，李乐林.深度学习在点云分类中的研究综述[J].计算机工程与应用，2022，58（1）：26-40.
WANG W X，LI L L.Review of deep learning in point cloud classification[J].Computer Engineering and Applications，2022，58（1）：26-40.
[2] SHI S S，WANG X G，LI H S.PointRCNN：3D object proposal generation and detection from point cloud[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），2019：770-779.
[3] WANG D Z，POSNER I.Voting for voting in online point cloud object detection[C]//Robotics：Science and Systems，2015.
[4] LIU Z，ZHOU S B，SUO C Z，et al.LPD-Net：3D point cloud learning for large-scale place recognition and environment analysis[C]//2019 IEEE/CVF International Conference on Computer Vision（ICCV），2019：2831-2840.
[5] SIEBERT S，TEIZER J.Mobile 3D mapping for surveying earthwork projects using an unmanned aerial vehicle（UAV） system[J].Automation in Construction，2014，41（41）：1-14.
[6] GUO Y，WANG H Y，HU Q Y，et al.Deep learning for 3D point clouds：a survey[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2021，43（12）：4338-4364.
[7] 景庄伟，管海燕，臧玉府，等.基于深度学习的点云语义分割研究综述[J].计算机科学与探索，2021，15（1）：1-26.
JING Z W，GUAN H Y，ZANG Y F，et al.Survey of point cloud semantic segmentation based on deep learning[J].Journal of Frontiers of Computer Science and Technology，2021，15（1）：1-26.
[8] QICR，SU H，MO K，et al.PointNet：deep learning on point sets for 3D classification and segmentation[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2017：77-85.
[9] QI C R，YI L，SU H，et al.PointNet++：deep hierarchical feature learning on point sets in a metric space[C]//Advances in Neural Information Processing Systems，2017.
[10] ABEYWICKRAMA T，CHEEMA M A，TANIAR D.K-nearest neighbors on road networks：a journey in experimentation and in-memory implementation[J].Proceedings of the VLDB Endowment，2016，9：492-503.
[11] LI Y Y，BU R，SUN M C，et al.PointCNN：convolution on x-transformed points[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems，2018：828-838.
[12] ZHAO H S，JIANG L，FU C W，et al.PointWeb：enhancing local neighborhood features for point cloud processing[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），2019：5565-5573.
[13] 谭台哲，黄永耀，杨卓，等.基于注意力机制的点云神经网络架构搜索方法[J].计算机工程与应用，2022，58（19）：142-151.
TAN T Z，HUANG Y Y，YANG Z，et al.Attention mechanism based point cloud neural architecture search method[J].Computer Engineering and Applications，2022，58（19）：142-151.
[14] 王宝乐，霍占强.基于MANet的三维点云特征提取方法研究[J].计算机工程与应用，2022，58（19）：267-275.
WANG B L，HUO Z Q.Research on feature extraction of 3D point cloud based on MANet[J].Computer Engineering and Applications，2022，58（19）：267-275.
[15] KIPF T N，WELLING M.Semi-supervised classification with graph convolutional networks[J].arXiv：1609.02907，2016.
[16] WANG Y，SUN Y，LIU Z，et al.Dynamic graph CNN for learning on point clouds[J].ACM Transactions on Graphics，2019，38（5）：1-12.
[17] JIANG M，WU Y，ZHAO T，et al.PointSIFT：a SIFT-like network module for 3D point cloud semantic segmentation[J].arXiv：1807.00652，2018.
[18] WU Z，SONG S，KHOSLA A，et al.3D ShapeNets：a deep representation for volumetric shapes[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2015：1912-1920.
[19] YI L，KIM V G，CEYLAN D，et al.A scalable active framework for region annotation in 3D shape collections[J].ACM Transactions on Graphics，2016，35（6）：1-12.
[20] KLOKOV R，LEMPITSKY V.Escape from cells：deep Kd-networks for the recognition of 3D point cloud models[C]//2017 IEEE International Conference on Computer Vision（ICCV），2017：863-872.
[21] ATZMON M，MARON H，LIPMAN Y.Point convolutional neural networks by extension operators[J].arXiv：1803.10091，2018.