Local Relation Convolution Network for 3D Point Cloud Classification and Segmentation

doi:10.3778/j.issn.1002-8331.2104-0209

Abstract

Abstract: Deep learning has been the main method of point cloud analysis, but the existing methods cannot fully refer to the local shape information while abstracting the point cloud feature, it is difficult to generate a suitable convolution for the shape feature because of the poor robustness of local shape transformations perception. Therefore, this paper proposes local relation convolution（LRConv）, a convolution operator that perceives shape features through comprehensive local relations. Firstly, by referring to the low-dimensional spatial relations among all the neighborhood points in the local point cloud, a local relation description independent of the order and rigid transformations of the points is defined. Then, it uses the multi-layer perceptron（MLP） to obtain the convolution weight of each point in different local areas from relation description. Finally, the features of the points are transformed by the convolution weights, and the abstract features of the local areas are aggregated.Benchmark experiments show that the classification accuracy of the LRConv on ModelNet is improved by 2.1 percentage points over PointNet++, and the segmentation mean intersection over union（mIoU） of LRConv is improved by 1.5 percentage points. The experimental results fully verify the effectiveness of LRConv in feature abstraction.

Key words: point cloud, local relationship, classification and segmentation, convolutional neural network（CNN）, deep learning

摘要： 深度学习已成为点云分析的主要方法，但是现有方法在点云特征抽象时无法充分参考局部形状信息，因此对局部形状变化感知的鲁棒性较差，难以针对形状特征生成合适的卷积核。为此，提出了局部关系卷积（local relation convolution，LRConv），一种通过全面局部关系感知形状特征的卷积算子。参考点云局部中所有邻域点之间的低维空间关系，定义了一种不依赖于点的顺序与刚性变换的局部关系描述；使用多层感知机从关系描述中学习得到局部区域中每个点对应的卷积权重；通过卷积权重来变换点的特征，并聚合局部区域的抽象特征。在基准测试实验中，LRConv分类网络在ModelNet上的分类准确率较PointNet++提高了2.1个百分点，LRConv零件分割网络在ShapeNet上的分割类别平均重合度较PointNet++提高了1.5个百分点。实验结果充分验证了LRConv在特征抽象中的有效性。

关键词: 点云, 局部关系, 分类与分割, 卷积神经网络, 深度学习

GAO Jinjin, LI Luyang. Local Relation Convolution Network for 3D Point Cloud Classification and Segmentation[J]. Computer Engineering and Applications, 2022, 58(19): 276-283.

高金金, 李潞洋. 基于局部关系卷积的点云分类与分割模型[J]. 计算机工程与应用, 2022, 58(19): 276-283.

References

[1] 卢宏涛，张秦川.深度卷积神经网络在计算机视觉中的应用研究综述[J].数据采集与处理，2016，31（1）：1-17.
LU H T，ZHANG Q C.Applications of deep convolutional neural network in computer vision[J].Journal of Data Acquisition and Processing，2016，31（1）：1-17.
[2] 周飞燕，金林鹏，董军.卷积神经网络研究综述[J].计算机学报，2017，40（6）：1229-1251.
ZHOU F Y，JIN L P，DONG J.Review of convolutional neural network[J].Chinese Journal of Computers，2017，40（6）：1229-1251.
[3] 徐姗姗.卷积神经网络的研究与应用[D].南京：南京林业大学，2013.
XU S S.Research and application of convolutional neural network[D].Nanjing：Nanjing Forestry University，2013.
[4] MATURANA D，SCHERER S.VOXNET：A 3D convolutional neural network for real-time object recognition[C]//Proceedings of the 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems（IROS），Hamburg，Germany，Sept 28-Oct 2，2015：922-928.
[5] WU Z，SONG S，KHOSLA A，et al.3D ShapeNets：A deep representation for volumetric shapes[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Boston，MA，USA，June 7-12，2015：1912-1920.
[6] BEN-SHABAT Y，LINDENBAUM M，FISCHER A.3Dmfv：Three-dimensional point cloud classification in real-time using convolutional neural networks[J].IEEE Robotics and Automation Letters，2018，3（4）：3145-3152.
[7] SU H，MAJI S，KALOGERAKIS E，et al.Multi-view convolutional neural networks for 3D shape recognition[C]//Proceedings of the IEEE International Conference on Computer Vision，Santiago，Chile，Dec 7-13，2015：945-953.
[8] LE T，BUI G，DUAN Y.A multi-view recurrent neural network for 3D mesh segmentation[J].Computers & Graphics，2017，66：103-112.
[9] BRONSTEIN M，BRUNA J，LECUN Y，et al.Geometric deep learning：Going beyond euclidean data[J].IEEE Signal Processing Magazine，2017，34（4）：18-42.
[10] QI C R，SU H，NIE?NER M，et al.Volumetric and multi-view CNNs for object classification on 3D data[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Las Vegas，NV，USA，June 27-30，2016：5648-5656.
[11] KLOKOV R，LEMPITSKY V.Escape from cells：Deep KD-networks for the recognition of 3D point cloud models[C]//Proceedings of the IEEE International Conference on Computer Vision，Venice，Italy，Oct 22-29，2017：863-872.
[12] QI C R，SU H，MO K，et al.Pointnet：Deep learning on point sets for 3D classification and segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，HI，USA，July 21-26，2017：652-660.
[13] QI C R，YI L，SU H，et al.PointNet++：Deep hierarchical feature learning on point sets in a metric space[C]//Advances in Neural Information Processing Systems，2017：5099-5108.
[14] LI Y，BU R，SUN M，et al.PointCNN：Convolution on X-transformed points[C]//Advances in Neural Information Processing Systems，2018.：820-830.
[15] KOMARICHEV A，ZHONG Z，HUA J.A-CNN：Annularly convolutional neural networks on point clouds[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Long Beach，CA，USA，June 15-20，2019：7421-7430.
[16] SU H，JAMPANI V，SUN D，et al.SPLATNet：Sparse lattice networks for point cloud processing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2018：2530-2539.
[17] WU W，QI Z，FUXIN L.PointConv：Deep convolutional networks on 3D point clouds[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Long Beach，CA，USA，June 15-20，2019：9621-9630.
[18] LIU Y，FAN B，XIANG S，et al.Relation-shape convolutional neural network for point cloud analysis[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Long Beach，CA，USA，June 15-20，2019：8895-8904.
[19] ZHANG Y，RABBAT M.A graph-CNN for 3D point cloud classification[C]//Proceedings of the 2018 IEEE International Conference on Acoustics，Speech and Signal Processing（ICASSP），Seoul，South Korea，April 22-27， 2018：6279-6283.
[20] TE G，HU W，ZHENG A，et al.RGCNN：Regularized graph CNN for point cloud segmentation[C]//Proceedings of the 26th ACM International Conference on Multimedia，Lisboa Portugal，Oct 10-14，2018：746-754.
[21] WANG Y，SUN Y，LIU Z，et al.Dynamic graph CNN for learning on point clouds[J].ACM Transactions on Graphics，2019，38（5）：1-12.
[22] WANG L，HUANG Y，HOU Y，et al.Graph attention convolution for point cloud semantic segmentation[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），Long Beach，CA，USA，June 15-20，2020：10296-10305.
[23] CHEN C F，QIAN S S，FANG Q，et al.HAPGN：Hierarchical attentive pooling graph network for point cloud segmentation[J].IEEE Transactions on Multimedia，2020，23：2335-2346.
[24] PASZKE A，GROSS S，MASSA F，et al.PyTorch：An imperative style，high-performance deep learning library[C]//Advances in Neural Information Processing Systems，2019：8026-8037.
[25] CHANG A X，FUNKHOUSER T，GUIBAS L，et al.ShapeNet：An information-rich 3D model repository[J].arXiv：1512.03012，2015.