Research on Point Cloud Classification and Segmentation of Cascaded Edge Convolution and Attention Mechanism

doi:10.3778/j.issn.1002-8331.2303-0221

Abstract

Abstract: In recent years, the classification and segmentation research of point cloud mostly adopts the method of extracting the features of point cloud with multi-level architecture, and obtains relatively stable high-level semantic features. However, the extraction of global features and neighborhood features are insufficient, and the feature fusion of context information is lacking. Therefore, a new LAM-EdgeCNN network is proposed in this paper, which adopts the cascade of edge convolution and attention mechanism to extract multi-level feature information from point clouds and obtain high-level feature information. Firstly, in order to enhance the capture of specific channel features and key spatial points, a lightweight LAM attention mechanism is proposed, which uses CAM feature channel attention to acquire the correlation of each channel and locate the capture of key channel features, so that the network pays more attention to specific channel features to reduce the information dispersion and feature redundancy. Secondly, SAM spatial attention mechanism is introduced to obtain the attention weight of the location information of the point space and increase the granularity of the shallow information. Finally, a combination of attention mechanism and edge convolution EdgeConv is used to enhance context awareness, fully extract and fuse the local features and context features of point cloud, and obtain the downstream task-oriented point cloud features. The model is applied to the public data set, and the experiment shows that the model has good effect and robustness in the tasks of point cloud classification, component segmentation and semantic segmentation.

Key words: point cloud classification, point cloud segmentation, lightweight type, attention mechanism, edge convolution

摘要： 近几年点云的分类分割研究多采用多层次架构提取点云特征的方法，提取到较为稳定的高层语义特征，但是全局特征和邻域特征提取不足并且缺乏对上下文信息的特征融合。因此，提出一种新的LAM-EdgeCNN网络，采用边缘卷积与注意力机制级联的方式对点云进行多层级特征提取，获取高层次特征信息。为了加强对特定通道特征和关键空间点的捕捉，提出一种轻量型LAM注意力机制，使用CAM特征通道注意力获取各通道的关联，定位关键通道特征的捕获，使网络更加关注特定通道特征以减少信息弥散和特征冗余；引入SAM空间注意力机制获取点空间的位置信息的注意力权重，增加获得浅层信息的细粒度。采用注意力机制与边缘卷积EdgeConv相结合的方式，增强上下文感知能力，充分提取和融合点云的局部特征和上下文特征，获得面向下游任务的点云特征。将模型应用于公开数据集，实验表明，模型在点云分类、部件分割、语义分割任务中取得良好效果且具有较好鲁棒性。

关键词: 点云分类, 点云分割, 轻量型, 注意力机制, 边缘卷积

WANG Qiuhong, XU Yang, JIANG Shiyi, XIONG Juju. Research on Point Cloud Classification and Segmentation of Cascaded Edge Convolution and Attention Mechanism[J]. Computer Engineering and Applications, 2024, 60(12): 170-180.

王秋红, 徐杨, 蒋诗怡, 熊举举. 级联边缘卷积与注意力机制的点云分类分割研究[J]. 计算机工程与应用, 2024, 60(12): 170-180.

References

[1] 卢健, 贾旭瑞, 周健, 等. 基于深度学习的三维点云分割综述[J]. 控制与决策, 2023, 38(3): 595-611.
LU J, JIA X R, ZHOU J, et al. A review of deep learning based on 3D point cloud segmentation[J]. Control and Decision, 2023, 38(3): 595-611.
[2] QI C R, SU H, MO K, et al. PointNet: deep learning on point sets for 3D classification and segmentation[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[3] QI C R, LI Y, HAO S, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017.
[4] WANG Y, SUN Y, LIU Z, et al. Dynamic graph CNN for learning on point clouds[J]. ACM Transactions on Graphics (TOG), 2019, 38(5): 1-12.
[5] ZHANG K, HAO M, WANG J, et al. Linked dynamic graph CNN: learning on point cloud via linking hierarchical features[J]. arXiv:1904.10014, 2019.
[6] SUN Q, LIU H, HE J, et al. DAGC: employing dual attention and graph convolution for point cloud based place recognition[C]//International Conference on Multimedia Retrieval, 2020.
[7] GUO M H, CAI J X, LIU Z N, et al. PCT: point cloud transformer[J]. Computational Visual Media, 2021, 7: 187-199.
[8] GAO J, LAN J, WANG B, et al. SDANet: spatial deep attention-based for point cloud classification and segmentation[J]. Machine Learning, 2022, 111(4): 1327-1348.
[9] LIU H, TIAN S. Deep 3D point cloud classification and segmentation network based on GateNet[J]. The Visual Computer, 2023, 40: 971-981.
[10] 苏鸣方, 胡立坤, 黄润辉. 基于上下文注意力的室外点云语义分割方法[J]. 计算机工程, 2023, 49(3): 248-256.
SU M F, HU L K, HUANG R H. Semantic segmentation method for outdoor point clouds based on contextual attention[J]. Computer Engineering, 2023, 49(3): 248-256.
[11] WOO S, PARK J, LEE J Y, et al. Cbam: convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 3-19.
[12] MITTAL P, SHARMA A, SINGH R, et al. Dilated convolution based RCNN using feature fusion for Low-Altitude aerial objects[J]. Expert Systems with Application, 2022, 199: 117106.
[13] TAN M, LE Q. Efficientnet: rethinking model scaling for convolutional neural networks[C]//International Conference on Machine Learning, 2019: 6105-6114.
[14] LIU Y, FAN B, XIANG S, et al. Relation-shape convolutional neural network for point cloud analysis[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 8895-8904.
[15] 沈露, 杨家志, 周国清, 等. 集自注意力与边卷积的点云分类分割模型[J]. 计算机工程与应用, 2023, 59(19): 106-113.
SHEN L, YANG J Z, ZHOU G Q, et al. Point cloud classi-fication segmentation model based on self?attention and edge convolution[J]. Computer Engineering and Applications, 2023, 59 (19): 106-113.
[16] 顾砾, 季怡, 刘纯平. 基于多模态特征融合的三维点云分类方法[J]. 计算机工程, 2021, 47(2): 279-284.
GU L, JI Y, LIU C P. Classification method of three-dimensional point cloud based on multiple modal feature fusion[J]. Computer Engineering, 2021, 47(2): 279-284.
[17] XIE J, XU Y, ZHENG Z, et al. Generative PointNet: deep energy-based learning on unordered point sets for 3D generation, reconstruction and classification[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
[18] LZA B, JFA C, LONG H A. Semantic segmentation of sparse 3D point cloud based on geometrical features for trellis-structured apple orchard-ScienceDirect[J]. Biosystems Engineering, 2020, 196: 46-55.
[19] XU Y, FAN T, XU M, et al. SpiderCNN: deep learning on point sets with parameterized convolutional filters[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 87-102.
[20] LI Y, BU R, SUN M, et al. PointCNN: convolution on X-transformed points[C]//Advances in Neural Information Processing Systems, 2018.
[21] ZHAO H, JIANG L, FU C W, et al. Pointweb: enhancing local neighborhood features for point cloud processing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 5565-5573.
[22] WANG C, SAMARI B, SIDDIQI K. Local spectral graph convolution for point set feature learning[C]//Proceedings of the European Conference on Computer Vision, 2018: 52-66.
[23] ZHANG W, SU S, WANG B, et al. Local k-NNs pattern in omni-direction graph convolution neural network for 3D point clouds[J]. Neurocomputing, 2020, 413: 487-498.
[24] NEZHADARYA E, TAGHAVI E, RAZANI R, et al. Adaptive hierarchical down-sampling for point cloud classification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 12956-12964.
[25] ZHAO H, JIANG L, JIA J, et al. Point transformer[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 16259-16268.
[26] MAO J, WANG X, LI H. Interpolated convolutional networks for 3d point cloud understanding[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019: 1578-1587.
[27] LU Q, CHEN C, XIE W, et al. PointNGCNN: deep convolutional networks on 3D point clouds with neighborhood graph filters[J]. Computers & Graphics, 2020, 86: 42-51.
[28] LIANG Z, YANG M, DENG L, et al. Hierarchical depthwise graph convolutional neural network for 3D semantic segmentation of point clouds[C]//2019 International Conference on Robotics and Automation (ICRA), 2019: 8152-8158.
[29] YANG J, ZHANG Q, NI B, et al. Modeling point clouds with self-attention and gumbel subset sampling[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 3323-3332.
[30] WANG L, HUANG Y, HOU Y, et al. Graph attention convolution for point cloud semantic segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 10296-10305.
[31] XU M, DING R, ZHAO H, et al. PAConv: position adaptive convolution with dynamic kernel assembling on point clouds[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 3173-3182.