Point Cloud Classification Segmentation Combining Inter-Region Structure Relations and Self-Attention Edge Convolution Network

doi:10.3778/j.issn.1002-8331.2304-0141

Abstract

Abstract: A new network framework, ISEC-Net (inter-region structure relations and self-attention edge convolution network), is proposed to address the problem of insufficient capture of context and relational features within a region in point cloud networks for deep learning. The network consists of two modules：IrConv (inter-region convolution) and SaConv (self-attention convolution). The SaConv module can extract finer edge features, while the IrConv can dynamically integrate local structural information into point features and adaptively capture inter-regional relationships. Extensive experiments are conducted on the ModelNet40 and ShapeNet datasets for point cloud classification and part segmentation. The results show that on the ModelNet40 dataset, the overall accuracy (OA) of the ISEC-Net model reaches 93.5%, and the average accuracy (mAcc) reaches 90.7%. On the ShapeNet dataset, the average intersection-over-union (mIoU) reaches 86.1%, and the part segmentation accuracy of guitar, headphone, cup and other parts in the single-class intersection-over-union (IoU) experiment is excellent. This demonstrates that compared with traditional dynamic graph convolutional networks, ISEC-Net can accurately capture the local features and fine structure of point clouds and enhance the aggregation of global features, thus having excellent effectiveness and generalization ability.

Key words: edge convolution, regional context, regional relationship, self-attention, deep learning

摘要： 针对深度学习点云网络中区域内上下文和关系特征捕获不充分问题，提出了一种新的网络框架ISEC-Net（inter-region structure relations and self-attention edge convolution network）。该网络由IrConv（inter-region convolution）模块和SaConv（self-attention convolution）模块组成，SaConv模块可以提取到更细粒的边特征，而IrConv可以动态地将局部结构信息集成到点特征中，然后自适应地捕获区域间关系。在ModelNet40数据集和ShapeNet数据集上分别对点云分类和部件分割进行了大量实验，实验结果表明，在ModelNet40数据集上，ISEC-Net模型的总体精度（OA）达到了93.5%，平均精度（mAcc）达到了90.7%；而在ShapeNet数据集上，平均并交比（mIoU）达到了86.1%，并且在单类并交比（IoU）实验中吉他、耳机、杯子等部件分割精度表现优异，说明ISEC-Net与传统的动态图卷积相比，能够精确地捕捉点云的局部特征和精细结构并加强全局特征的聚合，具有出色的有效性和泛化能力。

关键词: 边卷积, 区域上下文, 区域关系, 自注意力, 深度学习

LYU Zhiwei, YANG Jiazhi, ZHOU Guoqing, SHEN Lu. Point Cloud Classification Segmentation Combining Inter-Region Structure Relations and Self-Attention Edge Convolution Network[J]. Computer Engineering and Applications, 2024, 60(13): 171-179.

吕志玮, 杨家志, 周国清, 沈露. 结合区域结构关系和自注意力的边卷积点云分类分割网络[J]. 计算机工程与应用, 2024, 60(13): 171-179.

References

[1] LI Y, IBANEZ-GUZMAN J. Lidar for autonomous driving: the principles, challenges, and trends for automotive lidar and perception systems[J]. IEEE Signal Processing Magazine, 2020, 37(4): 50-61.
[2] AHMED S M, TAN Y Z, CHEW C M, et al. Edge and corner detection for unorganized 3D point clouds with application to robotic welding[C]//Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018: 7350-7355.
[3] RAHMAN M M, TAN Y, XUE J, et al. Notice of violation of IEEE publication principles: recent advances in 3D object detection in the era of deep neural networks: a survey[J]. IEEE Transactions on Image Processing, 2019, 29: 2947-2962.
[4] MATURANA D, SCHERER S. VoxNet: a 3D convolutional neural network for real-time object recognition[C]//Proceedings of the 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2015: 922-928.
[5] SU H, MAJI S, KALOGERAKIS E, et al. Multi-view convolutional neural networks for 3D shape recognition[C]//Proceedings of the IEEE International Conference on Computer Vision, 2015: 945-953.
[6] QI C R, SU H, MO K, et al. PointNet: deep learning on point sets for 3D classification and segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 652-660.
[7] QI C R, YI L, SU H, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space[C]//Advances in Neural Information Processing Systems, 2017, 30.
[8] WANG Y, SUN Y, LIU Z, et al. Dynamic graph CNN for learning on point clouds[J]. ACM Transactions on Graphics, 2019, 38(5): 1-12.
[9] ZHAO H, JIANG L, FU C W, et al. Pointweb: enhancing local neighborhood features for point cloud processing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 5565-5573.
[10] LANDRIEU L, SIMONOVSKY M. Large-scale point cloud semantic segmentation with superpoint graphs[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 4558-4567.
[11] LIU X, HAN Z, HONG F, et al. LRC-Net: learning discriminative features on point clouds by encoding local region contexts[J]. Computer Aided Geometric Design, 2020, 79: 101859.
[12] CHENG S, CHEN X, HE X, et al. PRA-Net: point relation-aware network for 3D point cloud analysis[J]. IEEE Transactions on Image Processing, 2021, 30: 4436-4448.
[13] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems, 2017, 30.
[14] WU Z, SONG S, KHOSLA A, et al. 3D shapenets: a deep representation for volumetric shapes[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015: 1912-1920.
[15] YI L, KIM V G, CEYLAN D, et al. A scalable active framework for region annotation in 3D shape collections[J]. ACM Transactions on Graphics, 2016, 35(6): 1-12.
[16] LOSHCHILOV I, HUTTER F. SGDR: stochastic gradient descent with warm restarts[J]. arXiv:1608.03983, 2016.
[17] LI J, CHEN B M, LEE G H. SO-Net: self-organizing network for point cloud analysis[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 9397-9406.
[18] 王宝乐, 霍占强. 基于MANet的三维点云特征提取方法研究[J]. 计算机工程与应用, 2022, 58(19): 267-275.
WANG B L, HUO Z Q. Research on feature extraction of 3D point cloud based on MANet[J]. Computer Engineering and Applications, 2022, 58(19): 267-275.
[19] YAN X, ZHENG C, LI Z, et al. PointasNL: robust point clouds processing using nonlocal neural networks with adaptive sampling[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2020.
[20] LI Y, BU R, SUN M, et al. PointCNN: convolution on x-transformed points[C]//Advances in Neural Information Processing Systems, 2018, 31.
[21] ATZMON M, MARON H, LIPMAN Y. Point convolutional neural networks by extension operators[J]. arXiv:1803.10091, 2018.
[22] GAO Y, LIU X, LI J, et al. LFT-Net: local feature transformer network for point clouds analysis[J]. IEEE Transactions on Intelligent Transportation Systems, 2023, 24(2): 2158-2168.
[23] LIU Y, FAN B, XIANG S, et al. Relation-shape convolutional neural network for point cloud analysis[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 8895-8904.
[24] LIN Z H, HUANG S Y, WANG Y C F. Convolution in the cloud: learning deformable kernels in 3D graph convolution networks for point cloud analysis[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 1800-1809.
[25] XIE Z, CHEN J, PENG B. Point clouds learning with attention-based graph convolution networks[J]. Neurocomputing, 2020, 402: 245-255.
[26] HAN X F, JIN Y F, CHENG H X, et al. Dual transformer for point cloud analysis[J]. IEEE Transactions on Multimedia, 2022, 25: 5638-5648.