集双向标准化和稀疏自注意力的点云分割

doi:10.3778/j.issn.1002-8331.2404-0319

摘要/Abstract

摘要： 随着激光雷达点云在智慧城市规划、自动导航与驾驶、同步定位与高精度制图等领域的广泛应用，点云特征提取作为场景感知与解译的基础而备受关注。然而激光雷达点云存在稀疏、无序、非结构等特性以及点云地物目标之间存在遮挡、远稀近密等问题，使得点云特征提取面临严峻挑战。当前深度学习模型在捕获点云几何信息时，难以有效提取海量非结构化数据的细粒度特征；在捕获语义信息时，稀疏数据环境下对局部空间关系的建模准确度不高。针对以上局限性，构建了一种全新的点云语义分割模型PointDTNet，该模型为解决特征编码阶段分组采样点不规则问题，设计出一种点云双向标准化模块，通过点标准化和逆向点标准化操作，以自适应地方式优化分组点和采样点的密度分布，有效阻止了不规则数据特征的传递。为增强模型对空间关系的建模能力，设计了一种作用于点云局部邻域的Transformer稀疏自注意力模块，该模块为平衡计算效率与模型复杂度，仅允许每个点与其邻域内的点进行交互，并通过位置嵌入的方式对局部邻域内点的相对位置进行编码，以自适应地分配每个点的特征权重。实验结果表明，PointDTNet模型在点云语义分割任务中取得了显著效果，其在S3DIS数据集“Area 5”上的OA、mAcc、mIoU分别达到84.68%、65.11%、54.98%；在相同实验条件下，与基模型PointNet++相比OA、mAcc、mIoU分别提高了1.23、7.72、2.07个百分点。

关键词: 激光雷达点云, 语义分割, 双向标准化, 稀疏自注意力, 多特征融合

Abstract: With the wide application of LiDAR point cloud in the fields of smart city, automatic driving, simultaneous localization and mapping, point cloud feature extraction has attracted much attention as the basis of scene perception and interpretation. However, point cloud has the characteristics of sparse, disordered, and unstructured, coupled with the problems of occlusion and varying density of objects, which makes feature extraction facing serious challenges. The current deep learning model is difficult to effectively extract the fine-grained features of massive non-structured data when capturing the geometric information of the point cloud; and it is also limited to accurately modeling the local spatial relationship under the sparse environment when capturing the semantic information. To this end, this paper proposes a novel point cloud semantic segmentation model, PointDTNet, which solves the irregularity of grouping and sampling points in the feature encoding stage by designing a point cloud dual normalization module, DualNorm, to optimize the density of grouping points and sampling points in an adaptive way through the operations of normalization and inverse normalization, effectively preventing the irregular data features from propagating. Furthermore, in order to enhance the model’s ability to model spatial relationships, a Transformer sparse self-attention module acting on the local neighborhood is designed, which, to balance the efficiency and complexity, only allows each point to interact with the points in its neighborhood, and encodes the relative positions of the points by means of positional embedding, so as to assign the feature weights of each point adaptively. The experimental results show that PointDTNet achieves remarkable results in point cloud semantic segmentation task, and its OA, mAcc, and mIoU on the S3DIS dataset “Area 5” reach 84.68%, 65.11%, and 54.98%, respectively. Under the same experimental conditions, the OA, mAcc, and mIoU are improved by 1.23, 7.72 and 2.07 percentage points, respectively, compared with the baseline model PointNet++.

Key words: LiDAR point cloud, semantic segmentation, dual normalization, sparse self-attention, multi-feature fusion

张蕊, 武益超, 黄官龙, 金玮. 集双向标准化和稀疏自注意力的点云分割[J]. 计算机工程与应用, 2025, 61(15): 189-198.

ZHANG Rui, WU Yichao, HUANG Guanlong, JIN Wei. Research on Point Cloud Segmentation Combining Dual Normalization and Sparse Self-Attention[J]. Computer Engineering and Applications, 2025, 61(15): 189-198.

参考文献

[1] MERKLE D, REITERER A. Overview of 3D point cloud annotation and segmentation techniques for smart city applications[C]//Proceedings of SPIE Remote Sensing Technologies and Applications in Urban Environments VII, 2022: 10-18.
[2] ORTEGA S, SANTANA J M, WENDEL J, et al. Generating 3D city models from open LiDAR point clouds: advancing towards smart city applications[M]//Open source geospatial science for urban studies: the value of open geospatial data. Cham: Springer International Publishing, 2021: 97-116.
[3] 陈宇文, 徐照. 建筑全景三维机载雷达点云信息采集方法仿真[J]. 计算机仿真, 2024, 41(4): 270-273.
CHEN Y W, XU Z. Simulation of point cloud information collection method for building panoramic 3D airborne radar[J]. Computer Simulation, 2024, 41(4): 270-273.
[4] ZHOU W, BERRIO J S, WORRALL S, et al. Automated evaluation of semantic segmentation robustness for autonomous driving[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(5): 1951-1963.
[5] DU J H, HUANG X C, XING M Y, et al. Improved 3D semantic segmentation model based on RGB image and LiDAR point cloud fusion for automantic driving[J]. International Journal of Automotive Technology, 2023, 24(3): 787-797.
[6] 杨咏嘉, 钟良琪, 闫胜业. 基于点云的自动驾驶下三维目标检测[J]. 计算机工程与设计, 2024, 45(4): 1093-1099.
YANG Y J, ZHONG L Q, YAN S Y. 3D object detection in automatic driving based on point cloud[J]. Computer Engineering and Design, 2024, 45(4): 1093-1099.
[7] YANG H, CHEN Y Y, LIU J X, et al. A 3D lidar SLAM system based on semantic segmentation for rubber-tapping robot[J]. Forests, 2023, 14(9): 1856.
[8] PU H Y, LUO J, WANG G, et al. Visual SLAM integration with semantic segmentation and deep learning: a review[J]. IEEE Sensors Journal, 2023, 23(19): 22119-22138.
[9] 高一, 王忠立, 王颖博. 基于不确定性建模的3D场景感知方法[J]. 机器人, 2024, 46(4): 450-464.
GAO Y, WANG Z L, WANG Y B. A 3D scene perception method based on uncertainty modeling[J]. Robot, 2024, 46(4): 450-464.
[10] 龚靖渝, 楼雨京, 柳奉奇, 等. 三维场景点云理解与重建技术[J]. 中国图象图形学报, 2023, 28(6): 1741-1766.
GONG J Y, LOU Y J, LIU F Q, et al. Scene point cloud understanding and reconstruction technologies in 3D space[J]. Journal of Image and Graphics, 2023, 28(6): 1741-1766.
[11] SU H, MAJI S, KALOGERAKIS E, et al. Multi-view convolutional neural networks for 3d shape recognition[C]//Proceedings of the IEEE International Conference on Computer Vision, 2015.
[12] MATURANA D, SCHERER S. VoxNet: a 3D convolutional neural network for real-time object recognition[C]//Proceedings of the 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway: IEEE, 2015: 922-928.
[13] MENG H Y, GAO L, LAI Y K, et al. VV-net: voxel VAE net with group convolutions for point cloud segmentation[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2019: 8499-8507.
[14] CHARLES R Q, HAO S, MO K C, et al. PointNet: deep lear-ning on point sets for 3D classification and segmentation[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 77-85.
[15] QI C R, YI L, SU H, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space[C]//Advances in Neural Information Processing Systems, 2017.
[16] ELDAR Y, LINDENBAUM M, PORAT M, et al. The farthest point strategy for progressive image sampling[J]. IEEE Transactions on Image Processing, 1997, 6(9): 1305-1315.
[17] MA X, QIN C, YOU H, et al. Rethinking network design and local geometry in point cloud: a simple residual MLP framework[J]. arXiv:2202.07123, 2022.
[18] COVER T, HART P. Nearest neighbor pattern classification[J]. IEEE Transactions on Information Theory, 1967, 13(1): 21-27.
[19] LIN L, WANG G R, ZHANG R, et al. Deep structured scene parsing by learning with image descriptions[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 2276-2284.
[20] WANG G H, ZHAI Q Y, LIU H. Cross self-attention network for 3D point cloud[J]. Knowledge-Based Systems, 2022, 247: 108769.
[21] ARMENI I, SENER O, ZAMIR A R, et al. 3D semantic parsing of large-scale indoor spaces[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 1534-1543.
[22] QIU S, ANWAR S, BARNES N. Geometric back-projection network for point cloud classification[J]. IEEE Transactions on Multimedia, 2021, 24: 1943-1955.
[23] CHEN C, FRAGONARA L Z, TSOURDOS A. GAPointNet: graph attention based point neural network for exploiting local feature of point cloud[J]. Neurocomputing, 2021, 438: 122-132.
[24] YE X Q, LI J M, HUANG H X, et al. 3D recurrent neural networks with context fusion for point cloud semantic segmentation[C]// Proceedings of the European Conference on Computer Vision. Cham: Springer, 2018: 415-430.
[25] WANG Y, SUN Y B, LIU Z W, et al. Dynamic graph CNN for learning on point clouds[J]. ACM Transactions on Graphics, 2019, 38(5): 1-12.
[26] HUANG Q G, WANG W Y, NEUMANN U. Recurrent slice networks for 3D segmentation of point clouds[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 2626-2635.
[27] MOEIN PEYGHAMBARZADEH S M, AZIZMALAYERI F, KHOTANLOU H, et al. Point-PlaneNet: plane kernel based convolutional neural network for point clouds analysis[J]. Digital Signal Processing, 2020, 98: 102633.
[28] QIU S, ANWAR S, BARNES N. PnP-3D: a plug-and-play for 3D point clouds[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(1): 1312-1319.
[29] DING R Y, YANG J H, JIANG L, et al. DODA: data-oriented sim-to-real domain adaptation for3D semantic segmentation[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer, 2022: 284-303.