基于Mamba的轻量级三维点云实例分割算法

doi:10.3778/j.issn.1002-8331.2405-0054

摘要/Abstract

摘要： 针对三维点云实例分割中的特征提取能力的不足、实例边缘的模糊性，以及在复杂场景中的实例识别困难的问题，提出了一种基于Mamba的轻量级三维点云实例分割算法。利用稀疏3D U-Net高效地对点云数据进行特征提取。为了增强模型对复杂场景的学习能力，进一步采用最远距离采样和球形查询聚类特征在节省计算量同时对信息进行二次提炼，这些处理后的特征利用混合专家模型最有效分配给不同专家网络，最后送入高效SSM模块，实现实例的精确查询。在ScanNetV2数据集上，取得了52.8%的mAP，并且在S3DIS等点云室内场景数据集上表现出优势，运行速率达到210?ms，实现了轻量级的优化。

关键词: 点云实例分割, 最远距离采样, 球查询

Abstract: Aiming at the shortage of feature extraction ability, the ambiguity of instance edge and the difficulty of instance recognition in complex scenes, a lightweight 3D point cloud instance segmentation algorithm based on Mamba is proposed. Firstly, sparse 3D U-Net is used to extract features from point cloud data efficiently. Secondly, in order to enhance the learning ability of the model for complex scenes, the farthest distance sampling and spherical query clustering features are further adopted to save computation and refine the information twice. These processed features are most effectively allocated to different expert networks using the hybrid expert model, and finally sent into the efficient SSM module to achieve accurate query of instances. Finally, on the ScanNetV2 dataset, the mAP of 52.8% is obtained, and the performance advantage is achieved on the S3DIS and other point cloud indoor scene data sets, and the running time is 210?ms, which helps to achieve lightweight optimization.

Key words: point cloud instance segmentation, farthest distance sampling, ball query

崔丽群, 郝思雅, 栾五洋. 基于Mamba的轻量级三维点云实例分割算法[J]. 计算机工程与应用, 2025, 61(8): 194-203.

CUI Liqun, HAO Siya, LUAN Wuyang. Lightweight 3D Point Cloud Instance Segmentation Algorithm Based on Mamba[J]. Computer Engineering and Applications, 2025, 61(8): 194-203.

参考文献

[1] 刘苏毅, 迟剑宁, 吴成东, 等. 基于递归切片网络的三维点云语义分割与实例分割[J]. 中国图象图形学报, 2023, 28(7): 2135-2150.
LIU S Y, CHI J N, WU C D, et al. Semantic segmentation and case segmentation of three-dimensional point cloud based on recursive slicing network[J]. Journal of Image and Graphics, 2023, 28(7): 2135-2150.
[2] 徐昕. 面向室内场景的三维实例分割方法研究[D]. 大庆: 东北石油大学, 2023.
XU X. Research on 3D case segmentation method for indoor scene[D]. Daqing: Northeast Petroleum University, 2023.
[3] 卢健, 贾旭瑞, 周健, 等. 基于深度学习的三维点云分割综述[J]. 控制与决策, 2023, 38(3): 595-611.
LU J, JIA X R, ZHOU J, et al. Review of 3D point cloud segmentation based on deep learning[J]. Control and Decision, 2023, 38(3): 595-611.
[4] 刘鹏. 基于类别信息和方向信息的多任务三维点云实例分割算法研究[D]. 武汉: 华中科技大学, 2022.
LIU P. Research on multi-task 3D point cloud instance segmentation algorithm based on category information and direction information[D]. Wuhan: Huazhong University of Science and Technology, 2022.
[5] 张扬. 三维场景融合补全实例分割的研究[D]. 成都: 电子科技大学, 2022.
ZHANG Y. Research on case segmentation of three-dimensional scene fusion and completion[D]. Chengdu: University of Electronic Science and Technology of China, 2022.
[6] 刘长吉, 郝志成, 杨锦程, 等. 基于实例分割的目标三维位置估计方法[J]. 液晶与显示, 2021, 36(11): 1535-1544.
LIU C J, HAO Z C, YANG J C, et al. 3D target position estimation method based on case segmentation[J]. Chinese Journal of Liquid Crystals and Displays, 2021, 36(11): 1535-1544.
[7] 李东威. 基于深度学习的三维实例分割技术研究[D]. 长沙: 湖南大学, 2021.
LI D W. Research on 3D instance segmentation technology based on deep learning[D]. Changsha: Hunan University, 2021.
[8] 黄漫, 黄勃, 高永彬. 引入深度补全与实例分割的三维目标检测[J]. 传感器与微系统, 2021, 40(1): 129-132.
HUANG M, HUANG B, GAO Y B. 3D object detection with deep completion and case segmentation[J]. Sensors & Microsystems, 2021, 40(1): 129-132.
[9] 顾军华, 李炜, 董永峰. 基于点云数据的分割方法综述[J]. 燕山大学学报, 2020, 44(2): 125-137.
GU J H, LI W, DONG Y F. Review of segmentation methods based on point cloud data[J]. Journal of Yanshan University, 2020, 44(2): 125-137.
[10] WANG W Y, YU R, HUANG Q G. SGPN: similarity group proposal network for 3D point cloud instance segmentation[J]. arXiv:1711.08588, 2017.
[11] YANG B, WANG J, CLARK R, et al. Learning object bounding boxes for 3D instance segmentation on point clouds[C]//Proceedings of the 33rd Conference on Neural Information Processing Systems, 2019: 1-10.
[12] HAN L, ZHENG T, LAN X, et al. OccuSeg: occupancy-aware 3D instance segmentation[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 2937-2946.
[13] JIANG L, ZHAO H, SHI S, et al. PointGroup: dual-set point grouping for 3D instance segmentation[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2020: 4866-4875.
[14] CHEN S Y, FANG J M, ZHANG Q, et al. Hierarchical aggregation for 3D instance segmentation[J]. arXiv:2108. 02350, 2021.
[15] HE T, SHEN C, HENGEL V D A. DyCO3D: robust instance segmentation of 3D point clouds through dynamic convolution[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2021: 354-363.
[16] GRAHAM B, ENGELCKE M, MAATEN V D L. 3D semantic segmentation with submanifold sparse convolutional networks[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2018: 9224-9232.
[17] JACOBS R A, JORDAN M I, NOWLAN S J, et al. Adaptive mixtures of local experts[J]. Neural Computation, 1991, 3(1): 79-87.
[18] GU A, DAO T. Mamba: linear-time sequence modeling with selective state spaces[J]. arXiv:2312.00752, 2023.
[19] RUBY U, YENDAPALLI V. Binary cross entropy with deep learning technique for Image classification[J]. International Journal of Advanced Trends in Computer Science and Engineering, 2020, 9(4): 5393-5397.
[20] LI X, SUN X, MENG Y, et al. Dice loss for data-imbalanced NLP tasks[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2020: 465-476.
[21] DAI A, CHANG A X, SAVVA M, et al. ScanNet: richly-annotated 3D reconstructions of indoor scenes[C]//Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2432-2443.
[22] THABET A, ALWASSEL H, GHANEM B. Self-supervised learning of local features in 3D point clouds[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2020: 4048-4052.