Computer Engineering and Applications ›› 2022, Vol. 58 ›› Issue (23): 254-260.DOI: 10.3778/j.issn.1002-8331.2106-0397

• Graphics and Image Processing • Previous Articles     Next Articles

Research on Attention Mechanism for Point Cloud Data

SUN Yijun, HU Hui, LI Ziyue, CHEN Yang, WU Shaoyi   

  1. School of Information Engineering, East China Jiaotong University, Nanchang 330013, China
  • Online:2022-12-01 Published:2022-12-01

适用于点云数据的注意力机制研究

孙一珺,胡辉,李子钥,陈阳,吴少奕   

  1. 华东交通大学 信息工程学院,南昌 330013

Abstract: As a plug and play method to improve the performance of network feature extraction, attention mechanism is widely used in natural language processing and image recognition. However, due to the irregularity and disorder of point cloud data, attention mechanism can not be directly applied to the field of point cloud. This paper proposes an attention mechanism suitable for point cloud. PointNet networks are used as the backbone network of point cloud feature extraction. Through the multi-angle pooling of point cloud data, the shared weight MLP(multi-layer perceptron) is used to obtain adaptive attention weight, and multiplied with the original feature to optimize input feature, so as to improve network performance and realize the application of attention mechanism in the field of point cloud. The attention mechanism designed in this paper can help the OA(overall accuracy) of PointNet(vanilla) and PointNet to improve 0.89 and 0.40?percentage points respectively in ModelNet40 classification task. In ShapeNet partial segmentation task, the mIoU(mean Intersection over Union) of PointNet can increase 1.38?percentage points. In the KITTI 3D detection task, the AP(average precision) of Frustum-PointNet in pedestrian and cyclist detection has been significantly improved. Experimental results show that the designed attention mechanism is effective and lightweight in multiple point cloud processing tasks.

Key words: deep learning, three-dimensional point cloud, attention mechanism, classification, segmentation, detection

摘要: 注意力机制作为一种即插即用的有效提高网络特征提取性能的手段,在自然语言处理、图像识别领域有着广泛的应用。然而由于点云数据的不规则性与无序性,使得注意力机制无法直接应用于点云领域。提出适用于点云的注意力机制,以PointNet类网络作为点云特征提取的骨干网络,通过对点云数据进行多角度池化,采用共享权重的多层感知器获取自适应注意力权重,并与原特征相乘以实现输入特征优化,从而提升网络性能,实现注意力机制在点云领域的应用。设计的适用于点云的注意力机制在ModelNet40分类任务上,帮助PointNet(vanilla)和PointNet网络的分类准确率分别提升0.89和0.40个百分点;在ShapeNet零件分割任务上,帮助PointNet网络的平均交并比提升1.38个百分点;在KITTI三维检测任务上,帮助基于视锥体法的融合检测Frustum-PointNet网络在行人和骑行者两种小物体的平均精度也取得了可观的提升。实验结果表明所设计的注意力机制在多种点云处理任务的有效性和轻量级特点。

关键词: 深度学习, 三维点云, 注意力机制, 分类, 分割, 检测