Computer Engineering and Applications ›› 2023, Vol. 59 ›› Issue (21): 159-166.DOI: 10.3778/j.issn.1002-8331.2303-0188

• Graphics and Image Processing • Previous Articles     Next Articles

Task-Driven Point Cloud Downsampling Method Based on Light-Transformer

YANG Yakun, WANG Anhong, FENG Zewen   

  1. School of Electronic Information Engineering, Taiyuan University of Science and Technology, Taiyuan 030024, China
  • Online:2023-11-01 Published:2023-11-01

任务驱动的轻量Transformer点云下采样方法

杨亚坤,王安红,冯泽文   

  1. 太原科技大学 电子信息工程学院,太原 030024

Abstract: Point cloud is one of the most important 3D data format that objectively represents the real world. However, the huge amount of data of point cloud limits its wider application. In order to simplify the point cloud and improve the efficiency of downstream applications, a task-driven point cloud downsampling method based on lightweight Transformer is proposed. The network includes feature extraction module and soft sampling module. In the feature extraction module, the state-of-the-art Transformer model is used to learn point cloud features, which is designed as a lightweight structure considering computing and storage resources. In the soft sampling module, MLP and Gumbel-Softmax are used to simulate the actual sampling process and obtain the downsampled point cloud. To make the downsampled point cloud suitable for subsequent application tasks, a joint loss function including task loss, sampling loss, and constraint loss is constructed for end-to-end training. To simplify training and facilitate practical application, on the basis of a task-driven point cloud downsampling method, a multi-rate downsampling method is also proposed, which uses a progressive structure and combines multiple sets of sampling losses to obtain point clouds at multiple sampling rates with a single model. Experiments of point cloud classification task on ModelNet40 dataset and reconstruction task on ShapeNetCore55 dataset show that the proposed method not only simplify the point cloud but also ensuring good task performance, especially when the number of downsampled points is small, the task performance is higher than that of similar methods.

Key words: 3D point cloud, downsampling, task-driven, multi-rate, Transformer

摘要: 点云是一种重要的三维数据格式,能直观地描绘真实世界,然而点云的巨大数据量限制了其更加广泛的应用。为了简化点云并提高下游应用效率,提出了一种基于轻量级Transformer的任务驱动点云下采样方法。该网络包括特征提取模块和软采样模块,在特征提取模块中采用最先进的Transformer模型学习点云特征,考虑到计算和存储资源有限,将其设计为轻量化结构;在软采样模块中利用MLP和Gumbel-Softmax来模拟实际采样过程,得到下采样点云。为使采样点云适合后续应用任务,构造了一个包含任务损失、采样损失和约束损失的联合损失函数用于网络端到端训练。此外,为简化训练并方便实际应用,在基于轻量化Transformer的任务驱动点云下采样网络的基础上,还提出了多倍率下采样方法,它采用渐进式结构,结合多组采样损失,实现一个模型得到多个采样率下的点云。通过在ModelNet40和ShapeNetCore55数据集上进行点云分类任务和重建任务实验表明,所提方法在简化点云数量的同时,分类精度和重建精度得到良好保证,尤其是下采样点数较少时,相比于同类算法,任务性能更高。

关键词: 三维点云, 下采样, 任务驱动, 多倍率, Transformer