Multistage Refinement of Deconvolution Point Cloud Complementation Network

doi:10.3778/j.issn.1002-8331.2205-0500

Abstract

Abstract: Point clouds, as an important representation of 3D objects, have been widely used in fields such as machine vision and computer vision. However, the 3D point cloud shape of a target object acquired by a sensor is often defective due to the influence of scanning equipment and complex external environmental factors. Currently, most shape complementation networks based on point cloud representations usually predict the complete point cloud shape by extracting global features through a simple encoder-decoder（ECD） structure. This not only ignores the importance of local geometric information, but also destroys the geometric structure of the original input point cloud and causes displacement loss. To solve the above problem, a multi-stage refined deconvolution point cloud complementation network is proposed. The network model mainly consists of two parts: the generation of rough complete point clouds and the smoothing of complete point clouds. Missing regions of only part of the point cloud are predicted by an encoder-decoder network containing the deconvolution operation, and the rough complete point cloud is obtained by stitching it with the input. A uniformly distributed complete point cloud model is generated by passing the coarse complete point cloud through a second encoder-decoder network based on an attention mechanism. Extensive experiments on the ShapeNet-Part dataset show that the proposed network model achieves more desirable results in both quantitative and visualization experimental results.

Key words: point cloud, shape completion, 3D point cloud completion, attention mechanism, deconvolution

摘要： 点云作为三维对象的一种重要表示方法，已被广泛应用于机器视觉和计算机视觉等领域。然而由于扫描设备和外界复杂环境因素的影响，通过传感器采集到的目标物体的3D点云形状往往是有缺陷的。目前，大多数基于点云表示的形状补全网络通常都是经过一个简单的编码器-解码器（encoder-decoder）结构通过提取全局特征来预测完整点云形状。这不仅忽略了局部几何信息的重要性，同时会破坏原始输入点云的几何结构，造成位移损失。为了解决上述问题，提出了一个多级精细化反卷积点云补全网络。网络模型主要包含两部分：粗糙完整点云的生成和完整点云的平滑。通过一个包含反卷积操作的编码器-解码器网络只预测部分点云的缺失区域，将其与输入拼接得到粗糙的完整点云。通过将粗糙完整点云经过第二个基于注意力机制的编码器-解码器网络生成一个分布均匀的完整点云模型。在ShapeNet-Part数据集上进行的大量实验表明：不论是在量化还是可视化实验结果中，提出的网络模型都能取得更加理想的结果。

关键词: 点云, 形状补全, 3D点云补全, 注意力机制, 反卷积

LU Chunmei , YANG Zhijing. Multistage Refinement of Deconvolution Point Cloud Complementation Network[J]. Computer Engineering and Applications, 2023, 59(17): 242-249.

陆春媚, 杨志景. 多级精细化反卷积点云补全网络[J]. 计算机工程与应用, 2023, 59(17): 242-249.

References

[1] GOYAL A，LAW H，LIU B，et al.Revisiting point cloud shape classification with a simple and effective baseline[C]//Proceedings of the IEEE/CVF Conference on Machine Learning，December 18-20，2021，Kuala Lumpur，Malaysia.New York：IEEE，2021：3809-3820.
[2] LIU H，JIA J，GONG N Z.PointGuard：provably robust 3D point cloud classification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，December 18-20，2021，Kuala Lumpur，Malaysia.New York：IEEE，2021：6186-6195.
[3] 罗岩，冯天波，邵洁，等.基于注意力及视觉Transformer的野外人脸表情识别[J].计算机工程与应用，2022，58（10）：200-207.
LUO Y，FENG T B，SHAO J，et al.Facial expression recognition in wild based on attention and vision Transformer[J].Computer Engineering and Applications，2022，58（10）：200-207.
[4] QI C R，ZHOU Y，NAJIBI M，et al.Offboard 3D object detection from point cloud sequences[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，December 18-20，2021，Kuala Lumpur，Malaysia.New York：IEEE，2021：6134-6144.
[5] 蔡正奕，赵杰煜，朱峰.融合图像特征的单阶段点云目标检测[J].计算机工程与应用，2023，59（9）：140-149.
CAI Z Y，ZHAO J Y，ZHU F.Single-stage object detection with fusion of point cloud and image feature[J].Computer Engineering and Applications，2023，59（9）：140-149.
[6] 兰红，陈浩，张蒲芬.集图卷积和三维方向卷积的点云分类分割模型[J].计算机工程与应用，2023，59（8）：182-191.
LAN H，CHEN H，ZHANG P F.Point cloud classification and segmentation model based on graph convolution and 3D direction convolution[J].Computer Engineering and Applications，2023，59（8）：182-191.
[7] HU W，FU Z，GUO Z.Local frequency interpretation and non-local self-similarity on graph for point cloud inpainting[J].IEEE Transactions on Image Processing，2019，28（8）：4087-4100.
[8] SUNG M，KIM V G，ANGST R，et al.Data-driven structural priors for shape completion[J].ACM Transactions on Graphics（TOG），2015，34（6）：1-11.
[9] MARTINOVIC A，VAN GOOL L.Bayesian grammar learning for inverse procedural modeling[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，Portland，OR，United States.New York：IEEE，2013：201-208.
[10] SHEN C H，FU H，CHEN K，et al.Structure recovery by part assembly[J].ACM Transactions on Graphics（TOG），2012，31（6）：1-11.
[11] TCHAPMI L P，KOSARAJU V，REZATOFIGHI H，et al.Topnet：structural point cloud decoder[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，June 15-21，2019，Long Beach，CA，United States.New York：IEEE，2019：383-392.
[12] WANG X，ANG JR M H，LEE G H.Cascaded refinement network for point cloud completion[C]//Proceedings of the IEEE /CVF Conference on Computer Vision and Pattern Recognition，June 14-19，2020，Seattle，United States.New York：IEEE，2020：790-799.
[13] QI C R，SU H，MO K，et al.Pointnet：deep learning on point sets for 3D classification and segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，July 21-26，2017，Honolulu，HI，United States.New York：IEEE，2017：652-660.
[14] QI C R，YI L，SU H，et al.PointNet++：deep hierarchical feature learning on point sets in a metric space[C]// Advances in Neural Information Processing Systems，2017.
[15] YUAN W，KHOT T，HELD D，et al.PCN：point completion network[C]//2018 International Conference on 3D Vision，September 5-8，2018，Verona，Italy.New York：IEEE，2018：728-737.
[16] HUANG Z，YU Y，XU J，et al.PF-Net：point fractal network for 3D point cloud completion[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，June 14-19，2020，Seattle，United States.New York：IEEE，2020：7662-7670.
[17] GUO M H，CAI J X，LIU Z N，et al.PCT：point cloud transformer[J].Computational Visual Media，2021，7（2）：187-199.
[18] LU D，XIE Q，GAO K，et al.3DCTN：3D convolution-transformer network for point cloud classification[J].IEEE Transactions on Intelligent Transportation Systems，2022，23（12）：24854-24865.
[19] CARION N，MASSA F，SYNNAEVE G，et al.End-to-end object detection with transformers[C]//European Conference on Computer Vision，August 23-28，2020，Glasgow，United Kingdom.Germany：IEEE，2020：213-229.
[20] WANG Z，WANG Y，AN L，et al.Local transformer network on 3D point cloud semantic segmentation[J].Information，2022，13（4）：198.
[21] LAI X，LIU J，JIANG L，et al.Stratified transformer for 3D point cloud segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2022：8500-8509.
[22] YANG Y，FENG C，SHEN Y，et al.Foldingnet：point cloud auto-encoder via deep grid deformation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：206-215.
[23] RUBNER Y，TOMASI C，GUIBAS L J.The earth mover’s distance as a metric for image retrieval[J].International Journal of Computer Vision，2000，40（2）：99-121.
[24] BORGEFORS G.Hierarchical chamfer matching：a parametric edge matching algorithm[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，1988，10（6）：849-865.
[25] WU Z R，SONG S R，KHOSLA A，et al.3D ShapeNets a deep representation for volumetric shapes[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，June 7-13，2015，Boston，MA，United States.New York：IEEE，2015：1912-1920.
[26] ZHAO Y，BIRDAL T，DENG H，et al.3D point capsule networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，June 15-21，2019，Long Beach，CA，United States.New York：IEEE，2019：1009-1018.
[27] LIU M，SHENG L，YANG S，et al.Morphing and sampling network for dense point cloud completion[C]//Proceedings of the AAAI Conference on Artificial Intelligence，February 7-20，2020，Corvallis，Oregon，United States.New York：IEEE，2020：11596-11603.