MF-Net：Multi-Scale Feature Point Cloud Completion Network Combined with Residual Network

doi:10.3778/j.issn.1002-8331.2301-0188

Abstract

Abstract: Aiming at the problem of semantic information loss caused by the current point cloud completion network focusing only on global features, a point cloud completion network based on multi-scale feature extraction of residual network is proposed. The network adopts an end-to-end idea. Firstly, to avoid the problem of incomplete single feature, the original input is sampled into three different scale point clouds. Then, the global features of the low-resolution point cloud extracted by different methods and the local features of the original point cloud are fused recursively in a cascade way to form feature vectors, and are put into the fully connected network to realize the prediction of coarse point cloud. Next, the splicing original point cloud and coarse point cloud are sent into the fine reconstruction unit, and then the attention mechanism is integrated into the fine reconstruction unit, and the residual network is used to complete from rough to fine. Finally, the joint loss function among coarse point clouds, dense point clouds and ground truth point clouds is calculated to improve the completion performance. Experiments on ShapeNet and KITTI data sets show that the proposed method has good completion effect on the incomplete point cloud, both in qualitative and quantitative comparison, and also show its generalization ability.

Key words: 3D point cloud, point cloud completion, multi-scale feature, attention mechanism, residual network

摘要： 针对目前点云补全网络只关注全局特征造成的语义信息丢失问题，提出了一个基于残差网络的多尺度特征提取的点云补全网络。网络采用端到端的思想，为避免单一特征不全面问题，将原始输入采样为三种不同尺度的点云；利用级联方式递归式融合不同方法提取的低分辨率点云的全局特征和原始点云的局部特征，形成特征向量并输入全连接网络，实现粗点云的预测；将拼接后的原始点云和粗点云送入精细重构单元，再在精细重构单元中融合注意力机制并利用残差网络进行由粗略到精细的补全；通过计算粗点云、稠密点云与真实点云之间的联合损失函数以提高补全性能。在ShapeNet数据集和KITTI数据集上的实验证明，无论是定性比较还是定量比较，提出的方法对残缺点云均具有较好的补全效果，同时也体现了该方法具有泛化能力。

关键词: 3D点云, 点云补全, 多尺度特征, 注意力机制, 残差网络

QIU Yunfei, ZHAO Jing, FANG Li. MF-Net：Multi-Scale Feature Point Cloud Completion Network Combined with Residual Network[J]. Computer Engineering and Applications, 2023, 59(22): 202-212.

邱云飞, 赵静, 方立. MF-Net：结合残差网络的多尺度特征点云补全网络[J]. 计算机工程与应用, 2023, 59(22): 202-212.

References

[1] LI J，HENG S Z，SHAO S S，et al.PointGroup：dual-set point grouping for 3D instance segmentation[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：4867-4876.
[2] 李维刚，樊响，梅洋，等.基于车载激光点云的道路标线提取方法[J].中国激光，2022，49（7）：0710001.
LI W G，FAN X，MEI Y，et al.Road marking extraction method based on vehicle laser point cloud[J].Chinese Journal of Lasers，2022 49（7）：0710001.
[3] 张磊，徐孝彬，曹晨飞，等.基于动态特征剔除的图像与点云融合的机器人位姿估计方法[J].中国激光，2022，49（6）：0610001.
ZHANG L，XU X B，CAO C F，et al.Robot pose estimation method based on image and point cloud fusion with dynamic feature elimination[J].Chinese Journal of Lasers，2022，49（6）：0610001.
[4] LIU H，JIA J，GONG N Z.PointGuard：provably robust 3D point cloud classification[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition，Kuala Lumpur，Dec 18-20，2021：6186-6195.
[5] 景庄伟，管海燕，臧玉府，等.基于深度学习的点云语义分割研究综述[J].计算机科学与探索，2021，15（1）：1-26.
JING Z W，GUAN H Y，ZANG Y F，et al.Survey of point cloud semantic segmentation based on deep learning[J].Journal of Frontiers of Computer Science and Technology，2021，15（1）：1-26.
[6] 兰红，陈浩，张蒲芬.集图卷积和三维方向卷积的点云分类分割模型[J].计算机工程与应用，2023，59（8）：182-191.
LAN H，CHEN H，ZHANG P F.Point cloud classification and segmentation model based on graph convolution and 3D direction convolution[J].Computer Engineering and Applications，2023，59（8）：182-191.
[7] GUO Y，WANG H，HU Q，et al.Deep learning for 3D point clouds：a survey[J].arXiv：1912.12033，2019.
[8] BERGER M，TAGLIASACCHI A，SEVERSKY L，et al.State of the art in surface reconstruction from point clouds[C]//Proceedings of the 35th Annual Conference of the European Association for Computer Graphics，State of the Art Reports，Strasbourg，Apr 7-11，2014：161-185.
[9] THRUN S，WEGBREIT B.Shape from symmetry[C]//Proceedings of the 10th IEEE International Conference on Computer Vision，2005：1824-1831.
[10] LI Y，DAI A，GUIBAS L，et al.Database-assisted object retrieval for real-time 3D reconstruction[J].Computer Graphics Forum，2015，34（2）：435-446.
[11] PAULY M，MITRA N，GIESEN J，et al.Example-based 3D scan completion[C]//Proceedings of the 3rd Eurographics Symposium on Geometry Processing，Vienna，Jul 4-6，2005：23-32.
[12] YANG Y Q，FENG C，SHEN Y，et al.FoldingNet：point cloud auto-encoder via deep grid deformation[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition，2018：206-215.
[13] YUAN W，KHOT T，HELD D，et al.PCN：point completion network[C]//Proceedings of the 2018 International Conference on 3D Vision，Verona，Sep 5-8，2018：728-737.
[14] WEN X，XIANG P，HAN Z Z，et al.PMP-Net：point cloud completion by learning multi-step point moving paths[C]//Proceedings of the 2021 IEEE Conference on Computer Vision and Pattern Recognition，Nashville，Jun 20-25，2021.
[15] LU D，XIE Q，GAO K，et al.3DCTN：3D convolution-transformer network for point cloud classification[J].IEEE Transactions on Intelligent Transportation Systems，2022，23（12）：24854-24865.
[16] QI C R，YI L，SU H，et al.PointNet++：deep hierarchical feature learning on point sets in a metric space[C]//Proceedings of the 2008 IEEE International Conference on Neural Information Processing Systems，Long Beach，2017：5105-5114.
[17] WANG Y，SUN Y B，LIU Z W，et al.Dynamic graph CNN for learning on point clouds[J].ACM Transactions on Graphics，2019，38（5）：1-12.
[18] WANG N，ZHANG Y，LI Z，et al.Pixel2Mesh：generating 3D mesh models from single RGB images[C]//Proceedings of the 15th European Conference on Computer Vision.Cham：Springer，2018：55-71.
[19] YIN Z，ONCEL T.VoxelNet：end-to-end learning for point cloud based 3D object detection[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition，Salt Lake City，2018：4490-4499.
[20] XIE H Z，YAO H X，ZHOU S C，et al.GRNet：gridding residual network for dense point cloud completion[C]//Proceedings of the 16th European Conference on Computer Vision.Cham：Springer，2020：365-381.
[21] QI C R，SU H，Mo K，et al.PointNet：deep learning on point sets for 3D classification and segmentation[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，Jul 21-26，2017：652-660.
[22] TCHAPMI L P，KOSARAJU V，REZATOFIGHI H，et al.TopNet：structural point cloud decoder[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition，Long Beach，Jun 15-21，2019：383-392.
[23] HUANG Z，YU Y，XU J，et al.PF-Net：point fractal network for 3D point cloud completion[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition，Seattle，Jun 13-19，2020：7659-7667.
[24] GOYAL P，FERRARA E.Graph embedding techniques，applications，and performance：a survey[J].Knowledge-Based Systems，2017，151：78-94.
[25] SHEN Y R，FENG C，YANAG Y Y，et al.Mining point cloud local structures by kernel correlation and graph pooling[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2018：4548-4557.
[26] RONNEBERGER O，FISCHER P，BROX T.U-Net：convolutional networks for biomedical image segmentation[C]//LNCS 9351：Proceedings of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention，Munich，Oct 5-9，2015.Cham：Springer，2015：234-241.
[27] WANG X，ANG JR M H，LEE G H.Cascaded refinement network for point cloud completion[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition，Seattle，Jun 14-19，2020：790-799.
[28] KINGMA D P，JIMMY B.Adam：a method for stochastic optimization[J].arXiv：1412.6980，2014.
[29] FAN H Q，SU H，GUIBAS L.A point set generation network for 3D object reconstruction from a single image[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，2017：2463-2471.
[30] TATARCHENKO M，RICHTER S R，RANFTL R，et al.What do single-view 3D reconstruction networks learn?[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：3405-3414.