Point Cloud Feature Extraction Network Based on 3D Feature Dynamic Fusion

doi:10.3778/j.issn.1002-8331.2209-0004

Abstract

Abstract: To solve the problem that the feature extraction methods currently used for point cloud registration do not adequately extract effective information from point cloud, a point cloud feature extraction network DFRUNet （3D feature dynamic fusion and residual u-net） based on 3D feature dynamic fusion is proposed. The network dynamically fuses the features of encoding and decoding modules through 3DFDF （3D feature dynamic fusion） module to extract sufficient information from the point cloud. Meanwhile, the SE-Res （squeeze and excitation residual） module is used to extract point cloud features. By dynamically adjusting the weights of significant areas, the area features are extracted to improve the quality of the extracted features. Secondly, map the features extracted from the network into high-dimensional space, and complete point cloud registration using RANSAC （random sample consensus） algorithm. The experimental results show that on the 3DMatch dataset, the FMR （feature-match recall） of the algorithm is 96.3%, which is 0.011 higher than that of the classical FCGF algorithm. The registration recall rate is 82.2% and increased by 0.014. This method fully extracts the effective information from point clouds and achieves a higher recall rate, which has reference value for other point cloud registration studies.

Key words: feature extraction, point cloud registration, feature dynamic fusion, deep learning

摘要： 针对目前用于点云配准的点云特征提取方法并未充分提取点云中的有效信息等问题，提出了一种基于3D特征动态融合的点云特征提取网络（3D feature dynamic fusion and residual u-net，DFRUNet）。该网络通过3DFDF（3D feature dynamic fusion）模块将编码和解码模块的特征动态融合，以充分提取点云中的有效信息；同时采用SE-Res（squeeze and excitation residual）模块来提取点云特征，通过动态调整显著区域的权重，对该区域特征进行重点提取，以提高所提取特征的质量。将网络所提取特征映射到高维空间中，采用随机采样一致性（random sample consensus，RANSAC）算法完成点云配准。实验结果表明，在3DMatch数据集上，该算法特征匹配召回率（feature-match recall，FMR）达到了96.3%，相较于经典的FCGF算法提高了0.011。配准召回率（registration recall）达到了82.2%，提高了0.014。该方法充分提取了点云中的有效信息，达到了更高的召回率，对其他点云配准研究具有参考价值。

关键词: 特征提取, 点云配准, 特征动态融合, 深度学习

SUN Liujie, ZHAI Renjie, WANG Wenju, PANG Maoran. Point Cloud Feature Extraction Network Based on 3D Feature Dynamic Fusion[J]. Computer Engineering and Applications, 2023, 59(24): 209-215.

孙刘杰, 翟仁杰, 王文举, 庞茂然. 基于3D特征动态融合的点云特征提取网络[J]. 计算机工程与应用, 2023, 59(24): 209-215.

References

[1] BELLO S A，SHANGSHU Y.Review：deep learning on 3D point clouds[J].Remote Sensing，2020，12（11）：1729.
[2] HUANG X S，MEI G F，ZHANG J，et al.A comprehensive survey on point cloud registration[EB/OL].（2021-05-05）[2022-08-20].https：//doi.org/10.48550/arXiv.2103.02690.
[3] QIN Z，YU H，WANG C，et al.Geometric transformer for fast and robust point cloud registration[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），2022.
[4] BAI X Y，LUO Z X，ZHOU L.PointDSC：robust point cloud registration using deep spatial consistency[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），2021：15854-15864.
[5] LEE J，KIM S，CHO M.Deep hough voting for robust global registration[C]//2021 IEEE/CVF International Conference on Computer Vision（ICCV），2021：15974-15983.
[6] FEDERICO T，SAMUELE S，LUIGI D S.Unique shape context for 3D data description[C]//ACM Workshop on 3D Object Retrieval，2010：57-62.
[7] 谭国威，伍吉仓.基于FPFH特征的三维点云配准方法研究[J].工程勘察，2022，50（4）：52-56.
TAN G W，WU J C.Research on 3D point cloud registration based on FPFH features[J].Geotechnical Investigation & Surveying，2022，50（4）：52-56.
[8] 秦红星，刘镇涛，谭博元.深度学习刚性点云配准前沿进展[J].中国图象图形学报，2022，27（2）：329-348.
QIN H X，LIU Z T，TAN B Y.Review on deep learning rigid point cloud registration[J].Journal of Image and Graphics，2022，27（2）：329-348.
[9] CHEN Z，SUN K，YANG F.SC2-PCR：a second order spatial compatibility for efficient and robust point cloud registration[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），2022.
[10] ZENG A，XIAO J.3DMatch：learning local geometric descriptors from RGB-D reconstructions[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2017：199-208.
[11] AOKI Y，GOFORTH H，SRIVATSAN R A，et al.PointNetLK：robust & efficient point cloud registration using pointNet[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），2019：7156-7165.
[12] BAI X Y，LUO Z X，ZHOU L，et al.D3feat：joint learning of dense detection and description of 3d local features[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），2020：6358-6366.
[13] 刘世光，王海荣，刘锦.快速四点一致性点云粗配准算法[J].山东大学学报（工学版），2019，49（2）：1-7.
LIU S G，WANG H R，LIU J.Fast 4-points congruent sets for coarse registration of 3D point cloud[J].Journal of Shandong University（Engineering Science），2019，49（2）：1-7.
[14] AO S，HU Q，YANG B，et al.SpinNet：learning a general surface descriptor for 3D point cloud registration[C]//Computer Vision and Pattern Recognition，2021：11753-11768.
[15] HORACHE S，DESCHAUD J E，GOULETTE F.3D point cloud registration with multi-scale architecture and self-supervised fine-tuning[EB/OL].（2021-10-14）[2022-08-20].https：//doi.org/10.48550/arXiv.2103.14533.
[16] CHOY C，PARK J，KOLTUN V.Fully convolutional geometric features[C]//2019 IEEE/CVF International Conference on Computer Vision（ICCV），2019：8957-8965.
[17] CHOY C，GWAK J Y，SAVARESE S.4D spatio-temporal convnets：minkowski convolutional neural networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），2020：3070-3079.
[18] RONNEBERGER O，FISCHER P，BROX T.U-Net：convolutional networks for biomedical image segmentation[C]//Medical Image Computing and Computer Assisted Intervention（MICCAI 2015），2015：1520-15285.
[19] OKTAY O，SCHLEMPER J，FOLGOC L L，et al.Attention U-Net：learning where to look for the pancreas[C]//The International Conference on Medical Imaging with Deep Learning，2018.
[20] JIE H，LI S，GANG S，et al.Squeeze-and-excitation networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2020，8：2011-2023.
[21] HASSANIN M，ANWAR S，RADWAN I，et al.Visual attention methods in deep learning：an in-depth survey[EB/OL].（2022-04-21）[2022-08-20].https：//doi.org/10.48550/arXiv.2204.07756.
[22] BRAUWERS G，FRASINCAR F.A general survey on attention mechanisms in deep learning[J].Transactions on Knowledge and Data Engineering，2023，35（4）：3279-3298.
[23] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Advances in Neural Information Processing Systems，2017：6000-6010.
[24] ZAN G，ZHOU C，WEGNER J D，et al.The perfect match：3D point cloud matching with smoothed densities[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），2019：5540-5549.
[25] JL A，QH A，MAA B.GESAC：robust graph enhanced sample consensus for point cloud registration[J].ISPRS Journal of Photogrammetry and Remote Sensing，2020，167：363-374.