Attention Mechanism Based Point Cloud Neural Architecture Search Method

doi:10.3778/j.issn.1002-8331.2102-0037

Abstract

Abstract: To address the problem that the design of a neural network for point cloud data require a lot of manual intervention, a neural network architecture search method based on attention mechanism and point convolution is proposed. Firstly, for the information fusion problem of point clouds of different scales, a multi-scale fusion module based on the attention mechanism is proposed. Secondly, for the problem of point cloud processing efficiency, a feature extraction module based on point convolution is designed as a candidate operation, and the multi-scale fusion module forms a search unit. The neural network cascaded by multiple search cells is formulated as the search space, and the search algorithm based on the differentiable neural network architecture is used to search for the optimal neural network. The experimental results on the public point cloud dataset ModelNet prove that the neural network obtained by this method achieves competitive accuracy and has fewer learnable parameters, and the method greatly reduces the workload of manual intervention. The ablation experimental results on this dataset show that adding the proposed attention mechanism-based multi-scale fusion module to the baseline model improves the accuracy by 1.1?percentage points.

Key words: point convolution, neural architecture search（NAS）, 3D point cloud, multi-scale fusion, attention mechanism

摘要： 为了解决设计面向点云数据的神经网络需要大量人工介入的问题，提出了基于注意力机制和点卷积的神经网络架构搜索方法。针对不同尺度点云的信息融合问题，提出了一种基于注意力机制的多尺度融合模块。针对点云的处理效率问题，设计了基于点卷积的特征提取模块作为候选操作，并与多尺度融合模块组成搜索单元。将多个搜索单元叠加成的神经网络作为搜索空间，并采用基于可微分神经网络架构搜索算法搜索出最优神经网络。在公开点云数据集ModelNet上的实验结果证明，该方法得到的神经网络具有领先的精度，同时具有较少的可学习参数，并且该方法大幅减少了人工介入的工作量。该数据集上的消融实验结果表明，在基线模型中加入提出的基于注意力机制的多尺度融合模块，精度提升了1.1个百分点。

关键词: 点卷积, 神经网络架构搜索（NAS）, 三维点云, 多尺度融合, 注意力机制

TAN Taizhe, HUANG Yongyao, YANG Zhuo, LIU Yang. Attention Mechanism Based Point Cloud Neural Architecture Search Method[J]. Computer Engineering and Applications, 2022, 58(19): 142-151.

谭台哲, 黄永耀, 杨卓, 刘洋. 基于注意力机制的点云神经网络架构搜索方法[J]. 计算机工程与应用, 2022, 58(19): 142-151.

References

[1] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[2] CHEN L C，PAPANDREOU G，KOKKINOS I，et al.Semantic image segmentation with deep convolutional nets，atrous convolution，and fully connected crfs[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，40（4）：834-848.
[3] HE K，GKIOXARI G，DOLLáR P，et al.Mask R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：2961-2969.
[4] WU W，QI Z，LI F.Pointconv：Deep convolutional networks on 3D point clouds[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2019：9621-9630.
[5] QI C R，SU H，MO K，et al.Pointnet：Deep learning on point sets for 3D classification and segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：652-660.
[6] DEFFERRARD M，BRESSON X，VANDERGHEYNST P.Convolutional neural networks on graphs with fast localized spectral filtering[C]//Advances in Neural Information Processing Systems，2016：3844-3852.
[7] MONTI F，BOSCAINI D，MASCI J，et al.Geometric deep learning on graphs and manifolds using mixture model cnns[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：5115-5124.
[8] WU Z，SONG S，KHOSLA A，et al.3D shapenets：A deep representation for volumetric shapes[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2015：1912-1920.
[9] SU H，MAJI S，KALOGERAKIS E，et al.Multi-view convolutional neural networks for 3D shape recognition[C]//Proceedings of the IEEE International Conference on Computer Vision，2015：945-953.
[10] QI C R，YI L，SU H，et al.Pointnet++：Deep hierarchical feature learning on point sets in a metric space[C]//Advances in Neural Information Processing Systems，2017：5099-5108.
[11] LI J，CHEN B M，LEE G H.So-net：Self-organizing network for point cloud analysis[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：9397-9406.
[12] THOMAS H，QI C R，DESCHAUD J E，et al.KpConv：Flexible and deformable convolution for point clouds[J].arXiv：1904.08889，2019.
[13] REAL E，MOORE S，SELLE A，et al.Large-scale evolution of image classifiers[C]//Proceedings of International Conference on Machine Learning，2017：2902-2911.
[14] ZOPH B，LE Q V.Neural architecture search with reinforcement learning[J].arXiv：1611.01578，2016.
[15] ZOPH B，VASUDEVAN V，SHLENS J，et al.Learning transferable architectures for scalable image recognition[J].arXiv：1707.07012，2017.
[16] LIU H，SIMONYAN K，YANG Y.Darts：Differentiable architecture search[J].arXiv：1806.09055，2018.
[17] XIE S，ZHENG H，LIU C，et al.Snas：stochastic neural architecture search[J].arXiv：1812.09926，2018.
[18] LI G，QIAN G，DELGADILLO I C，et al.SGAS：Sequential greedy architecture search[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition，2020.
[19] RIEGLER G，ULUSOYS A O，GEIGER A.Octnet：Learning deep 3D representations at high resolutions[J].arXiv：1611. 05009，2016.
[20] ZHANG Z，HUA B S，YEUNG S K.Shellnet：Efficient point cloud convolutional neural networks using concentric shells statistics[J].arXiv：1908.06295，2019.
[21] CAI H，ZHU L，HAN S.Proxylessnas：Direct neural architecture search on target task and hardware[J].arXiv：1812.00332，2018.
[22] BROCK A，LIM T，RITCHIE J M，et al.Smash：One-shot model architecture search through hypernetworks[J].arXiv：1708.05344，2017.
[23] WANG X，GIRSHICK R，GUPTA A，et al.Non-local neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：7794-7803.
[24] WOO S，PARK J，LEE J Y，et al.CBAM：Convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision，2018：3-19.
[25] ELDAR Y，LINDENBAUM M，PORAT M，et al.The farthest point strategy for progressive image sampling[J].IEEE Transactions on Image Processing，1997，6（9）：1305-1315.
[26] SIMONOVSKY M，KOMODAKIS N.Dynamic edge-conditioned filters in convolutional neural networks on graphs[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：3693-3702.
[27] MATURANA D，SCHERER S.VoxNet：A 3D convolutional neural network for real-time object recognition[C]//Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems，2015：922-928.
[28] QI C R，SU H，NIE?NER M，et al.Volumetric and multi-view cnns for object classification on 3D data[C]//Proceedings of the IEEE Computer Vision and Pattern Recognition，2016：5648-5656.
[29] SU H，MAJI S，KALOGERAKIS E，et al.Multi-view convolutional neural networks for 3D shape recognition[C]//Proceedings of the IEEE International Conference on Computer Vision，2015：945-953.
[30] BEN-SHABAT Y，LINDENBAUM M，FISCHER A. 3DMFV：Three-dimensional point cloud classification in real-time using convolutional neural networks[J].IEEE Robotics and Automation Letters，2018，3（4）：3145-3152.
[31] WANG C，BABAK S，KALEEM S.Local spectral graph convolution for point set feature learning[C]//Proceedings of the European Conference on Computer Vision，2018：52-66.