Deep Ensemble Learning for Diversified 3D Model Classification

doi:10.3778/j.issn.1002-8331.2111-0315

Abstract

Abstract: The current 3D model classification methods based on deep learning are mostly worked on specific tasks. Considering that these methods do not perform well and lack of universality in the diversified 3D model classification tasks, it proposes a general end-to-end deep ensemble learning E2E-DEL, which consists of a set of several base learners and an ensemble learner so as to automatically learn composite features of complex 3D models. And it uses a hierarchical iterative learning strategy to comprehensively consider the feature learning ability of different levels of networks and balance the learning effect of base learners and the ensemble learner. So the model can be adaptive in the diversified 3D model classification tasks. Based on this, it proposes a multi-view deep ensemble learning：MV-DEL, which can be used in general, fine-grained, and zero-shot 3D model classification tasks. Experiments on several public datasets show that the proposed method has good generalization and universality.

Key words: deep learning, deep ensemble learning, 3D model classification

摘要： 基于深度学习的三维模型分类方法大都面向特定的具体任务，在面向三维模型多样化分类任务时表现不佳，泛用性不足。为此，提出了一种通用的端到端的深度集成学习模型E2E-DEL（end-to-end deep ensemble learning），由多个初级学习器和一个集成学习器组成，可以自动学习复杂三维模型的复合特征信息；并使用层次迭代式学习策略，综合考量不同层次网络的特征学习能力，合理平衡各个初级学习器的子特征学习和集成学习器的集成特征学习效果，自适应于三维模型多样化分类任务。基于此，设计了一种面向多视图的深度集成学习网络MV-DEL（multi-view deep ensemble learning），应用于一般性、细粒度、零样本三种不同类型的三维模型分类任务中。在多个公开数据集上的实验验证了该方法具有良好的泛化性与普适性。

关键词: 深度学习, 深度集成学习, 三维模型分类

BAI Shaojin, BAI Jing, SI Qinglong, JI Hui, YUAN Tao. Deep Ensemble Learning for Diversified 3D Model Classification[J]. Computer Engineering and Applications, 2023, 59(5): 222-231.

白少进, 白静, 司庆龙, 姬卉, 袁涛. 面向三维模型多样化分类的深度集成学习[J]. 计算机工程与应用, 2023, 59(5): 222-231.

References

[1] WEI X S，WU J，CUI Q.Deep learning for fine-grained image analysis：a survey[EB/OL].（2019-06-06）[2021-09-07].https：//arxiv.org/pdf/1907.03069.pdf.
[2] 祝钧桃，姚光乐，张葛祥，等.深度神经网络的小样本学习综述[J].计算机工程与应用，2021，57（7）：22-33.
ZHU J T，YAO G L，ZHANG G X，et al.Survey of few shot learning of deep neural network[J].Computer Engineering and Applications，2021，57（7）：22-33.
[3] XIAN Y，LAMPERT C H，SCHIELE B，et al.Zero-shot learning—a comprehensive evaluation of the good，the bad and the ugly[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2018，41（9）：2251-2265.
[4] WU Z，SONG S，KHOSLA A，et al.3D ShapeNets：a deep representation for volumetric shapes[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Boston，Jun 8-10，2015.Los Alamitos：IEEE Computer Society Press，2015：1912-1920.
[5] PICKUP D，SUN X，ROSIN P L，et al.SHREC’15 track：canonical forms for non-rigid 3D shape retrieval[C]//Eurographics Workshop on 3D Object Retrieval，2015：316-327.
[6] JIN X，DAI G，FAN Z，et al.DeepShape：deep-learned shape descriptor for 3D shape retrieval[J].IEEE Transactions on Pattern Analysis & Machine Intelligence，2017，39（7）：1335-1345.
[7] MATURANA D，SCHERER S.VoxNet：a 3D convolutional neural network for real-time object recognition[C]//Proceedings of Intelligent Robots and Systems，Hamburg，Sep 18-Oct 2，2015.Washington DC：IEEE Computer Society Press，2015：922-928.
[8] QI C R，SU H，MO K，et al.PointNet：deep learning on point sets for 3D classification and segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，Jul 21-26，2017.Los Alamitos：IEEE Computer Society Press，2017：652-660.
[9] QI C R，YI L，SU H，et al.PointNet++：deep hierarchical feature learning on point sets in a metric space[C]//Advances in Neural Information Processing Systems，Long Bench，Dec 4-9，2017.Cambridge：MIT Press，2017：5105-5114.
[10] LI Y，BU R，SUN M，et al.Pointcnn：convolution on x-transformed points[C]//Advances in Neural Information Processing Systems，Montreal，Dec 3-8，2018.Cambridge：MIT Press，2018：828-838.
[11] 白静，司庆龙，秦飞巍.轻量级实时点云分类网络LightPointNet[J].计算机辅助设计与图形学学报，2019，31（4）：102-111.
BAI J，SI Q L，QIN F W.Lightweight real-time point cloud classification network LightPointNet[J].Journal of Computer-Aided Design and Computer Graphics，2019，31（4）：102-111.
[12] SU H，MAJI S，KALOGERAKIS E，et al.Multi-view convolutional neural networks for 3D shape recognition[C]//Proceedings of the IEEE International Conference on Computer Vision，Boston，Jun 8-10，2015.Los Alamitos：IEEE Computer Society Press，2015：945-953.
[13] SHI B，BAI S，ZHOU Z，et al.DeepPano：deep panoramic representation for 3-D shape recognition[J].IEEE Signal Processing Letters，2015，22（12）：2339-2343.
[14] SINHA A，BAI J，RAMANI K.Deep learning 3D shape surfaces using geometry images[C]//Proceedings of the European Conference on Computer Vision，Amsterdam，Oct 8-16，2016.Berlin，Heidelberg：Springer，2016：223-240.
[15] 白静，司庆龙，秦飞巍.基于卷积神经网络和投票机制的三维模型分类与检索[J].计算机辅助设计与图形学学报，2019，31（2）：123-134.
BAI J，SI Q L，QIN F W.3D model classification and retrieval based on CNN and voting scheme[J].Journal of Computer-Aided Design and Computer Graphics，2019，31（2）：123-134.
[16] WANG C，PELILLO M，SIDDIQI K.Dominant set clustering and pooling for multi-view 3D object recognition[EB/OL].（2019-06-04）[2021-09-07].https：//arxiv.org/pdf/1906.01592.pdf.
[17] HAN Z，SHANG M，LIU Z.et al.SeqViews2SeqLabels：learning 3D global features via aggregating sequential views by RNN with attention[J].IEEE Transactions on Image Processing，2018，28（2）：658-672.
[18] KANEZAKI A，MATSUSHITA Y，NISHIDA Y.RotationNet：joint object categorization and pose estimation using multiviews from unsupervised viewpoints[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Utah，Jun 18-22，2018.Los Alamitos：IEEE Computer Society Press，2018：5010-5019.
[19] LIU X，HAN Z，LIU Y S，et al.Fine-grained 3D shape classification with hierarchical part-view attention[J].IEEE Transactions on Image Processing，2021，30：1744-1758.
[20] GE W，LIN X，YU Y.Weakly supervised complementary parts models for fine-grained image classification from the bottom up[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Long Beach，Jun 16-20，2019.Los Alamitos：IEEE Computer Society Press，2019：3029-3038.
[21] WANG Y，MORARIU V I，DAVIS L S.Learning a discriminative filter bank within a CNN for fine-grained recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Utah，Jun 18-22，2018.Los Alamitos：IEEE Computer Society Press，2018：4148-4157.
[22] LIN T Y，ROYCHOWDHURY A，MAJI S.Bilinear CNN models for fine-grained visual recognition[C]//Proceedings of the IEEE International Conference on Computer Vision，Boston，Jun 8-10，2015.Los Alamitos：IEEE Computer Society Press，2015：1449-1457.
[23] GAO Y，BEIJBOM O，ZHANG N，et al.Compact bilinear pooling[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Las Vegas，Jun 26-Jul 1，2016.Los Alamitos：IEEE Computer Society Press，2016：317-326.
[24] YU C，ZHAO X，ZHENG Q，et al.Hierarchical bilinear pooling for fine-grained visual recognition[C]//Proceedings of the European Conference on Computer Vision，Munich，Sep 8-14，2018.Berlin，Heidelberg：Springer，2018：574-589.
[25] LAROCHELLE H，ERHAN D，BENGIO Y.Zero-data learning of new tasks[C]//Proceedings of AAAI Conference on Artificial Intelligence，Chicago，Jul 13-17，2008.Palo Alto，CA：AAAI Press，2008：646-651.
[26] LAMPERT C H，NICKISCH H，HARMELING S.Learning to de tect unseen object classes by between-class attribute transfer[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Miami，Jun 20-25，2009.Los Alamitos：IEEE Computer Society Press，2009：951-958.
[27] FROME A，CORRADO G S，SHLENS J，et al.Devise：a deep visual-semantic embedding model[C]//Advances in Neural Information Processing Systems，LakeTahoe，Dec 5-10，2013.Cambridge：MIT Press，2013：2121-2129.
[28] KODIROV E，XIANG T，GONG S.Semantic autoencoder for zero-shot learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，Jul 21-26，2017.Los Alamitos：IEEE Computer Society Press，2017：3174-3183.
[29] XIAN Y，LORENZ T，SCHIELE B，et al.Feature generating networks for zero-shot learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Utah，Jun 18-22，2018.Los Alamitos：IEEE Computer Society Press，2018：5542-5551.
[30] SARIYILDIZ M B，CINBIS R G.Gradient matching generative networks for zero-shot learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Long Beach，Jun 16-20，2019.Los Alamitos：IEEE Computer Society Press，2019：2168-2178.
[31] CHERAGHIAN A，RAHMAN S，PETERSSON L.Zero-shot learning of 3D point cloud objects[C]//Proceedings of the 16th International Conference on Machine Vision Applications，Tokyo，May 27-31，2019.Los Alamitos：IEEE Computer Society Press，2019：1-6.
[32] CHERAGHIAN A，RAHMAN S，PETERSSON L.Mitigating the hubness problem for zero-shot learning of 3D objects[EB/OL].（2019-07-15）[2019-09-07].https：//arxiv.org/abs/1907.06371.pdf.
[33] CHERAGHIAN A，RAHMAN S，CAMPBELL D，et al.Transductive zero-shot learning for 3D point cloud classi-fication[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision，Snowmass Village，Mar 1-5，2020.Los Alamitos：IEEE Computer Society Press，2020：923-933.
[34] KRIZHEVSKY A，SUTSKEVER I，HINTON G E.Image-Net classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems，2012：1097-1105.
[35] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Las Vegas，Jun 26-Jul 1，2016.Los Alamitos：IEEE Computer Society Press，2016：770-778.
[36] HOWARD A G，ZHU M，CHEN B，et al.MobileNets：efficient convolutional neural networks for mobile vision applications[EB/OL].（2017-04-17）[2021-09-07].https：//arxiv.org/pdf/1704.04861.pdf.