基于草图的细粒度类级三维模型检索数据集

doi:10.3778/j.issn.1002-8331.2406-0143

摘要/Abstract

摘要： 近年来，细粒度三维模型分类在计算机图形学和计算机视觉领域受到越来越多的关注。如何有效检索细粒度类级三维模型引起了学者的关注。构建了一个基于草图的细粒度类级三维模型检索数据集，并命名为FGCL-SBSR，以支持草图到元类别下特定子类三维模型的检索。该数据集中三维模型取自FG3D数据集，而草图则是通过招募志愿者绘制的方式完成收集。在草图收集完成后，对其进行了筛选，并据此对数据集进行了合理的划分。具体来说，FGCL-SBSR中Airplane子数据集包含12个类，其中三维模型1?388个，草图1?286张；Chair子数据集包含25个类，其中三维模型2?321个，草图2?102张。该数据集在子类别上建立了草图与三维模型之间的对应关系，并且保持了草图的抽象性、稀疏性、多样性和代表性。最后，通过不同训练集训练模型以及不同方法在该数据集中的实验结果证明了该数据集可以很好地适用于基于草图的细粒度类级三维模型检索任务，充分体现了其必要性、一致性和有效性。

关键词: 深度学习, 细粒度类级, 基于草图的三维模型检索

Abstract: Recently, fine-grained 3D shape classification has received growing attention in the community of computer graphics and computer vision. The paper constructs a sketch-based fine-grained class-level 3D shape retrieval dataset, named FGCL-SBSR, to support the use of sketches to retrieve 3D shapes of specific subclasses under a metaclass. The 3D shapes in this dataset are sourced from the FG3D dataset, while the sketches are collected through recruiting volunteers to draw them. After the sketches are collected, they are filtered and the dataset is accordingly divided rationally. Specifically, the Airplane sub-dataset in FGCL-SBSR contains 12 classes, including 1 388 3D shapes and 1 286 sketches, and the Chair sub-dataset contains 25 classes, including 2 321 3D shapes and 2 102 sketches. The dataset establishes a correspondence between sketches and 3D shapes in terms of subcategories and maintains the abstraction, sparseness, diversity and representativeness of the sketches. Finally, the experimental results of training models with different training sets and different methods on this paper’s dataset prove that this paper’s dataset can be well suited for the task of sketch-based fine-grained class-level 3D shape retrieval, fully demonstrating its necessity, consistency and effectiveness.

Key words: deep learning, fine-grained class-level, sketch-based 3D shape retrieval

郑虎, 白静, 晏浩, 苏雅雯. 基于草图的细粒度类级三维模型检索数据集[J]. 计算机工程与应用, 2025, 61(19): 226-236.

ZHENG Hu, BAI Jing, YAN Hao, SU Yawen. Fine-Grained Class-Level Sketch-Based 3D Shape Retrieval Dataset[J]. Computer Engineering and Applications, 2025, 61(19): 226-236.

参考文献

[1] XIE J, DAI G X, ZHU F, et al. DeepShape: deep-learned shape descriptor for 3D shape retrieval[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(7): 1335-1345.
[2] ZHAO Y, LIANG Q, MA R X, et al. JFLN: joint feature learning network for 2D sketch based 3D shape retrieval[J]. Journal of Visual Communication and Image Representation, 2022, 89: 103668.
[3] FU R, ZHANG Y C, YANG J, et al. ROSA-net: rotation-robust structure-aware network for fine-grained 3D shape retrieval[C]//Proceedings of the International Conference on Computational Visual Media. Singapore: Springer, 2024: 295-319.
[4] SHAO H H, BAI J, WU R S, et al. FGPNet: a weakly supervised fine-grained 3D point clouds classification network[J]. Pattern Recognition, 2023, 139: 109509.
[5] QI A R, GRYADITSKAYA Y, SONG J F, et al. Toward fine-grained sketch-based 3D shape retrieval[J]. IEEE Transactions on Image Processing, 2021, 30: 8595-8606.
[6] CHEN X, ZHONG Z, ZHOU D B. Spatially aligned sketch-based fine-grained 3D shape retrieval[J]. Neural Computing and Applications, 2023, 35(22): 16607-16617.
[7] LIU X H, HAN Z Z, LIU Y S, et al. Fine-grained 3D shape classification with hierarchical part-view attention[J]. IEEE Transactions on Image Processing, 2021, 30: 1744-1758.
[8] YUAN J F, LI B, LU Y J, et al. SHREC’18 track: 2D scene sketch-based 3D scene retrieval[C]//Proceedings of the Eurographics Workshop on 3D Object Retrieval, 2018: 29-36.
[9] YUAN J F, ABDUL-RASHID H, LI B, et al. SHREC’19 track: extended 2D scene sketch-based 3D scene retrieval[C]//Proceedings of the Eurographics Workshop on 3D Object Retrieval, 2019.
[10] LI B, LU Y, GODIL A, et al. SHREC’13 track: large scale sketch-based 3D shape retrieval[C]//Proceedings of the Eurographics Workshop on 3D Object Retrieval, Girona, Spain, 2013: 89-96.
[11] LI B, LU Y, LI C, et al. SHREC’14 track: extended large scale sketch-based 3D shape retrieval[C]//Proceedings of the Eurographics Workshop on 3D Object Retrieval, Strasbourg, France, 2014: 121-130.
[12] LI B, LU Y J, DUAN F Q, et al. SHREC’16 track: 3D sketch-based 3D shape retrieval[C]//Proceedings of the Eurographics Workshop on 3D Object Retrieval, Lisbon, Portugal, 2016.
[13] XU P, HOSPEDALES T M, YIN Q, et al. Deep learning for free-hand sketch: a survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45(1): 285-312.
[14] YU Q, LIU F, SONG Y Z, et al. Sketch me that shoe[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 799-807.
[15] EITZ M, HILDEBRAND K, BOUBEKEUR T, et al. An evaluation of descriptors for large-scale image retrieval from sketched feature lines[J]. Computers & Graphics, 2010, 34(5): 482-498.
[16] WANG B R, ZHOU Y. Doodle to object: practical zero-shot sketch-based 3D shape retrieval[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2023, 37(2): 2474-2482.
[17] SU H, MAJI S, KALOGERAKIS E, et al. Multi-view convolutional neural networks for 3D shape recognition[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2015: 945-953.
[18] EITZ M, HAYS J, ALEXA M. How do humans sketch objects?[J]. ACM Transactions on Graphics, 2012, 31(4): 1-10.
[19] LI B, LU Y J, GODIL A, et al. A comparison of methods for sketch-based 3D shape retrieval[J]. Computer Vision and Image Understanding, 2014, 119: 57-80.
[20] HE X W, ZHOU Y, ZHOU Z C, et al. Triplet-center loss for multi-view 3D object retrieval[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 1945-1954.
[21] DAI W D, LIANG S. Cross-modal guidance network for sketch-based 3D shape retrieval[C]//Proceedings of the 2020 IEEE International Conference on Multimedia and Expo. Piscataway: IEEE, 2020: 1-6.
[22] LIANG S, DAI W D, WEI Y C. Uncertainty learning for noise resistant sketch-based 3D shape retrieval[J]. IEEE Transactions on Image Processing, 2021, 30: 8632-8643.
[23] WANG F, CHENG J, LIU W Y, et al. Additive margin softmax for face verification[J]. IEEE Signal Processing Letters, 2018, 25(7): 926-930.
[24] 白静, 拖继文, 白少进, 等. 基于自适应多类中心和半异构网络的三维模型草图检索[J]. 图学学报, 2022, 43(1): 36-43.
BAI J, TUO J W, BAI S J, et al. Adaptive multi-class centers and semi-heterogeneous network for sketch-based 3D model retrieval[J]. Journal of Graphics, 2022, 43(1): 36-43.
[25] 杨瞻源, 白静, 李文静, 等. 面向三维模型草图检索的三元层次度量网络[J]. 计算机辅助设计与图形学学报, 2024, 36(11): 1791-1804.
YANG Z Y, BAI J, LI W J, et al. Triplet hierarchical metric network for sketch-based 3D shape retrieval[J]. Journal of Computer-Aided Design & Computer Graphics, 2024, 36(11): 1791-1804.
[26] LI W J, BAI J, ZHENG H. D2GL: dual-level dual-scale graph learning for sketch-based 3D shape retrieval[J]. Pattern Recognition, 2024, 156: 110768.