Abstract: Three-dimensional large language model（3D LLM）, as an important cross-modal learning approach, can not only process linguistic data but also integrate and comprehend diverse modalities such as 3D point clouds, images, and videos, promoting the development in scene understanding, reasoning, and generative tasks. With the increasing demand for spatial perception and multimodal data processing in intelligent systems, the application of 3D LLM is becoming more significant. For the development of 3D LLM, its characteristics, research directions, challenges and research objectives are explored. The network framework of 3D LLM is discussed, including the construction of multi-source 3D datasets, data preprocessing and feature extraction, multimodal feature fusion, model pre-training and efficient optimization strategies, as well as the application of a variety of downstream tasks, and the evaluation methods of 3D LLM are analyzed, covering the model comprehensive performance comparison, zero-sample learning and generalization ability analysis. Finally, the research limitations of 3D LLM are briefly described, application prospects are envisioned, and directions for future research can be proposed.

Key words: 3D large language model（3D LLM）, multimodal learning, point cloud, neural network, feature fusion

摘要： 三维大语言模型（3D LLM）作为一种重要的跨模态学习方法，不仅能够处理语言数据，还能融合和理解三维点云、图像和视频等多种数据，推动场景理解、推理和生成任务的发展。随着智能系统对空间感知和多模态数据处理需求的增加，三维大语言模型的应用需求日益增长。针对三维大语言模型发展，探讨了其特点、研究方向、面临的挑战及研究目标，讨论了三维大语言模型的网络框架，包括多源三维数据集构建、数据预处理与特征提取、多模态特征融合、模型预训练与高效优化策略，以及多种下游任务的应用，分析了三维大语言模型的评估方法，涵盖模型综合性能比较、零样本学习及泛化能力分析。简述了三维大语言模型的研究局限，展望了应用前景，并提出了未来可进行研究的方向。

关键词: 三维大语言模型（3D LLM）, 多模态学习, 点云, 神经网络, 特征融合