Computer Engineering and Applications ›› 2024, Vol. 60 ›› Issue (4): 21-38.DOI: 10.3778/j.issn.1002-8331.2303-0218

• Research Hotspots and Reviews • Previous Articles     Next Articles

Survey of Neural Radiance Fields for Multi-View Synthesis Technologies

MA Hansheng, ZHU Yuhua, LI Zhihui, YAN Lei, SI Yiyi, LIAN Yimeng, ZHANG Yuhan   

  1. 1. College of Information Science and Engineering, Henan University of Technology, Zhengzhou 450001, China
    2. Key Laboratory of Grain Information Processing and Control, Ministry of Education, Henan University of Technology, Zhengzhou 450001, China
    3. Collaborative Innovation Center of Grain Storage Security in Henan Province, Henan University of Technology, Zhengzhou 450001, China
  • Online:2024-02-15 Published:2024-02-15

神经辐射场多视图合成技术综述

马汉声,祝玉华,李智慧,阎磊,司艺艺,连一萌,张钰涵   

  1. 1. 河南工业大学  信息科学与工程学院,郑州  450001
    2. 河南工业大学  粮食信息处理与控制教育部重点实验室,郑州  450001
    3. 河南工业大学  粮食储藏安全河南省协同创新中心,郑州  450001

Abstract: Rendering realistic virtual scenes from images has been a long-standing research goal in the fields of computer graphics and computer vision. NeRF (neural radiance fields) is an emerging method based on deep neural networks, which achieves realistic rendering by learning the radiance field of each point in the scene. By using neural radiance fields, not only realistic images but also realistic three-dimensional scenes can be generated, making it have a wide range of application prospects such as virtual reality, augmented reality and computer games. However, its basic model has shortcomings such as low training efficiency, poor generalization ability, insufficient interpretability, susceptible to lighting and material changes, inability to handle dynamic scenes, and other deficiencies that may result in suboptimal rendering results in certain situations. With the continuous popularity of this field, a large amount of research has been carried out, yielding impressive results in terms of efficiency and accuracy. In order to track the latest research in this field, this paper provides a review and summary of the key algorithms in recent years. This paper first outlines the background and principles of neural radiance fields, and briefly introduces the evaluation metrics and public datasets in this field. Then, a classification discussion is conducted on the key improvements to the model, mainly including: the optimization of basic NeRF model parameters, the improvement in rendering speed and inference ability, the enhancement of spatial representation and lighting ability, the improvement in camera pose and sparse view synthesis methods for static scene, and the development in dynamic scene modeling field. Subsequently, the speed and performance of various models are classified, compared and analyzed, and the main model evaluation indicators and open datasets in this field are briefly introduced. Finally, the future development trend of neural radiance field is prospected.

Key words: neural radiance fields (NeRF), view synthesis, neural rendering, scene representation, deep learning, 3D reconstruction

摘要: 如何从图像中渲染出较为真实的虚拟场景一直是计算机图形学与计算机视觉领域的研究目标之一。神经辐射场是一种基于深度神经网络的新兴方法,它通过学习场景中每个点的辐射场来实现较为真实的渲染效果。通过神经辐射场不仅可以生成逼真的图像,而且可以生成具有真实感的三维场景,在虚拟现实、增强现实和计算机游戏等领域有着广泛的应用前景。然而,其基础模型存在训练效率低、泛化能力差、可解释性不足、易受光照和材质变化影响以及无法处理动态场景等问题,在某些情况下无法获得最佳的渲染结果。大量基于此研究的工作陆续展开,且在效率和精度等方面都取得了出色的成果。为了跟踪该领域最新研究成果,对近年来神经辐射场领域的关键算法进行回顾和综述。首先介绍了神经辐射场的产生背景及原理,对后续关键改进模型进行分类探讨。主要涵盖以下几个方面:对神经辐射场基本模型参数的优化,在渲染速度与推理能力方面的提升,对空间表达和光照能力的改善,针对静态场景相机位姿估计与稀疏视图合成方法的改进,以及在动态场景建模领域的发展。对各种模型的速度与性能进行分类对比与分析,并简要介绍了该领域主要模型评估指标与公开数据集。最后对神经辐射场未来发展趋势进行展望。

关键词: 神经辐射场(NeRF), 视图合成, 神经渲染, 场景表达, 深度学习, 三维重建