Research on Rapid Reconstruction System for Augmented Reality Environment

doi:10.3778/j.issn.1002-8331.2205-0019

Abstract

Abstract: In response to the problems of long environment reconstruction time, high equipment threshold and difficulty in positioning outdoor environment reconstruction when performing environment reconstruction with existing augmented reality technology, the AR environment rapid reconstruction system is designed and implemented. Using Google ARCore as the basis for development and calling on environmental information estimation tools to obtain 3D model spatial coordinates and environmental depth information. A new multiple target tracking and matching algorithm is designed to parse target features and generate correspondence with the 3D model. The BeiDou global navigation satellite system（GNSS） receiver is added to improve the original positioning module, and the parameter conversion method is used to localize the high-precision coordinates. Finally, a split pose matching method is used to correct the reconstructed pose of the 3D model in the environment. Experimental results show that the improved system has a wide range of applications, and has the characteristics of not losing the position of the AR environment after leaving the camera, it enhances the user’s orientation perception ability in the AR environment reconstruction, and reduces the difficulty of matching different 3D models and target features in the AR environment.

Key words: augmented reality, Google ARCore, multiple target tracking and matching, BeiDou global navigation satellite, parametric conversion method, split pose matching

摘要： 针对现有增强现实技术在进行环境重建时存在环境重建时间长、设备门槛高、户外环境重建定位困难等问题，设计并实现了AR环境快速重建系统。以谷歌ARCore作为开发基础，调用环境信息估算工具获取三维模型空间坐标和环境深度信息；设计一种新的多重目标跟踪匹配算法，以解析目标特征，并与三维模型生成对应关系；加入北斗全球卫星信号（global navigation satellite system，GNSS）接收机对原有的定位模块进行改进，采用参数转换法对高精度坐标进行本地化处理；使用分割位姿匹配法校正环境中的三维模型重建位姿。实验结果证明，该改进系统适用范围广，拥有脱离摄像头后AR环境不丢失位置等特点，增强了AR环境重建中用户的方位感知能力，降低了AR环境中不同三维模型和目标特征的匹配处理难度。

关键词: 增强现实, 谷歌ARCore, 多重目标跟踪匹配, 北斗全球卫星, 参数转换法, 分割位姿匹配

PENG Ruinan, WAN Taoruan, GONG Linming. Research on Rapid Reconstruction System for Augmented Reality Environment[J]. Computer Engineering and Applications, 2023, 59(15): 290-299.

彭睿男, 万韬阮, 巩林明. 增强现实环境快速重建系统研究[J]. 计算机工程与应用, 2023, 59(15): 290-299.

References

[1] CHANDLER J，FRYER J.Autodesk 123D catch：how accurate is it[J].Geomatics World，2013，2（21）：28-30.
[2] NEWCOMBE R A，IZADI S，HILLIGES O，et al.Kinectfusion：real-time dense surface mapping and tracking[C]//2011 10th IEEE International Symposium on Mixed and Augmented Reality，2011：127-136.
[3] IZADI S，KIM D，HILLIGES O，et al.Kinectfusion：real-time 3d reconstruction and interaction using a moving depth camera[C]//Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology，2011：559-568.
[4] WHELAN T，SALAS-MORENO R F，GLOCKER B，et al.ElasticFusion：real-time dense SLAM and light source estimation[J].The International Journal of Robotics Research，2016，35（14）：1697-1716.
[5] NEWCOMBE R A，FOX D，SEITZ S M.Dynamicfusion：reconstruction and tracking of non-rigid scenes in real-time[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2015：343-352.
[6] 赵洋，刘国良，田国会，等.基于深度学习的视觉SLAM综述[J].机器人，2017，39（6）：889-896.
ZHAO Y，LIU G L，TIAN G H，et al.A survey of visual SLAM based on deep learning[J].Robot，2017，39（6）：889-896.
[7] PRISACARIU V A，K?HLER O，GOLODETZ S，et al.Infinitam v3：a framework for large-scale 3d reconstruction with loop closure[J].arXiv：1708.00783，2017.
[8] DAI A，NIE?NER M，ZOLLH?FER M，et al.Bundlefusion：real-time globally consistent 3d reconstruction using on-the-fly surface reintegration[J].ACM Transactions on Graphics（ToG），2017，36（4）：1.
[9] PENCZEK P A，ZHU J，SCHR?DER R，et al.Three dimensional reconstruction with contrast transfer compensation from defocus series[M]//Single-particle cryo-electron microscopy：the path toward atomic resolution：selected papers of joachim frank with commentaries，2018：232-239.
[10] PORPIGLIA F，FIORI C，CHECCUCCI E，et al.Hyperaccuracy three-dimensional reconstruction is able to maximize the efficacy of selective clamping during robot-assisted partial nephrectomy for complex renal masses[J].European Urology，2018，74（5）：651-660.
[11] WANG G，QIN X，SHEN J，et al.Quantitative analysis of microscopic structure and gas seepage characteristics of low-rank coal based on CT three-dimensional reconstruction of CT images and fractal theory[J].Fuel，2019，256：115900.
[12] YIN L，WANG X，NI Y.Flexible three-dimensional reconstruction via structured-light-based visual positioning and global optimization[J].Sensors，2019，19（7）：1583.
[13] HUANG L，DA F，GAI S.Research on multi-camera calibration and point cloud correction method based on three-dimensional calibration object[J].Optics and Lasers in Engineering，2019，115：32-41.
[14] HU Y，CHEN Q，FENG S，et al.A new microscopic telecentric stereo vision system-calibration，rectification，and three-dimensional reconstruction[J].Optics and Lasers in Engineering，2019，113：14-22.
[15] LANHAM M.Learn ARCore-fundamentals of google ARCore：learn to build augmented reality apps for android，Unity，and the web with Google ARCore 1.0[M].[S.l.]：Packt Publishing Ltd，2018：10-25.
[16] DU R，TURNER E，DZITSIUK M，et al.DepthLab：real-time 3D interaction with depth maps for mobile augmented reality[C]//Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology，2020：829-843.
[17] 刘权.基于智能移动终端的移动增强现实应用研究与实现[D].昆明：云南大学，2013：1-5.
LIU Q.Research and implementation of mobile augmented reality application based on intelligent mobile terminal[D].Kunming：Yunnan University，2013：1-5.
[18] LI P，ZHANG X H，GUO F.Ambiguity resolved precise point positioning with GPS and BeiDou[J].Journal of Geodesy，2017，91（1）：25-40.
[19] GENG J，CHEN X，PAN Y，et al.PRIDE PPP-AR：an open-source software for GPS PPP ambiguity resolution[J].GPS Solutions，2019，23（4）：1-10.
[20] 陈靖，王涌天，刘越，等.适用于户外增强现实系统的混合跟踪定位算法[J].计算机辅助设计与图形学学报，2010，22（2）：204-209.
CHEN J，WANG Y T，LIU Y，et al.Hybrid tracking for outdoor augmented reality system[J].Journal of Computer-Aided Design and Computer Graphics，2010，22（2）：204-209.
[21] NEEDHAM，MASS.Smartphone shipments declined in the fourth quarter but 2021 was still a growth year with a 5.7% increase in shipments，according to IDC[EB/OL].（2022-01-27）[2022-02-27].https：//www.idc.com/getdoc.jsp?containerId=prUS48830822.
[22] MORAR A，B?LU?OIU M A，MOLDOVEANU A，et al.Evaluation of the ARCore indoor localization technology[C]//2020 19th RoEduNet Conference：Networking in Education and Research（RoEduNet），2020：1-5.
[23] 周玲，陈强.城市地下管网增强现实系统研究[J].计算机工程与应用，2020，56（1）：251-256.
ZHOU L，CHEN Q.Research on augmented reality system of urban underground pipe network[J].Computer Engineering and Applications，2020，56（1）：251-256.
[24] TAMURA Y，MATSUI M，PAGNINI L C，et al.Measurement of wind-induced response of buildings using RTK-GPS[J].Journal of Wind Engineering and Industrial Aerodynamics，2002，90（12/15）：1783-1793.
[25] JIANG Y，FANG Y，YAO C，et al.A design of indoor & outdoor navigation system[C]//International Conference on Communication Technology and Application（ICCTA 2011），Beijing，China，October 2011.UK：Institution of Engineering and Technology，2011：14-16.
[26] MU X，CHEN J，ZHOU Z，et al.Accurate initial state estimation in a monocular visual-inertial SLAM system[J].Sensors，2018，18（2）：506-516.
[27] YANG S，BAUM M.Extended Kalman filter for extended object tracking[C]//2017 IEEE International Conference on Acoustics，Speech and Signal Processing（ICASSP），2017：4386-4390.
[28] VANCLOOSTER A，MAEYER P D.Combining indoor and outdoor navigation：the current approach of route planners[M]//Advances in location-based services.Berlin，Heidelberg：Springer，2012：283-303.
[29] GOU Z Y，LIU C R，FAN S H，et al.A new measurement model of GPS and inclinometer in coordinate transform[C]//International Conference on Advances in Energy and Environmental Science，2013：37-63.
[30] LEE S C，RIZAL S，AHN H J.Analysis of the position estimation error of a local positioning system utilizing mobile anchors[EB/OL].（2018-10-04）[2022-10-29].https：//www.preprints.org/manuscript/201810.0086/v1.