Tracking and Registration Method Based on Point Cloud Matching for Augmented Reality Facing Work System

doi:10.3778/j.issn.1002-8331.2110-0379

Abstract

Abstract: This paper proposes a tracking and registration method based on point cloud matching of augmented reality to solve the occlusion problem in facing work of construction robot. Firstly, the point cloud of the target model and the operating environment is matched and the location of target is initialized. Secondly, the improved correlation filter tracking algorithm is adopted to track the target and obtain the target position. Then, the ICP（iterative closest point） method is applied to estimate the pose of target. Finally, pose optimization is added in the tracking registration process. In order to track the target more accurately, the scale-adaptive correlation filter tracking based on multiple features is proposed. Through the panel installation experiments are carried out and the results show that the proposed method has good accuracy and real-time performance. The minimum positioning error is 2.88 mm, and the effect of virtual and reality fusion is perfect.

Key words: augmented reality, tracking and registration, point cloud matching, facing work

摘要： 针对建筑机器人饰面作业过程中常因视觉遮挡导致作业效率低的问题，使用增强现实解决遮挡并提出一种基于点云匹配的增强现实跟踪注册方法。利用目标模型点云与作业环境点云的匹配进行目标的初始定位；利用改进的相关滤波跟踪算法对目标进行跟踪获取目标位置；基于迭代最近点法对目标位姿进行估计。在跟踪注册过程中加入位姿优化，保证目标位姿估计精度。为了更加准确地跟踪目标位置，提出一种特征融合和尺度自适应的改进相关滤波目标跟踪算法。通过板材安装实验，表明跟踪注册方法精确性、实时性均较好，最小识别误差达到2.88?mm，具有良好的虚实融合效果。

关键词: 增强现实, 跟踪注册, 点云匹配, 饰面作业

JIA Xiaohui, FENG Chongyang, LIU Jinyue. Tracking and Registration Method Based on Point Cloud Matching for Augmented Reality Facing Work System[J]. Computer Engineering and Applications, 2023, 59(6): 291-298.

贾晓辉, 冯重阳, 刘今越. 基于点云匹配的AR饰面作业系统跟踪注册方法[J]. 计算机工程与应用, 2023, 59(6): 291-298.

References

[1] 刘笑含.干挂安装板材机器人系统关键技术研究[D].天津：河北工业大学，2011.
LIU X H.Research on the key technology of a robot system used for installing the dry-hanging slabstone[D].Tianjin：Hebei University of Technology，2011.
[2] 程志伟.基于力反馈和视觉的板材安装定位研究[D].天津：河北工业大学，2010.
CHENG Z W.Research on slabstone installation and positioning based on force feedback and vision[D].Tianjin：Hebei University of Technology，2010.
[3] 刘智光，于菲，张靓，等.基于模糊自适应阻抗控制的机器人接触力跟踪[J].工程设计学报，2015，22（6）：569-575.
LIU Z G，YU F，ZHANG L，et al.Force tracking research for robot based on fuzzy adaptive impedance control algorithm[J].Chinese Journal of Engineering Design，2015，22（6）：569-575.
[4] LI X，YI W，CHI H L，et al.A critical review of virtual and augmented reality （VR/AR） applications in construction safety[J].Automation in Construction，2018，86：150-162.
[5] 韩玉仁，李铁军，杨冬.增强现实中三维跟踪注册技术概述[J].计算机工程与应用，2019，55（21）：26-35.
HAN Y R，LI T J，YANG D.Overview of 3D tracking registration technology in augmented reality[J].Computer Engineering and Applications，2019，55（21）：26-35.
[6] KHAN D，ULLAH S，RABBI I.Factors affecting the design and tracking of AR ToolKit markers[J].Computer Standards & Interfaces，2015，41：56-66.
[7] 肖鸿.面向复杂产品装配现场的移动三维模型关键技术研究[D].西安：西北工业大学，2014.
XIAO H.Research on the key technologies of mobile three-dimensional model for assembly site of complex product[D].Xi’an：Northwestern Polytechnical University，2014.
[8] TOMBARI F，FRANCHI A，STEFANO D L.BOLD features to detect texture-less objects[C]//2103 IEEE International Conference on Computer Vision，1-8 Dec 2013，Sydney，Australia，2013：1265-1272.
[9] 张昊鹏，郭宇，汤鹏洲，等.基于图像匹配的增强现实装配系统跟踪注册方法[J].计算机集成制造系统，2021，27（5）：1281-1291.
ZHANG H P，GUO Y，TANG P Z，et al.Tracking and registration method based on image matching for augmented reality aided assembly system[J].Computer Integrated Manufacturing Systems，2021，27（5）：1281-1291.
[10] WANG Y，ZHANG S，YANG S.A LINE-MOD-based markerless tracking approach for AR applications[J].The International Journal of Advanced Manufacturing Technology，2017，89：1699-1707.
[11] MUR-ARTAL R，MONTIEL J M M，TARDOS J D.ORB-SLAM：a versatile and accurate monocular SLAM system[J].IEEE Transactions on Robotics，2015，31（5）：1147-1163.
[12] QIN T，LI P L，SHEN S J.VINS-Mono：a robust and versatile monocular visual-inertial state estimator[J].IEEE Transactions on Robotics，2018，34（4）：1004-1020.
[13] 雍玖，王阳萍，雷晓妹.MEEM跟踪和改进ORB特征检测的三维注册方法[J].计算机工程与应用，2019，55（24）：147-153.
YONG J，WANG Y P，LEI X M.3D registration method for MEEM tracking and improving ORB feature detection[J].Computer Engineering and Applications，2019，55（24）：147-153.
[14] 雍玖，王阳萍，党建武，等.改进TLD与ORB的AR系统长时跟踪注册方法[J].计算机工程与应用，2021，57（7）：178-184.
YONG J，WANG Y P，DANG J W，et al.Improved TLD and ORB based long term tracking registration method for AR system[J].Computer Engineering and Applications，2021，57（7）：178-184.
[15] ADKOWSKI R.Object tracking with a range camera for augmented reality assembly assistance[J].Journal of Computing and Information Science in Engineering，2015，16（1）：011004.
[16] NEWCOMBE R A.KinectFusion：real-time dense surface mapping and tracking[C]//10th IEEE International Symposium on Mixed and Augmented Reality，Basel，Switzerland，2011：127-136.
[17] HENRIQUES J，CASEIRO R，MARTINS P，et al.High-speed tracking with kernelized correlation filters[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，37（3）：583-596.
[18] BERTINETTO L，VALMADRE J，GOLODETZ S，et al.Staple：complementary learners for real-time tracking[C]//IEEE Conference on Computer Vision and Pattern Recognition，Las Vegas，USA，2016：1401-1409.
[19] 张红颖，胡文博.多特征融合的尺度自适应相关滤波跟踪算法[J].系统工程与电子技术，2019，41（5）：951-957.
ZHANG H Y，HU W B.Scale-adaptive correlation filter tracking based on multiple features[J].Systems Engineering and Electronics，2019，41（5）：951-957.
[20] DANELLJAN M，H?GER G，KHAN F，et al.Accurate scale estimation for robust visual tracking[C]//British Machine Vision Conference，Nottingham，2014.
[21] HENRY P，KRAININ M，HERBST E，et al.RGB-D mapping：using kinect-style depth cameras for dense 3D modeling of indoor environments[J].International Journal of Robotics Research，2012，31（5）：647-663.
[22] WU Y，LIM J，YANG M.Online object tracking：a benchmark[C]//IEEE Conference on Computer Vision and Pattern Recognition，Portland，USA，2013：2411-2418.
[23] 任结，周余，于耀，等.基于ORB自然特征AR实时系统实现[J].计算机应用研究，2012，29（9）：3594-3596.
REN J，ZHOU Y，YU Y，et al.Real-time augmented reality based on ORB[J].Applications Research of Computers，2012，29（9）：3594-3596.