面向施工机器人定位的多模态数据融合方法研究综述

doi:10.3778/j.issn.1002-8331.2402-0008

摘要/Abstract

摘要： 施工机器人的定位数据源种类繁多，融合多模态数据不仅有助于提升建筑项目中施工机器人的定位性能，同时也方便施工机器人的协同作业。数据融合方法旨在通过不同数据源的优势互补，改进数据采集及处理方法等，实现施工机器人的定位和数据共享，支持施工机器人定位精度、实时性或鲁棒性等的提高，从而提高整体建筑施工效率和项目管理水平。已有不少针对特定场景探索施工机器人定位的数据融合方法相关研究成果，但尚无针对施工机器人定位的数据融合方法相关研究综述。经系统的检索，首先，按照是否与先验数据融合，将其分为先验数据与传感器实时数据融合和多种传感器数据融合两类进行分析；然后，对数据融合方法进行对比分析；最后，总结和展望了施工机器人多模态数据融合方法的未来研究方向。从研究结果分析，现阶段已有的研究成果中，施工机器人定位的数据源选择差异性较大，定位效果差异也很大。该综述可为相关领域的进一步研究提供参考。

关键词: 施工机器人, 定位, 数据融合方法, 多模态数据

Abstract: The types of localization data sources for construction robots are diverse, and the fusion of multimodal data not only helps to improve the localization performance of construction robots in building projects but also facilitates their collaborative operations. Data fusion methods aim to enhance construction robot localization and data sharing by leveraging the advantages of different data sources, improving data collection and processing methods, this supports the improvement of construction robot localization accuracy, real-time performance, and robustness, ultimately enhancing overall construction efficiency and project management. While there are existing research outcomes on specific scenarios exploring data fusion methods for construction robot positioning, there is currently no comprehensive review article on this topic. Through systematic retrieval, this paper first categorizes the analysis into two types based on whether it integrates with prior data：fusion of prior data and real-time sensor data fusion, and fusion of data from multiple sensors. Subsequently, a comparative analysis of data fusion methods is conducted. Finally, the paper summarizes and anticipates the future research directions of multimodal data fusion methods for construction robots. The analysis of current research results reveals significant variations in the choice and effectiveness of positioning data sources for construction robots. This review can serve as a reference for further research in related fields.

Key words: construction robot, localization, data fusion method, multimodal data

李佳益, 马智亮, 陈礼杰, 季鑫霖. 面向施工机器人定位的多模态数据融合方法研究综述[J]. 计算机工程与应用, 2024, 60(15): 11-23.

LI Jiayi, MA Zhiliang, CHENG Lijie, JI Xinlin. Comprehensive Review of Multimodal Data Fusion Methods for Construction Robot Localization[J]. Computer Engineering and Applications, 2024, 60(15): 11-23.

参考文献

[1] TONG S H, XU W, ZHANG X C, et al. Experimental and theoretical analysis on truss construction robot: automatic grasping and hoisting of concrete composite floor slab[J]. Journal of Field Robotics, 2023, 40: 272-288.
[2] DING L Y, JIANG W G, ZHOU Y, et al. BIM-based task-level planning for robotic brick assembly through image-based 3D modeling[J]. Advanced Engineering Informatics, 2020, 43: 100993.
[3] JI J J, SHAN J, MISYURIN S Y, et al. Localization on a-prior information of plane extraction[J]. Plos One, 2023, 18(5).
[4] CHEN J J, LI S, LU W S. Align to locate registering photogrammetric point clouds to BIM for robust[J]. Building and Environment, 2022, 209: 1-14.
[5] 帅子沛. 室内建筑抹灰机器人智能导航研究[D]. 成都: 电子科技大学, 2022.
SHUAI Z P. Research on intelligent navigation of plastering robots for indoor buildings[D]. Chengdu: University of Electronic Science and Technology of China, 2022.
[6] ZHANG Z J, CHENG X J, YANG B L, et al. Exploration of indoor barrier-free plane intelligent lofting system combining BIM and multi-sensors[J]. Remote Sensing, 2020, 12(20): 3306.
[7] LUNDEEN K M, KAMAT V R, MENASSA C, et al. Autonomous motion planning and task execution in geometrically adaptive robotized construction work[J]. Automation in Construction, 2019, 100: 24-45.
[8] 程时伟. 人机交互概论[M]. 杭州: 浙江大学出版社, 2018.
CHENG S W. Introduction to human-computer interaction[M]. Hanghzou: Zhejiang University Press, 2018.
[11] HUANG J H, JUNGINGER S, LIU H, et al. Indoor positioning systems of mobile robots a review[J]. Robots, 2023, 12(2): 47.
[12] 李学友. IMU/DGPS 辅助航空摄影测量原理、方法及实践[D]. 郑州: 中国人民解放军信息工程大学, 2005.
LI X Y. IMU/DGPS-supported photogrammetry-theory, approaches and practice, institute of surveying and mapping[D]. Zhengzhou: Information Engineering University, 2005.
[13] SHEN D, XU Y H, HUANG Y K. Research on 2D-SLAM of indoor mobile robot based on laser radar[C]//Proceedings of the 2019 4th International Conference on Automation, Control and Robotics Engineering, Shenzhen, China, July 19-21, 2019.
[14] FU Q, YU H S, LAI L H, et al. A robust RGB-D SLAM system with points and lines for low texture indoor environments[J]. IEEE Sensors Journal, 2019, 19(21): 9908-9920.
[15] YANG N, STUMBERG L V, WANG R, et al. D3VO: deep depth, deep pose and deep uncertainty for monocular visual odometry[C]//Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), 2020: 1278-1289.
[16] SURIYA D. M, SRIVENKATA K S, SUNDARRAJAN G K S, et al. A robust approach for improving the accuracy of IMU based indoor mobile robot localization[C]//Proceedings of the 13th International Conference on Informatics in Control, 2016: 436-445.
[17] ILYAS M, KHAW H Y, SELVARAJ N M, et al. Robot-assisted object detection for construction automation data and information-driven approach[J]. IEEE/ASME Transaction on Mechatronics, 2021, 26(6): 2845-2856.
[18] 吴海兵. 基于RFID相位特征的机器人定位方法研究[D]. 武汉: 华中科技大学, 2020.
WU H B. Research of phase-based RFID localization method for robot[D]. Wuhan: Huazhong University of Science and Technology, 2020.
[19] LIU J, LI T R, XIE P, et al. Urban big data fusion based on deep learning: an overview[J]. Information Fusion, 2020, 53: 123-133.
[20] 张毅, 杜凡宇, 罗元, 等. 一种融合激光和深度视觉传感器的SLAM地图创建方法[J]. 计算机应用研究, 2016, 33(10): 2970-2972.
ZHANG Y, DU F Y, LUO Y, et al. Map-building approach based on laser and depth visual sensor fusion SLAM[J]. Application Research of Computers, 2016, 33(10): 2970-2972.
[21] 张红, 程传祺, 徐志刚, 等. 基于深度学习的数据融合方法研究综述[J]. 计算机工程与应用, 2020, 56(24): 1-11.
ZHANG H, CHENG C Q, XU Z G, et al. Survey of data fusion based on deep learning[J]. Computer Engineering and Applications, 2020, 56(24): 1-11.
[22] WU J, SU Y H, CHENG Y W et al. Multi-sensor information fusion for remaining useful life prediction of machining tools by adaptive network based fuzzy inference system[J]. Applied Soft Computing, 2018, 68: 13-23.
[23] 吴艳. 多传感器数据融合算法研究[D]. 西安: 西安电子科技大学, 2004.
WU Y. Revisions to the JDL data fusion model[D]. Xi’an: Xidian University, 2004.
[24] XU Z, GUO S, SONG T, ZENG L D. Localization of mobile robot aided for large-scale construction based on optimized artificial landmark map in ongoing scene[J]. Computer Mode-
ling in Engineering & Sciences, 2022(3): 1853-1882.
[25] 杨清梅, 王立权, 王岚, 等. 基于数据融合的拱泥机器人定位方法[J]. 哈尔滨工程大学学报, 2003, 24(4): 363-367.
YANG Q M, WANG L Q, WANG L, et al. Location of move-in-mud robot based on data fusion[J]. Journal of Harbin Engineering University, 2003, 24(4): 363-367.
[26] 艾青林, 刘刚江, 徐巧宁. 动态环境下基于改进几何与运动约束的机器人RGB-D SLAM算法[J]. 机器人, 2021, 43(2): 167-176.
AI Q L, LIU G J, XU Q N, et al. An RGB-D SLAM algorithm for robot based on the improved geometric and motion constraints in dynamic environment[J]. Robot, 2021, 43(2): 167-176.
[27] 汪生浩, 于铭铭, 朱猛猛, 等. 应用于移动机器人的钢筋交叉点视觉检测系统研究[J]. 工业控制计算机, 2023, 36(6): 83-85.
WANG S H, YU M M, ZHU M M, et al. Research on visual inspection system of rebar intersection applied to mobile robot[J]. Industrial Control Computer, 2023, 36(6): 83-85.
[28] 王洲, 杨明欣, 王新媛. 基于多传感器融合的多旋翼无人机近地面定位算法[J].成都信息工程大学学报, 2018(3):261-267.
WANG Z，YANG M X,WANG X Y. Location algorithm based on multi-sensor fusion for multi-rotor aerial vehiclesto flight near the ground[J]. Journal of Chengdu University of Information Technology, 2018(3):261-267.
[29] 李中道, 刘元盛, 常飞翔, 等. 室内环境下UWB与LiDAR融合定位算法研究[J]. 计算机工程与应用, 2021, 57(6): 260-266.
LI Z D, LIU Y S, CHANG F X, et al. Research on UWB and LiDAR fusion positioning algorithm in indoor environment[J]. Computer Engineering and Applications, 2021, 57(6): 260-266.
[30] 李帅鑫, 李广云, 王力, 等. LiDAR/IMU紧耦合的实时定位方法[J], 自动化学报, 2021, 47(6): 1377-1389.
LI S X, LI G Y, WANG L, et al. LiDAR/IMU tightly coupled real-time localization method[J]. Acta Automatica Sinica, 2021, 47(6): 1377-1389.
[31] 董伯麟, 柴旭. 基于IMU/视觉融合的导航定位算法研究[J]. 压电与声光, 2020, 42(5): 724-728.
DONG B L, CHAI X. Research on navigation and localization algorithm based on IMU/vision fusion[J]. Piezoelectricity and Acoustooptics, 2020, 42(5): 724-728.
[32] 章弘凯, 陈年生, 代作晓, 等. 一种多层次数据融合的SLAM定位算法[J]. 机器人, 2021, 43(6): 641-652.
ZHANG H K, CHEN N S, DAI Z X, et al. A multi-level data fusion localization algorithm for SLAM[J]. Robot, 2021, 43(6): 641-652.
[33] 李昀泽. 基于激光雷达的室内机器人SLAM研究[D]. 广州: 华南理工大学, 2017.
LI Y Z. Research on SLAM of indoor robot based on LiDAR[D]. Guangzhou: South China University of Technology, 2017.
[34] 陈小伟. 基于BIM特征的施工机器人定位建图与环境理解方法研究[D]. 天津: 河北工业大学, 2022.
CHENG X W. Research on construction robot localization mapping and environment understanding based on BIM features[D]. Tianjin: Hebei University of Technology, 2022.
[35] PEAVY M, KIM P, OVEDIRAN H, et al. Integration of real-time semantic building map updating with adaptive monte carlo localization (AMCL) for robust indoor mobile robot localization[J]. Advanced Engineering Informatics, 2023, 13(2): 909.
[36] LI C, ZHUANG Y, YAN F. Deep sensor fusion between 2D laser scanner and IMU for mobile robot localization[J]. IEEE Sensors Journal, 2021, 21(6): 8501-8509.
[37] LI H J, MAO Y, YOU W, et al. A neural network approach to indoor mobile robot localization[C]//Proceedings of the 19th International Symposium on Distributed Computing and Applications for Business Engineering and Science, 2020.
[38] ZHAO X, CHEAH C C. BIM-based indoor mobile robot initialization for construction automation using object detection[J]. Automation in Construction, 2022, 146: 104647.
[39] SCHAUB L, PODKOSOVA L, SCHOENAUER C, et al. Point cloud to BIM registration for robot localization and augmented reality[C]//Proceedings of the IEEE International Symposium on Mixed and Augmented Reality, 2022: 77-84.
[40] MOURA M S, RIZZO C, SERRANO D. BIM-based loca-
lization and mapping for mobile robots in construction[C]//Proceedings of the IEEE International Conference on Autonomous Robot Systems and Competitions, 2021: 12-18.
[41] 刘今越, 陈小伟, 贾晓辉, 等. BIM校正累计误差的激光里程计求解方法[J]. 仪器仪表学报, 2022, 43(1): 93-102.
LIU J Y, CHENG X W, JIA X H, et al. The solution method of laser odometer for BIM correction of cumulative error[J]. Chinese Journal of Scientific Instrument, 2022, 43(1): 93-102.
[42] CHEN Y, PERVAN B. Landmark augmentation for mobile robot localization safety[J]. IEEE Robotics and Automation Letters, 2021, 6(1): 119-126.
[43] HALDER S, AFSARI K, KOWSKI J S, et al. Real-time and remote construction progress monitoring with a quadruped robot using augmented reality[J]. Buildings, 2022, 12(11).
[44] 焦传佳. 基于Apriltag图像识别的移动机器人定位研究[J]. 电子测量与仪器学报, 2021, 35(1): 110-119.
JIAO C J. Research on positioning of mobile robot based on AprilTag image recognition[J]. Journal of Electronic Measurement and Instrumentation, 2021, 35(1): 110-119.
[45] KAYHANI N, MCCABE B, ABDELAAL A, et al. Tag-based indoor localization of UAVs in construction environments: opportunities and challenges in practice[C]//Proceedings of the Construction Research Congress, 2020: 226-235.
[46] KAYHANIA N, ZHAO W, MCCABE B, et al. Tag-based visual-inertial localization of unmanned aerial vehicles in indoor construction environments using an on-manifold extended Kalman filter[J]. Automation in Construction, 2022, 135: 1-20.
[47] LI Z Q, HUANG J D. Study on the use of Q-R codes as landmarks for indoor positioning preliminary results[C]//Proceedings of the IEEE/ION Position, Location and Navigation Symposium, 2018: 1270-1276.
[48] 王家恩, 肖献强. 基于 QR 码视觉定位的移动机器人复合导航方法研究[J]. 仪器仪表学报, 2018, 39(8): 230-238.
WANG J N, XIAO X Q. Mobile robot integrated navigation method based on QR code vision positioning[J]. Chinese Journal of Scientific Instrument, 2018, 39(8): 230-238.
[49] 郭金迪. 面向建筑环境的自主打磨移动机器人研究[D]. 沈阳: 东北大学, 2020.
GUO J D. Research on autonomous polishing mobile robot for building environment[D]. Shenyang: Northeastern University, 2020.
[50] 张帅. 地砖铺设机器人系统设计及运动规划研究[D]. 南京: 南京航空航天大学, 2020.
ZHANG S. Research on design and motion planning of robotic floor-tiling system[D]. Nanjing: Nanjing University of Aeronautics and Astronautics, 2020.
[51] LI X, JIANG X, LIU Y. Construction robot localization system based on multi-sensor fusion and 3D construction drawings[C]//Proceedings of the International Conference on Robotics and Biomimetics, 2021: 27-31.
[52] KIM P, PARK J, CHO Y K, et al. UAV-assisted autonomous mobile robot navigation for as-is 3D data collection and registration in cluttered environments[J]. Automation in Construction, 2019, 106: 102918.
[53] KIM P, PARK J, CHO Y K. As-is geometric data collection and 3D visualization through the collaboration between UAV and UGV[C]//Proceedings of the International Symposium on Automation and Robotics in Construction, 2019.
[54] 王舜. 瓷砖铺贴机器人的瓷砖三维空间定位方法研究 [J]. 国外电子测量技术, 2022, 41(10): 85-91.
WANG S. Design of tile three-dimensional space positioning system for floor-tiling robot[J]. Foreign Electronic Measurement Technology, 2022, 41(10): 85-91.
[55] 许镟. 无人推土机系统位姿估计与室外场景建图技术研究[D]. 武汉: 华中科技大学, 2022.
XU X. Research on pose estimation and outdoor scene mapping technology of unmanned bulldozer system[D]. Wuhan: Huazhong University of Science and Technology, 2022.
[56] 赵洋, 刘国良, 田国会, 等. 基于深度学习的视觉SLAM综述[J]. 机器人, 2017, 39(6): 889-896.
ZHAO Y, LIU G L, TIAN G H, et al. A survey of visual SLAM based on deep learning[J]. Robot, 2017, 39(6): 889-896.
[57] MENG T, JING X Y, YAN Z, et al. A survey on machine learning for data fusion[J]. Information Fusion, 2020, 57: 115-129.
[58] 程志伟. 基于力反馈和视觉的板材安装定位研究[D]. 天津: 河北工业大学, 2010.
CHENG Z W. Research on slabstone installation and positioning based on force feedback and vision[D]. Tianjin: Hebei University of Technology, 2010.
[59] 孙建刚, 高娜, 杨冬, 等. 建筑板材安装机器人视觉系统及标定方法研究[J]. 河北工业大学学报, 2013, 42(4): 64-68.
SUN J G, GAO N, YANG D, et al. Research of the building slabstone installing robot vision system and the calibration method[J]. Journal of Hebei University of Technology, 2013, 42(4): 64-68.
[60] WANG W D, ZHAO H P, CHI S Y, et al. A control method for hydraulic manipulators in automatic emulsion filling[J]. Automation in Construction, 2018, 91: 92-99.
[61] ZHAO Q J, LI X F, LU J X, et al. Monocular vision-based parameter estimation for mobile robotic painting[J]. IEEE Transactions on Instrumentation and Measurement, 2018, 68(10): 3589-3599.
[62] TSURUTA T, MIURA K, MIYAGUCHI M. Mobile robot for marking free access floors at construction sites[J]. Automation in Construction, 2019, 107: 102912.

[1]	李俊, 曹林, 张帆, 杜康宁, 郭亚男. 分布统计特征的孪生网络目标跟踪方法[J]. 计算机工程与应用, 2024, 60(8): 213-224.
[2]	刘建华, 王楠, 白明辰. 手机室内场景要素实例化现实增强方法研究进展[J]. 计算机工程与应用, 2024, 60(7): 58-69.
[3]	何文昊, 葛海波, 程梦洋, 安玉, 马赛. 边缘注意力及反向定位的伪装目标检测算法[J]. 计算机工程与应用, 2024, 60(7): 229-237.
[4]	王嘉怡, 陈子洋, 袁小晨, 赵艮平. 面向跨视角地理定位的感知特征融合网络[J]. 计算机工程与应用, 2024, 60(3): 255-262.
[5]	李健, 胡瑞娟, 张克亮, 刘海砚. 基于任务转化的事件抽取通用框架[J]. 计算机工程与应用, 2024, 60(15): 133-142.
[6]	刘瑢琦, 王红雨, 韩佼志. 面向无人机目标的检测与实时跟随[J]. 计算机工程与应用, 2024, 60(11): 319-327.
[7]	刘铭哲, 徐光辉, 唐堂, 钱晓健, 耿明. 激光雷达SLAM算法综述[J]. 计算机工程与应用, 2024, 60(1): 1-14.
[8]	黄志滨, 陈桪. 基于PSO-GA算法的无人机集群森林火灾探查方法[J]. 计算机工程与应用, 2023, 59(9): 289-294.
[9]	刘嘉敏, 陈圣伦, 王智慧, 李豪杰. 标志辅助的多特征融合定位算法[J]. 计算机工程与应用, 2023, 59(6): 187-195.
[10]	彭泽鑫, 曾碧, 刘建圻. 基于全局特征点匹配的全局定位方法[J]. 计算机工程与应用, 2023, 59(3): 264-275.
[11]	张雯玮, 傅启凡, 王营冠, 傅衡成, 魏智, 丁华泽. 改进PSPNet的炮弹火焰分割算法及应用[J]. 计算机工程与应用, 2023, 59(23): 154-164.
[12]	张俊青, 熊玉洁, 孙宪坤, 高永彬. 结合双金字塔特征融合与级联定位的车牌检测[J]. 计算机工程与应用, 2023, 59(2): 240-252.
[13]	高辉, 邓淼磊, 赵文君, 陈法权, 张德贤. 基于弱监督的改进Transformer在人群定位中的应用[J]. 计算机工程与应用, 2023, 59(19): 92-98.
[14]	许春冬, 刘昊, 闵源, 甄雅迪. 基于双重注意力的声音事件定位与检测[J]. 计算机工程与应用, 2023, 59(19): 99-105.
[15]	王静, 胡少毅, 郭苹, 金玉楚. 改进场景坐标回归网络的室内相机重定位方法[J]. 计算机工程与应用, 2023, 59(15): 160-168.