Research of Outdoor Robot Localization Method on Fusion of Multi-camera and IMU

doi:10.3778/j.issn.1002-8331.2008-0257

Abstract

Abstract: Improving the accuracy and robustness of SLAM algorithm is the key to solve the autonomous localization problem of outdoor mobile robot. MCSI-VINS, a method which integrates multi-camera and IMU, is proposed to solve the problems of monocular camera being vulnerable to occlusion in outdoor complex environment, camera moving too fast, image blurriness, algorithm accuracy and robustness decline under pure robot rotation, and cumulative offset of low-precision IMU. The layout of multiple cameras can make up for the case that the single eye is blocked, the rich visual information can make up for the accumulated error caused by the low-precision IMU, and the IMU fusion can solve the localization problem in the case that the camera moves too fast, the image is blurred, and the rotation is pure. It mainly introduces the data preprocessing, initialization and optimization localization module of the MCSI-VINS. This method is based on the VINS-MONO framework, but different from it, a method to build buffer based on queue data structure is proposed in the data pretreatment stage, and an initialization method based on the main camera is proposed in the front end. In the back end, the error term of other cameras is added and the time stamp is optimized. A mobile robot experiment platform is built based on ROS. Experiments verify that the proposed algorithm is more robust and accurate than the classical monocular and binocular VINS algorithms.

Key words: multi-camera, inertial measurement unit（IMU）, outdoor mobile robot, robot operating system（ROS）

摘要： 提高SLAM（simultaneous localization and mapping）算法的精度和鲁棒性是解决室外移动机器人自主定位问题的关键。针对单目相机在室外复杂环境下易受到遮挡、相机移动过快、图像模糊以及机器人纯旋转下算法精度和鲁棒性下降、低精度IMU（inertial measurement unit）的累积偏移等问题，提出了一种多目相机与IMU融合的方案——MCSI-VINS（multi-camera and single-IMU-visual inertial navigation system）。多相机的布置可弥补单目受到遮挡的情形，丰富的视觉信息可弥补低精度IMU带来的累积误差，融合IMU可解决相机移动过快、图像模糊、纯旋转情形下的定位问题。MCSI-VINS包括数据预处理、初始化和优化定位模块。此方法基于VINS-MONO（multi-camera and single-IMU-monocular）框架，而与VINS-MONO不同的是，在数据预处理阶段提出了一种基于队列数据结构构建缓冲区的方法，前端提出了一种基于主相机的初始化方法，在后端中添加了其他相机的误差项并对时间戳进行了优化。用基于ROS（robot operating system）的移动机器人进行实验表明，提出的算法较经典的单目、双目VINS算法相比，鲁棒性更强，精度更高。

关键词: 多目相机, 惯性测量单元（IMU）, 室外移动机器人, 机器人操作系统（ROS）

DONG Rong, LI Maohai, LIN Rui, LIU Shiqi, DING Wen. Research of Outdoor Robot Localization Method on Fusion of Multi-camera and IMU[J]. Computer Engineering and Applications, 2022, 58(3): 289-296.

董荣, 厉茂海, 林睿, 刘仕琦, 丁文. 多相机与IMU融合的室外机器人定位方法研究[J]. 计算机工程与应用, 2022, 58(3): 289-296.

References

[1] DAVISON A J，REID I D，MOLTON N D，et al.MonoSLAM：real-time single camera SLAM[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2007，29（6）：1052-1067.
[2] KLEIN G，MURRAY D.Parallel tracking and mapping for small AR workspaces[C]//2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality，2007：225-234.
[3] NEWCOMBE R A，LOVEGROVE S J，DAVISON A J.DTAM：densetracking and mapping in real-time[C]//Proceedings of IEEE International Conference on Computer Vision.Los Alamitos：IEEE Computer Society Press，2011：2320-2327.
[4] ENGEL J，SCH T，CREMERS D.LSD-SLAM：large-scale direct monocular SLAM[C]//European Conference on Computer Vision.[S.l.]：Springer International Publishing，2014.
[5] FORSTER C，PIZZOLI M，SCARAMUZZA D.SVO：fast semi-direct monocular visual odometry[C]//Proceedings of IEEE International Conference on Robotics and Automation.Los Alamitos：IEEE Computer Society Press，2014：15-22.
[6] ENGEL J，KOLTUN V，CREMERS D.Direct sparse odometry[J].IEEE Transactions on Pattern Analysis & Machine Intelligence，2018，40（3）：611-625.
[7] MUR-ARTAL R，MONTIEL J M M，TARDOS J D.ORB-SLAM：a versatile and accurate monocular SLAM system[J].IEEE Transactions on Robotics，2017，31（5）：1147-1163.
[8] MUR-ARTAL R，TARDOS J D.ORB-SLAM2：an open-source SLAM system for monocular，stereo and RGB-D cameras[J].IEEE Transactions on Robotics，2017，33（5）：1255-1262.
[9] PLESS R.Using many cameras as one[C]//IEEE Computer Society Conference on Computer Vision and Pattern Recognition，2003.
[10] MOHAMMAD EHAB MOHAMMAD R.Multiple camera pose estimation[D].Hongkong：The Chinese University of Hongkong，2008.
[11] KAESS M，DELLAERT F.Visual SLAM with a multi-camera rig[R].Georgia Institute of Technology，2006.
[12] HARRIS C G，STEPHENS M.A combined corner and edge detector[C]//Alvey Vision Conference，1988.
[13] HEE LEE G，FAUNDORFER F，POLLEFEYS M.Motion estimation for self-driving cars with a generalized camera[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2013：2746-2753.
[14] HARMAT A，SHARF I，TRENTINI M.Parallel tracking and mapping with multiple cameras on an unmanned aerial vehicle[C]//International Conference on Intelligent Robotics and Applications，2012：421-432.
[15] HARMAT A，TRENTINI M，SHARF I.Multi-camera tracking and mapping for unmanned aerial vehicles in unstructured environments[J].Journal of Intelligent & Robotic Systems，2015，78（2）：291-317.
[16] TRIBOU M J.Multi-camera parallel tracking and mapping with non-overlapping fields of view[J].International Journal of Robotics Research，2015，34（2）：1480-1500.
[17] URBAN S，HINZ S.MultiCol-SLAM-a modular real-time multi-camera SLAM system[J].arXiv：1610.07336，2016.
[18] YANG S，SCHERER S A，YI X，et al.Multi-camera visual SLAM for autonomous navigation of micro aerial vehicles[J].Robotics and Autonomous Systems，2017，93：116-134.
[19] YANG S，SCHERER S A，ZELL A，et al.Visual SLAM for autonomous MAVs with dual cameras[C]//IEEE International Conference on Robotics and Automation，2014：5227-5232.
[20] LYNEN S，ACHTELIK M W，WEISS S，et al.A robust and modular multi-sensor fusion approach applied to MAV navigation[C]//IEEE/RSJ International Conference on Intelligent Robots & Systems，2013.
[21] LEUTENEGGER S，LYNEN S，BOSSE M，et al.Keyframe-based visual-inertial odometry using nonlinear optimization[J].The International Journal of Robotics Research，2015，34（3）：314-334.
[22] MOURIKIS A I，ROUMELIOTIS S I.A multi-state constraint Kalman filter for vision-aided inertial navigation[C]//IEEE International Conference on Robotics & Automation，2007.
[23] BLOESCH M，OMARI S，HUTTER M，et al.Robust visual inertial odometry using a direct EKF-based approach[C]//2015 IEEE/RSJ International Conference on Intelligent Robots and Systems（IROS），2015：298-304.
[24] QIN T，LI P L，SHEN S J，et al.VINS-Mono：a robust and versatile monocular visual-inertial state estimator[J].IEEE Transactions on Robotics，2018，34（4）：1004-1020.