Visual SLAM Loop Closure Algorithm Based on Semantic and Geometric Consistency

doi:10.3778/j.issn.1002-8331.2307-0114

Abstract

Abstract: Loop closure is the key to eliminate the cumulative error in simultaneous localization and mapping (SLAM) system. When the illumination conditions or the viewing angle change greatly, the traditional appearance-based loop closure methods often fail. Aiming at this situation, an object-level loop closure method is proposed based on ORBSLAM2 framework. Firstly, semantic information obtained by object detection and the feature point information are used to construct object-level semantic map. Then semantic map is abstracted into topological map and landmarks into nodes, node information is described by color histogram, and combined geometric relationship between nodes, a graph matching method is proposed to realize loop detection based on semantic and geometric consistency constraints. When a loop is detected, the loop is corrected by object alignment. Finally, experiments are carried out on the published TUM and USTC data sets, and the results show that the accuracy of the proposed system is 49.58% higher than that of ORBSLAM2 on average, and the constructed semantic map shows a good positioning effect.

Key words: visual simultaneous localization and mapping, loop detection, color histogram, geometric consistency, loop correction

摘要： 回环检测是消除同时定位与地图构建（simultaneous localization and mapping，SLAM）系统中累计误差的关键所在，在光照条件或视角变化较大的情况下，传统的基于外观的回环检测方法往往失效。针对这种情况，在ORBSLAM2的框架基础上提出一种物体级的回环检测方法。利用目标检测获得的语义信息和特征点信息构建物体级语义地图。将语义地图抽象成拓扑图并将地标抽象成节点，用颜色直方图描述节点信息，结合节点间的几何关系，基于语义和几何一致性约束，提出一种图匹配方法实现回环检测。当检测到回环时，通过物体对齐的方式进行回环校正。在公开的TUM和USTC数据集上进行实验，结果表明提出的系统精度较ORBSLAM2平均提高了49.58%，并且构建的语义地图显示出良好的定位效果。

关键词: 视觉同时定位与地图构建, 回环检测, 颜色直方图, 几何一致性, 回环校正

ZHANG Gan, ZHOU Fei, ZHANG Kuo, LI Jiahui. Visual SLAM Loop Closure Algorithm Based on Semantic and Geometric Consistency[J]. Computer Engineering and Applications, 2024, 60(20): 180-188.

张干, 周非, 张阔, 李嘉辉. 基于语义和几何一致性的视觉SLAM回环检测算法[J]. 计算机工程与应用, 2024, 60(20): 180-188.

References

[1] CHEN K, ZHANG J, LIU J, et al. Semantic visual simultaneous localization and mapping: a survey[J]. arXiv:2209.06428, 2022.
[2] 高贵, 伍宣衡, 王忠美, 等. V-SLAM深度学习闭环检测研究进展与展望[J]. 计算机工程与应用, 2022, 58(11): 47-59.
GAO G, WU X H, WANG Z M, et al. Research progress and prospect of V-SLAM deep learning loop closure detection[J]. Computer Engineering and Applications, 2022, 58(11): 47-59.
[3] QIN T, LI P, SHEN S. VINS-Mono: a robust and versatile monocular visual-inertial state estimator[J]. IEEE Transactions on Robotics, 2018, 34(4): 1004-1020.
[4] MUR-ARTAL R, TARDóS J D. ORB-SLAM2: an open-source SLAM system for monocular, stereo, and RGB-D cameras[J]. IEEE Transactions on Robotics, 2017, 33(5): 1255-1262.
[5] CAMPOS C, ELVIRA R, RODRíGUEZ J J G, et al. ORB-SLAM3: an accurate open-source library for visual, visual-inertial, and multimap SLAM[J]. IEEE Transactions on Robotics, 2021, 37(6): 1874-1890.
[6] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60: 91-110.
[7] RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB: an efficient alternative to SIFT or SURF[C]//Proceedings of the 2011 International Conference on Computer Vision, 2011: 2564-2571.
[8] OLIVA A, TORRALBA A. Building the gist of a scene: the role of global image features in recognition[J]. Progress in Brain Research, 2006, 155: 23-36.
[9] LI J, MEGER D, DUDEK G. Semantic mapping for view-invariant relocalization[C]//Proceedings of the 2019 International Conference on Robotics and Automation, 2019: 7108-7115.
[10] YANG S, SCHERER S. CubeSLAM: monocular 3-D object SLAM[J]. IEEE Transactions on Robotics, 2019, 35(4): 925-938.
[11] HAN X, YANG L. SQ-SLAM: monocular semantic SLAM based on superquadric object representation[J]. arXiv:2209.
10817, 2022.
[12] SHAN M, FENG Q, ATANASOV N. OrcVIO: object residual constrained visual-inertial odometry[C]//Proceedings of the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2020: 5104-5111.
[13] LI J, KOREITEM K, MEGER D, et al. View-invariant loop closure with oriented semantic landmarks[C]//Proceedings of the 2020 IEEE International Conference on Robotics and Automation, 2020: 7943-7949.
[14] XU K, WANG C, CHEN C, et al. AirCode: a robust object encoding method[J]. IEEE Robotics and Automation Letters, 2022, 7(2): 1816-1823.
[15] KIM J J Y, URSCHLER M, RIDDLE P J, et al. Closing the loop: graph networks to unify semantic objects and visual features for multi-object scenes[C]//Proceedings of the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2022: 4352-4358.
[16] GAWEL A, DEL DON C, SIEGWART R, et al. X-view: graph-based semantic multi-view localization[J]. IEEE Robotics and Automation Letters, 2018, 3(3): 1687-1694.
[17] LIU Y, PETILLOT Y, LANE D, et al. Global localization with object-level semantics and topology[C]//Proceedings of the 2019 International Conference on Robotics and Automation, 2019: 4909-4915.
[18] GUO X, HU J, CHEN J, et al. Semantic histogram based graph matching for real-time multi-robot global localization in large scale environment[J]. IEEE Robotics and Automation Letters, 2021, 6(4): 8349-8356.
[19] ZHOU B, MENG Y, KAI F. Object-based loop closure with directional histogram descriptor[C]//Proceedings of the 2022 IEEE 18th International Conference on Automation Science and Engineering, 2022: 1346-1351.
[20] WANG C Y, BOCHKOVSKIY A, LIAO H Y M. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023: 7464-7475.
[21] NICHOLSON L, MILFORD M, SüNDERHAUF N. QuadricSLAM: dual quadrics from object detections as landmarks in object-oriented SLAM[J]. IEEE Robotics and Automation Letters, 2018, 4(1): 1-8.
[22] WU Y, ZHANG Y, ZHU D, et al. EAO-SLAM: monocular semi-dense object SLAM based on ensemble data association[C]//Proceedings of the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2020: 4966-4973.
[23] HARIRI S, KIND M C, BRUNNER R J. Extended isolation forest[J]. IEEE Transactions on Knowledge and Data Engineering, 2019, 33(4): 1479-1489.
[24] 尤波, 徐义飞, 李彬, 等. 基于轮廓信息与颜色直方图的图像匹配[J]. 自动化技术与应用, 2019, 38(2): 145-150.
YOU B, XU Y F, LI B, et al. Image matching based on contour Information and color histogram[J]. Techniques of Automation and Applications, 2019, 38(2): 145-150.
[25] STURM J, ENGELHARD N, ENDRES F, et al. A benchmark for the evaluation of RGB-D SLAM systems[C]//Proceedings of the 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012: 573-580.
[26] LIN S, WANG J, XU M, et al. Topology aware object-level semantic mapping towards more robust loop closure[J]. IEEE Robotics and Automation Letters, 2021, 6(4): 7041-7048.