Accessible Space Detection Based on Monocular Vision in Unstructured Environment

doi:10.3778/j.issn.1002-8331.2106-0464

Abstract

Abstract: Three-dimensional obstacle avoidance mainly adopts 3-D laser radar or deep learning-based obstacle recognition, but the former is expensive, while the latter training cost is high and unstable. In order to realize low-cost three-dimensional space obstacle avoidance scheme, an accessible space detection method based on monocular camera is proposed. The method identifies obstacles by feature points. To solve the scale inconsistency in the monocular SLAM, the proposed method imposes certain constraints on camera height and rotation plane. Obstacle distance solver and cost solver are designed to calculate visual cost map and distinguish accessible space using the feature points of the front space. The proposed method only requires a low-cost monocular camera to complete the accessible space detection task, and can easily transplant to lightweight mobile devices. The calculated visual cost map is also conducive to the subsequent path planning task. Experiments on the KITTI dataset show that the average operation speed of the proposed method can reach 20 frame/s, which meets the requirements of real-time obstacle avoidance for the robot. The scale recovery error for monocular SLAM is 2.5%~4.9%.

Key words: unstructured environment, monocular vision, visual cost map, accessible space

摘要： 目前三维避障主要采用三维激光雷达或者基于深度学习的障碍物识别，但前者价格昂贵，后者训练成本高且不稳定。为了稳定、鲁棒地实现低成本三维空间避障方案，提出了一种基于单目相机的可通行区域检测方法。该方法利用特征点标识障碍物，通过对相机高度和旋转平面加以约束，解决单目SLAM中的尺度不一致问题，并设计了障碍物距离求解器和代价求解器对小车前方区域特征点进行处理，计算出视觉代价地图，划分可通行区域。该方法优势在于仅需要低成本的单目相机便可完成可通行区域检测任务，可方便地移植到轻量级的移动设备，计算得到的视觉代价地图亦有利于小车后续的路径规划任务。在KITTI数据集上进行的实验表明，该方法的平均运算速度能达到20?frame/s，能满足小车实时避障的要求，对于单目SLAM的尺度恢复误差为2.5%~4.9%。

关键词: 非结构化环境, 单目视觉, 视觉代价地图, 可通行区域

GUO Zhixing, ZENG Bi, LIU Jianqi, CHEN Wenxuan, WANG Junfeng. Accessible Space Detection Based on Monocular Vision in Unstructured Environment[J]. Computer Engineering and Applications, 2022, 58(24): 298-306.

郭植星, 曾碧, 刘建圻, 陈文轩, 王俊丰. 非结构化环境下的单目视觉可通行区域检测[J]. 计算机工程与应用, 2022, 58(24): 298-306.

References

[1] YU H，ZHANG F，HUANG P，et al.Autonomous obstacle avoidance for UAV based on fusion of radar and monocular camera[C]//2020 IEEE/RSJ International Conference on Intelligent Robots and Systems，2020.
[2] 袁红斌，曹会群，欧群雍.基于激光测距雷达和机器视觉的障碍物检测[J].现代雷达，2021，43（5）：57-62.
YUAN H B，CAO H Q，OU Q Y.Obstacle detection based on lidar and machine vision[J].Modern Radar，2021，43（5）：57-62.
[3] 赵一兵，王荣本，李琳辉，等.基于多传感器信息的前方障碍物检测[J].计算机工程与应用，2007，43（26）：174-177.
ZHAO Y B，WANG R B，LI L H，et al.Approach of obstacle detection based on laser sensor and single camera[J].Computer Engineering and Applications，2007，43（26）：174-177.
[4] MUR-ARTAL R，TARDóS J D.ORB-SLAM2：an open-source slam system for monocular，stereo，and RGB-D cameras[J].IEEE Transactions on Robotics，2017，33（5）：1-8.
[5] ENGEL J，KOLTUN V，CREMERS D.Direct sparse odometry[J].IEEE Transactions on Pattern Analysis & Machine Intelligence，2018，40（3）：611-625.
[6] ENGEL J，SCHPS T，CREMERS D.LSD-SLAM：large-scale direct monocular SLAM[C]//European Conference on Computer Vision.Cham：Springer，2014.
[7] 刘浩敏，章国锋，鲍虎军.基于单目视觉的同时定位与地图构建方法综述[J].计算机辅助设计与图形学学报，2016，28（6）：855-868.
LIU H M，ZHANG G F，BAO H J.A survey of monocular simultaneous localization and mapping[J].Journal of Computer-Aided Design and Graphics，2016，28（6）：855-868.
[8] 夏梦琪，郝琨，赵璐.基于全卷积编解码网络的单目图像深度估计[J].计算机工程与应用，2021，57（14）：231-236.
XIA M Q，HAO K，ZHAO L.Monocular image depth estimation based on fully convolutional encoder-decoder network[J].Computer Engineering and Applications，2021，57（14）：231-236.
[9] EIGEN D，PUHRSCH C，FERGUS R.Depth map prediction from a single image using a multi-scale deep network[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems，2014.
[10] EIGEN D，FERGUS R.Predicting depth，surface normals and semantic labels with a common multi-scale convolutional architecture[C]//2015 IEEE International Conference on Computer Vision，2014.
[11] WANG R，PIZER S M，FRAHM J M.Recurrent neural network for （un-）supervised learning of monocular videovisual odometry and depth[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019.
[12] RIEGLER G，LIAO Y，DONNE S，et al.Connecting the dots：learning representations for active monocular depth estimation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019.
[13] LIANG B，PEARS N，CHEN Z.Affine height landscapes for monocular mobile robot obstacle avoidance[C]//Proceedings of Intelligent Autonomous Systems，2004：863-872.
[14] ZHOU J，LI B.Homography-based ground detection for a mobile robot platform using a single camera[C]//IEEE International Conference on Robotics & Automation，2015.
[15] JIN Z，LI B.Robust ground plane detection with normalized homography in monocular sequences from a robot platform[C]//IEEE International Conference on Image Processing，2007.
[16] 郑争光，任小洪，程亚军，等.未知环境下单目视觉移动机器人路径规划[J].计算机工程与应用，2018，54（3）：255-258.
ZHENG Z G，REN X H，CHENG Y J，et al.Monocular visual mobile robot path planning in unknown environment[J].Computer Engineering and Applications，2018，54（3）：255-258.
[17] LENSER S，VELOSO M.Visual sonar：fast obstacle avoidance using monocular vision[C]//IEEE/RSJ International Conference on Intelligent Robots & Systems，2003.
[18] OHYA A，KOSAKA A，KAK A.Vision-based navigation by a mobile robot with obstacle avoidance using single-camera vision and ultrasonic sensing[J].IEEE Transactions on Robotics & Automation，1998，14（6）：969-978.
[19] LIN C H，JIANG S Y，PU Y J，et al.Robust ground plane detection for obstacle avoidance of mobile robots using a monocular camera[C]//2010 IEEE/RSJ International Conference on Intelligent Robots and Systems，2010.
[20] SOUHILA K，KARIM A.Optical flow based robot obstacle avoidance[J].International Journal of Advanced Robotic Systems，2008，4（1）.
[21] CHERUBINI A，COLAFRANCESCO M，ORIOLO G，et al.Comparing appearance-based controllers for nonholonomic navigation from a visual memory[C]//ICRA 2009 Workshop on Safe Navigation in Open and Dynamic Environments：Application to Autonomous Vehicles，2009.
[22] COURBON J，MEZOUAR Y，MARTINET P.Autonomous navigation of vehicles from a visual memory using a generic camera model[J].IEEE Transactions on Intelligent Transportation Systems，2009，10（3）：392-402.
[23] LAMIRAUX F，LEFEBVRE O.Sensor-based trajectory deformation：application to reactive navigation of nonholonomic robots[M]//CHESI G，HASHIMOTO K.Visual servoing via advanced numerical methods.[S.l.]：Springer LNCIS Series，2010：315-334.
[24] ZHU Y，MOTTAGHI R，KOLVE E，et al.Target-driven visual navigation in indoor scenes using deep reinforcement learning[C]//2017 IEEE International Conference on Robotics and Automation，2017：3357-3364.
[25] YANG W，WANG X L，FARHADI A，et al.Visual semantic navigation using scenepriors[J].arXiv：1810.06543，2018.
[26] 李鹏，阮晓钢，朱晓庆，等.基于深度强化学习的区域化视觉导航方法[J].上海交通大学学报，2021，55（5）：575-585.
LI P，RUAN X G，ZHU X Q，et al.A regionalalization visual navigation method based on deep reinforcement learning[J].Journal of Shanghai Jiaotong University，2021，55（5）：575-585.