Research on Dynamic Feature Visual Odometry in Road Environment

doi:10.3778/j.issn.1002-8331.2105-0313

Abstract

Abstract: To deal with the navigation and positioning of vehicles in the road environment, a monocular visual odometry based on the dynamic semantic features of the road environment is proposed. A self-supervised convolutional neural network is designed to model the monocular continuous image, no longer rely on artificially designed feature and directly predict the depth map and the pose vector. To deal with the destruction of photometric consistency by dynamic objects in the road environment, it is proposed to use semantic prior to improve the accuracy of visual odometry. Two fully connected layers are designed to estimate the rotation and translation vectors respectively. The experimental results show that the method is comparable to that of the traditional method, and it has superior performance in the road environment.

Key words: visual odometry, semantic segmentation, self-supervision, convolutional neural network, monocular image

摘要： 针对道路环境下移动车辆导航和定位的问题，提出了一种基于道路环境动态语义特征的单目视觉里程计。设计了一个自监督的卷积神经网络，对单目连续图像建模，直接预测深度图和位姿向量，不再依赖人工设计的特征点。针对道路环境下动态物体破坏光度一致性的问题，提出利用语义先验信息提高视觉里程计精度。设计两个全连接层分别估计旋转和平移向量。实验结果表明，该方法得到了与传统视觉里程计比肩的精度，并且在道路环境下具有优越的性能。

关键词: 视觉里程计, 语义分割, 自监督, 卷积神经网络, 单目图像

YANG Binchao, XU Xinying, CHENG Lan, FENG Zhou. Research on Dynamic Feature Visual Odometry in Road Environment[J]. Computer Engineering and Applications, 2022, 58(23): 197-204.

杨斌超, 续欣莹, 程兰, 冯洲. 道路环境下动态特征视觉里程计研究[J]. 计算机工程与应用, 2022, 58(23): 197-204.

References

[1] 祝朝政，何明，杨晟，等.单目视觉里程计研究综述[J].计算机工程与应用，2018，54（7）：20-28.?
ZHU C Z，HE M，YANG S，et al.Survey of monocular visual odometry[J].Computer Engineering and Applications，2018，54（7）：20-28.
[2] 蒋秉川，万刚，李锋，等.机器人超高分辨率立体网格导航地图建模研究[J].系统仿真学报，2017，29（11）：2709-2717.
JIANG B C，WAN G，LI F，et al.Robotic navigation map construction with ultra-high resolution volumetric grid[J].Journal of System Simulation，2017，29（11）：2709-2717.
[3] 曹凯，杨雪梦，姚敏茹，等.基于混合运动估计方法的视觉里程计[J].计算机工程与应用，2019，55（5）：175-180.?
CAO K，YANG X M，YAO M R，et al.Visual odometry via hybrid motion estimation method[J].Computer Engineering and Applications，2019，55（5）：175-180.
[4] ENGEL J，STURM J，CREMERS D.Semi-dense visual odometry for a monocular camera[C]//2014 IEEE International Conference on Computer Vision（ICCV），2013：1449-1456.
[5] YANG X，XUE T，LUO H，et al.Fast and accurate visual odometry from a monocular camera[J].Frontiers of Computer Science，2019，13（6）：1326-1336.
[6] YANG N，STUMBERG L，WANG R，et al.D3VO：deep depth，deep pose and deep uncertainty for monocular visual odometry[C]//2020 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2020：1281-1292.
[7] 张航，陈彬，薛含章，等.基于无人机和LIDAR的三维场景建模研究[J].系统仿真学报，2017，29（9）：1914-1920.
ZHANG H，CHEN B，XUE H Z，et al.Research on 3D scene modeling based on UAV and LIDAR[J].Journal of System Simulation，2017，29（9）：1914-1920.
[8] WANG S，CLARK R，WEN H，et al.DeepVO：towards end-to-end visual odometry with deep recurrent convolutional neural networks[C]//2017 IEEE International Conference on Robotics and Automation（ICRA），2017：2043-2050.
[9] LI R，WANG S，LONG Z，et al.UnDeepVO：monocular visual odometry through unsupervised deep learning[C]//2018 IEEE International Conference on Robotics and Automation（ICRA），2018：7286-7291.
[10] ROMAO G S，REINALDO B，BERNARDES C L.Problem-based learning in primary care at University of Ribeiro Preto（UNAERP） in Brazil[J].Advances in Medical Education & Practice，2018，9：27-30.
[11] SHEN T，LUO Z，LEI Z，et al.Beyond photometric loss for self-supervised ego-motion estimation[C]//2019 International Conference on Robotics and Automation（ICRA），2019：6359-6365.
[12] YIN Z，SHI J.GeoNet：unsupervised learning of dense depth，optical flow and camera pose[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），2018：1983-1992.
[13] VIJAYANARASIMHAN S，RICCO S，SCHMID C，et al.SfM-Net：learning of structure and motion from video[J].
arXiv：1704.07804，2017.
[14] ZEILER M，KRISHNAN D，TAYLOR G W，et al.Deconvolutional networks[C]//2010 IEEE Computer Vision and Pattern Recognition（CVPR），2010：2528-2535.
[15] ZHOU T，BROWN M，SNAVELY N，et al.Unsupervised learning of depth and ego-motion from video[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2017：6612-6619.
[16] WANG X，YU K，DONG C，et al.Recovering realistic texture in image super-resolution by deep spatial feature transform[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），2018：606-615.
[17] CHEN L，ZHU Y，PAPANDREOU G，et al.Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//2018 European Conference on Computer Vision（ECCV），2018.
[18] JADERBERG M，SIMONYAN K，ZISSERMAN A，et al.Spatial transformer networks[C]//2015 Neural Information Processing Systems（NIPS），2015：2017-2025.
[19] WANG Z，BOVIK A C，SHEIKH H R，et al.Image quality assessment：from error visibility to structural similarity[J].IEEE Transactions on Image Processing，2004，13（4）：600-612.
[20] YANG C Y，MA C，YANG M H.Single-image super-resolution：a benchmark[C]//2014 European Conference on Computer Vision（ECCV）.Zurich，Switzerland：Springer，2014：372-386.
[21] LAI W S，HUANG J B，HU Z，et al.A comparative study for single image blind deblurring[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2016：1701-1709.
[22] MAHJOURIAN R，WICKLE M，ANGELOVA A.Unsupervised learning of depth and ego-motion from monocular video using 3D geometric constraints[C]//2018 IEEE Computer Vision and Pattern Recognition（CVPR），2018：5667-5675.
[23] GEIGER A，LENZ P，URTASUN R.Are we ready for autonomous driving?The KITTI vision benchmark suite[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2012：3354-3361.
[24] CORDTS M，OMRAN M，RAMOS S，et al.The cityscapes dataset for semantic urban scene understanding[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2016：3213-3223.
[25] EVERINGHAM M，ESLAMI S M A，VAN GOOL L，et al.The?PASCAL?visual object classes challenge：a retrospective[J].International Journal of Computer Vision，2015，111（1）：98-136.
[26] EIGEN D，PUSHRSCH C，FERGUS R.Depth map prediction from a single image using a multi-scale deep network[C]//2014 Neural Information Processing Systems（NIPS），2014：2366-2374.
[27] KINGMA D，BA J.Adam：a method for stochastic optimization[C]//2015 International Conference on Learning Representations（ICLR），2015.
[28] LIU F，SHEN C，LIN G，et al.Learning depth from single monocular images using deep convolutional neural fields[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2016，38（10）：2024-2039.
[29] YANG Z，WANG P，XU W，et al.Unsupervised learning of geometry with edge-aware depth-normal consistency[J].arXiv：1711.03665，2017.
[30] MURARTAL R，TARDOS J D.ORB-SLAM2：an open-source SLAM system for monocular，stereo，and RGB-D cameras[J].IEEE Transactions on Robotics，2017，33（5）：1255-1262.
[31] MURARTAL R，MONTIEL J M M，TARDOS J D.ORB-SLAM：a versatile and accurate monocular SLAM system[J].IEEE Transactions on Robotics，2015，31（5）：1147-1163.