Computer Engineering and Applications ›› 2022, Vol. 58 ›› Issue (20): 1-15.DOI: 10.3778/j.issn.1002-8331.2203-0480
• Research Hotspots and Reviews • Previous Articles Next Articles
ZHI Henghui, YIN Chenyang, LI Huibin
Online:
2022-10-15
Published:
2022-10-15
职恒辉,尹晨阳,李慧斌
ZHI Henghui, YIN Chenyang, LI Huibin. Review of Visual Odometry Methods Based on Deep Learning[J]. Computer Engineering and Applications, 2022, 58(20): 1-15.
职恒辉, 尹晨阳, 李慧斌. 基于深度学习的视觉里程计方法综述[J]. 计算机工程与应用, 2022, 58(20): 1-15.
Add to citation manager EndNote|Ris|BibTeX
URL: http://cea.ceaj.org/EN/10.3778/j.issn.1002-8331.2203-0480
[1] DURRANTWHYTE H,BAILEY T.Simultaneous localization and mapping:part I[J].IEEE Robotics & Automation Magazine,2006,13(3):108-117. [2] MORAVEC H P.Obstacle avoidance and navigation in the real world by a seeing robot rover[D].Palo Alto:Stanford University,1980. [3] MATTHIES L,SHAFER S.Error modeling in stereo navigation[J].IEEE Journal on Robotics and Automation,1987,3(3):239-248. [4] NISTER D,NARODITSKY O,BERGEN J,et al.Visual odometry[C]//Proc of IEEE Conference on Computer Vision and Pattern Recognition,2004:652-659. [5] MATTHIES L,MAIMONE M,JOHNSON A,et al.Computer visionon mars[J].International Journal of Computer Vision,2007,75(1):67-92. [6] MALLESON C,GILBERT A,TRUMBLE M,et al.Real-time full body motion capture from video and IMUs[C]//Proceedings of International Conference on 3D Vision,2017. [7] WANG T,LING H.Gracker:a graph-based planar object tracker[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2018,40(6):1494-1501. [8] CRAIGHEAD J,MURPHY R,BURKE J,et al.A survey of commercial and open source unmanned vehicle simulators[C]//Proceedings 2007 IEEE International Conference on Robotics and Automation,2007:852-857. [9] WANG S,CLARK R,WEN H,et al.End-to-end,sequence-to-sequence probabilistic visual odometry through deep neural networks[J].The International Journal of Robotics Research,2018,37(4/5):513-542. [10] 李宇波,朱效洲,卢惠民,等.视觉里程计技术综述[J].计算机应用研究,2012,29(8):2801-2805. LI Y B,ZHU X Z,LU H M,et al.Review on visual odometry technology[J].Application Research of Computers,2012,29(8):2801-2805. [11] FUENTES-PACHECO J,RUIZ-ASCENCIO J,RENDóN-MANCHA J M.Visual simultaneous localization and mapping:a survey[J].Artificial Intelligence Review,2015,43(1):55-81. [12] HE M,ZHU C,HUANG Q,et al.A review of monocular visual odometry[J].Visual Computer,2019,36(5):1053-1065. [13] LI C L,SHANG J N,LI F.Summary of the development of odometer technology[J].Software Guide,2019,18(12):6-10. [14] 祝朝政,何明,杨晟,等.单目视觉里程计研究综述[J].计算机工程与应用,2018,54(7):20-28. ZHU C Z,HE M,YANG S,et al.Survey of monocular visual odometry[J].Computer Engineering and Applications,2018,54(7):20-28. [15] 赵洋,刘国良,田国会,等.基于深度学习的视觉SLAM综述[J].机器人,2017,39(6):889-896. ZHAO Y,LIU G L,TIAN G H,et al.A survey of visual SLAM based on deep learning[J].Robot,2017,39(6):889-896. [16] 刘瑞军,王向上,张晨,等.基于深度学习的视觉SLAM综述[J].系统仿真学报,2020,32(7):1244-1256. LIU R J,WANG X S,ZHANG C,et al.A survey on visual SLAM based on deep learning[J].Journal of System Simulation,2020,32(7):1244-1256. [17] 李少朋,张涛.深度学习在视觉SLAM中应用综述[J].空间控制技术与应用,2019,45(2):1-10. LI S P,ZHANG T.A survey of deep learning application in visual SLAM[J].Aerospace Control and Application,2019,45(2):1-10. [18] 刘旺.基于深度学习的视觉里程计技术研究[D].成都:电子科技大学,2020. LIU W.Research on visual odometry technology based on deep learning[D].Chengdu:University of Electronic Science and Technology of China,2020. [19] 陈涛,范林坤,李旭川,等.基于深度学习的智能车辆视觉里程计技术发展综述[J].汽车技术,2021(1):10. CHEN T,FAN L K,LI X C,et al.Review on the development of deep learning-based vision odometer technologies for intelligent vehicles[J].Automobile Technology,2021(1):10. [20] SHI J.Good features to track[C]//1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition,1994:593-600. [21] LOWE D G.Distinctive image features from scale-invariant key points[J].International Journal of Computer Vision,2004,60(2):91-110. [22] BAY H,TUYTELAARS T,GOOL L V.SURF:speeded up robust features[C]//Proceedings of European Conference on Computer Vision,2006:404-417. [23] RUBLEE E,RABAUD V,KONOLIGE K,et al.ORB:an efficient alternative to SIFT or SURF[C]//Proceedings of IEEE International Conference on Computer Vision,2012:2564-2571. [24] LEUTENEGGER S,CHLI M,SIEGWART R Y.BRISK:binary robust invariant scalable keypoints[C]//Proceedings of International Conference on Computer Vision,2011:2548-2555. [25] DAVISON A J,REID L D,MOLTON N D.MonoSLAM:real-time single camera SLAM[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2007,29(6):1052-1067. [26] KLEIN G,MURRAY D.Parallel tracking and mapping for small AR workspaces[C]//IEEE & ACM International Symposium on Mixed & Augmented Reality,2008. [27] TRIGGS B,MCLAUCHLAN P F,HARTLEY R I,et al.Bundle adjust ment a modern synthesis[C]//Proceedings of International Workshop on Vision Algorithms:Theory and Practice,1999:298-372. [28] MUR-ARTAL R,MONTIEL J M,TARDOS J D.ORB-SLAM:a versatile and accurate monocular SLAM system[J].IEEE Transactions on Robotics,2015,31(5):1147-1163. [29] MUR-ARTAL R,TARDOS J D.ORB-SLAM2:an open- source SLAM system for monocular,stereo and RGB-D cameras[J].IEEE Transactions on Robotics,2017,33(5):1255-1262. [30] CAMPOS C,ELVIRA R,RODRíGUEZ J J G,et al.ORB-SLAM3:an accurate open-source library for visual,visual-inertial,and multimap SLAM[J].IEEE Transactions on Robotics,2021,37(6):1874-1890. [31] NEWCOMBE R A,LOVEGROVE S J,DAVISON A J.DTAM:dense tracking and mapping in real-time[C]//2011 International Conference on Computer Vision,2011:2320-2327. [32] ENGEL J,SCH?PS T,CREMERS D.LSD-SLAM:large-scale direct monocular SLAM[C]//European Conference on Computer Vision.Cham:Springer,2014:834-849. [33] ENGEL J,KOLTUN V,CREMERS D.Direct sparse odometry[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2017,40(3):611-625. [34] KENDALL A,GRIMES M,CIPOLLA R.Posenet:a convolutional network for real-time 6-dof camera relocali- zation[C]//Proceedings of the IEEE International Conference on Computer Vision,2015:2938-2946. [35] SZEGEDY C,LIU W,JIA Y,et al.Going deeper with con-volutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE,2015:1-9. [36] KENDALL A,CIPOLLA R.Geometric loss functions for camera pose regression with deep learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2017:5974-5983. [37] SAPUTRA M,GUSMAO P D,WANG S,et al.Learning monocular visual odometry through geometry-aware curriculum learning[C]//2019 International Conference on Robotics and Automation(ICRA),2019. [38] WALCH F,HAZIRBAS C,LEAL-TAIXE L,et al.Image-based localization using lstms for structured feature correlation[C]//Proceedings of the IEEE International Conference on Computer Vision,2017:627-637. [39] WANG S,CLARK R,WEN H,et al.Deepvo:towards end-to-end visual odometry with deep recurrent convolutional neural networks[C]//2017 IEEE International Conference on Robotics and Automation(ICRA),2017:2043-2050. [40] CLARK R,WANG S,MARKHAM A,et al.Vidloc:a deep spatio-temporal model for 6-dof video-clip relocalization[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2017:6856-6864. [41] JIAO J,JIAO J C,MO Y,et al.Magicvo:end-to-end mono- cular visual odometry through deep bi-directional recurrent convolutional neural network[J].arXiv:1811.10964,2018. [42] COSTANTE G,MANCINI M.Uncertainty estimation for data-driven visual odometry[J].IEEE Transactions on Robotics,2020,36(6):1738-1757. [43] KENDALL A,CIPOLLA R.Modelling uncertainty in deep learning for camera relocalization[C]//IEEE International Conference on Robotics and Automation(ICRA),2016:4762-4769. [44] KAYGUSUZ N,MENDEZ O,BOWDEN R.MDN-VO:estimating visual odometry with confidence[C]//2021 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS),2021:3528-3533. [45] KOUMIS A S,PREISS J A,SUKHATME G S.Estimating metric scale visual odometry from videos using 3D convolutional networks[C]//2019 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS),2020. [46] LIN Y,LIU Z,HUANG J,et al.Deep global-relative networks for end-to-end 6-dof visual localization and odometry[C]//Pacific Rim International Conference on Artificial Intelligence.Cham:Springer,2019:454-467. [47] PERETROUKHIN V,CLEMENT L,KELLY J.Reducing drift in visual odometry by inferring sun direction using a Bayesian convolutional neural network[C]//2017 IEEE International Conference on Robotics and Automation(ICRA),2017:2035-2042. [48] COSTANTE G,CIARFUGLIA T A.LS-VO:learning dense optical subspace for robust visual odometry estimation[J].IEEE Robotics and Automation Letters,2018,3(3):1735-1742. [49] HUANG Y,ZHAO B,GAO C,et al.Learning optical flow with R-CNN for visual odometry[C]//2021 IEEE International Conference on Robotics and Automation(ICRA),2021:14410-14416. [50] XUE F,WANG X,WANG J,et al.Deep visual odometry with adaptive memory[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,44(2):940-954. [51] KUO X Y,LIU C,LIN K C,et al.Dynamic attention-based visual odometry[C]//2020 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS),2020. [52] XUE F,WANG Q,WANG X,et al.Guided feature selection for deep visual odometry[C]//Asian Conference on Computer Vision.Cham:Springer,2018:293-308. [53] JADERBERG M,SIMONYAN K,ZISSERMAN A,et al.Spatial transformer networks[C]//Advances in Neural Information Processing Systems,2015:1329-1336. [54] GARG R,VIJAYKUMAR B G,CARNEIRO G.Unsupervised CNN for single view depth estimation:geometry to the rescue[C]//14th European Conference on Computer Vision(ECCV),2016:740-756. [55] ZHOU T,BROWN M,SNAVELY N,et al.Unsupervised learning of depth and ego-motion from video[C]//30th IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR),2017:1122-1131. [56] VIJAYANARASIMHAN S,RICCO S,SCHMID C,et al.Sfm-net:learning of structure and motion from video[J].arXiv:1704.07804,2017. [57] WANG R,PIZER S M,FRAHM J M.Recurrent neural network for (un-)supervised learning of monocular video visual odometry and depth[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR),2019. [58] YANG N,STUMBERG L,WANG R,et al.D3vo:deep depth,deep pose and deep uncertainty for monocular visual odometry[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2020:1281-1292. [59] KIM U H,KIM S H,KIM J H.Simvodis:simultaneous visual odometry,object detection,and instance segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,44(1):428-441. [60] ZHANG J,SUI W,WANG X,et al.Deep online correction for monocular visual odometry[C]//2021 IEEE International Conference on Robotics and Automation(ICRA),2021:14396-14402. [61] LIANG Z,WANG Q,YU Y.Deep unsupervised learning based visual odometry with multi-scale matching and latent feature constraint[C]//2021 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS),2021:2239-2246. [62] GODARD C,MAC AODHA O,BROSTOW G J.Unsuper vised monocular depth estimate with left-right consistency[C]//30th IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR),2017:6602-6611. [63] WANG Z,BOVIK A C,SHEIKH H R,et al.Image quality assessment:from error visibility to structural similarity[J].IEEE Transactions on Image Processing,2004,13(4):600-612. [64] ALMALIOGLU Y,SAPUTRA M,GUSMAO P,et al.GANVO:unsupervised deep monocular visual odometry and depth estimation with generative adversarial networks[C]//2019 International Conference on Robotics and Automation(ICRA),2019. [65] GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial nets[C]//Advances in Neural Information Processing Systems,2014. [66] LI S,XUE F,WANG X,et al.Sequential adversarial learning for self-supervised deep visual odometry[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision,2019:2851-2860. [67] FENG T,GU D.SGANVO:unsupervised deep visual odometry and depth estimation with stacked generative adversarial networks[J].IEEE Robotics and Automation Letters,2019,4(4):4431-4437. [68] IYER G,KRISHNA MURTHY J,GUPTA G,et al.Geometric consistency for self-supervised end-to-end visual odometry[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops,2018:267-275. [69] ZHAN H,GARG R,WEERASEKERA C S,et al.Unsupervised learning of monocular depth estimation and visual odometry with deep feature reconstruction[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2018:340-349. [70] LI R,WANG S,LONG Z,et al.Undeepvo:monocular visual odometry through unsupervised deep learning[C]//2018 IEEE International Conference on Robotics and Automation(ICRA),2018:7286-7291. [71] BABU V M,DAS K,MAJUMDAR A,et al.Undemon:unsupervised deep network for depth and ego-motion estimation[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS),2018:1082-1088. [72] LIU Q,LI R,HU H,et al.Using unsupervised deep learning technique for monocular visual odometry[J].IEEE Access,2019,7:18076-18088. [73] BIAN J W,ZHAN H,WANG N,et al.Unsupervised scale-consistent depth learning from video[J].International Journal of Computer Vision,2021,129(9):2548-2564. [74] ALMALIOGLU Y,SANTAMARIA-NAVARRO A,MORRELL B,et al.Unsupervised deep persistent monocular visual odome- try and depth estimation in extreme environments[C]//2021 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS),2020:3534-3541. [75] YIN Z,SHI J.Geonet:unsupervised learning of dense depth,optical flow and camera pose[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2018:1983-1992. [76] ZOU Y,JI P,TRAN Q H,et al.Learning monocular visual odometry via self-supervised long-term modeling[C]//European Conference on Computer Vision.Cham:Springer,2020:710-727. [77] XUE F,WANG X,LI S,et al.Beyond tracking:selecting memory and refining poses for deep visual odometry[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2019:8575-8583. [78] LI S,WANG X,CAO Y,et al.Self-supervised deep visual odometry with online adaptation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2020:6339-6348. [79] LI S,WU X,CAO Y,et al.Generalizing to the open world:deep visual odometry with online adaptation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2021:13184-13193. [80] PRASAD V,DAS D,BHOWMICK B.Epipolar geometry based learning of multi-view depth and ego-motion from monocular sequences[C]//Proceedings of the 11th Indian Conference on Computer Vision,Graphics and Image Processing,2018:1-9. [81] SHEN T,LUO Z,ZHOU L,et al.Beyond photometric loss for self-supervised ego-motion estimation[C]//2019 International Conference on Robotics and Automation(ICRA),2019:6359-6365. [82] GAO X S,HOU X R,TANG J,et al.Complete solution classification for the perspective-three-point problem[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2003,25(8):930-943. [83] RADWAN N,VALADA A,BURGARD W.Vlocnet++:deep multitask learning for semantic visual localization and odometry[J].IEEE Robotics and Automation Letters,2018,3(4):4407-4414. [84] LI Y,USHIKU Y,HARADA T.Pose graph optimization for unsupervised monocular visual odometry[C]//2019 International Conference on Robotics and Automation(ICRA),2019. [85] BARNES D,MADDERN W,PASCOE G,et al.Driven to distraction:self-supervised distractor learning for robust monocular visual odometry in urban environments[C]//2018 IEEE International Conference on Robotics and Automation,2018:1894-1900. [86] WAGSTAFF B,PERETROUKHIN V,KELLY J.Self-supervised deep pose corrections for robust visual odometry[C]//2020 IEEE International Conference on Robotics and Automation(ICRA),2020:2331-2337. [87] GEIGER A,LENZ P,STILLER C,et al.Vision meets robotics:the kitti dataset[J].The International Journal of Robotics Research,2013,32(11):1231-1237. [88] BURRI M,NIKOLIC J,GOHL P,et al.The EuRoC micro aerial vehicle datasets[J].The International Journal of Robotics Research,2016,35(10):1157-1163. [89] STURM J,ENGELHARD N,ENDRES F,et al.A benchmark for the evaluation of RGB-D SLAM systems[C]//2012 IEEE/RSJ International Conference on Intelligent Robots and Systems,2012:573-580. [90] CORDTS M,OMRAN M,RAMOS S,et al.The cityscapes dataset for semantic urban scene understanding[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2016:3213-3223. [91] SHAMWELL E J,LINDGREN K,LEUNG S,et al.Unsupervised deep visual-inertial odometry with online error correction for RGB-D imagery[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2019,42(10):2478-2493. [92] ESFAHANI M A,WANG H,WU K,et al.AbolDeepIO:a novel deep inertial odometry network for autonomous vehicles[J].IEEE Transactions on Intelligent Transportation Systems,2019,21(5):1941-1950. |
[1] | GAO Guangshang. Survey on Attention Mechanisms in Deep Learning Recommendation Models [J]. Computer Engineering and Applications, 2022, 58(9): 9-18. |
[2] | JI Meng, HE Qinglong. AdaSVRG: Accelerating SVRG by Adaptive Learning Rate [J]. Computer Engineering and Applications, 2022, 58(9): 83-90. |
[3] | LUO Xianglong, GUO Huang, LIAO Cong, HAN Jing, WANG Lixin. Spatiotemporal Short-Term Traffic Flow Prediction Based on Broad Learning System [J]. Computer Engineering and Applications, 2022, 58(9): 181-186. |
[4] | Alim Samat, Sirajahmat Ruzmamat, Maihefureti, Aishan Wumaier, Wushuer Silamu, Turgun Ebrayim. Research on Sentence Length Sensitivity in Neural Network Machine Translation [J]. Computer Engineering and Applications, 2022, 58(9): 195-200. |
[5] | CHEN Yixiao, Alifu·Kuerban, LIN Wenlong, YUAN Xu. CA-YOLOv5 for Crowded Pedestrian Detection [J]. Computer Engineering and Applications, 2022, 58(9): 238-245. |
[6] | FANG Yiqiu, LU Zhuang, GE Junwei. Forecasting Stock Prices with Combined RMSE Loss LSTM-CNN Model [J]. Computer Engineering and Applications, 2022, 58(9): 294-302. |
[7] | SHI Jie, YUAN Chenxiang, DING Fei, KONG Weixiang. Survey of Building Target Detection in SAR Images [J]. Computer Engineering and Applications, 2022, 58(8): 58-66. |
[8] | XIONG Fengguang, ZHANG Xin, HAN Xie, KUANG Liqun, LIU Huanle, JIA Jionghao. Research on Improved Semantic Segmentation of Remote Sensing [J]. Computer Engineering and Applications, 2022, 58(8): 185-190. |
[9] | YANG Jinfan, WANG Xiaoqiang, LIN Hao, LI Leixiao, YANG Yanyan, LI Kecen, GAO Jing. Review of One-Stage Vehicle Detection Algorithms Based on Deep Learning [J]. Computer Engineering and Applications, 2022, 58(7): 55-67. |
[10] | WANG Bin, LI Xin. Research on Multi-Source Domain Adaptive Algorithm Integrating Dynamic Residuals [J]. Computer Engineering and Applications, 2022, 58(7): 162-166. |
[11] | TAN Shuqiu, TANG Guofang, TU Yuanya, ZHANG Jianxun, GE Panjie. Classroom Monitoring Students Abnormal Behavior Detection System [J]. Computer Engineering and Applications, 2022, 58(7): 176-184. |
[12] | ZHANG Meiyu, LIU Yuehui, HOU Xianghui, QIN Xujia. Automatic Coloring Method for Gray Image Based on Convolutional Network [J]. Computer Engineering and Applications, 2022, 58(7): 229-236. |
[13] | ZHANG Zhuangzhuang, QU Licheng, LI Xiang, ZHANG Minghao, LI Zhaolu. Traffic Flow Prediction with Missing Data Based on Spatial-Temporal Convolutional Neural Networks [J]. Computer Engineering and Applications, 2022, 58(7): 259-265. |
[14] | XU Jie, ZHU Yukun, XING Chunxiao. Research on Financial Trading Algorithm Based on Deep Reinforcement Learning [J]. Computer Engineering and Applications, 2022, 58(7): 276-285. |
[15] | WANG Jing, WANG Kai, YAN Yingjian. Research on Side Channel Attack Technology Based on Conditional Generation Against Network [J]. Computer Engineering and Applications, 2022, 58(6): 110-117. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||