SLAM System Based on Depth Estimation Network SS-Net

doi:10.3778/j.issn.1002-8331.2106-0191

Abstract

Abstract: The monocular SLAM system cannot satisfy high-precision positioning in the existing SLAM method. Therefore, a SLAM system based on depth estimation network is proposed. This system integrates the sobel-boundary-induced and scene-aggregated network（SS-Net） system on the traditional ORB-SLAM system, and rely on monocular to achieve accuracy localization. The SS-Net uses for depth estimation considers the important role of the different depth relationship and boundary in depth prediction. Based on boundary-induced and scene-aggregated network（BS-Net）, SS-Net incorporates an edge detection（ED） block, and a stripe refinement（SR） block. The SS-Net network can consider the deep correlation between different regions, extract important edges, and integrate network features below different levels, it can process a single frame of image, and then obtain depth estimate of the entire sequence. Experiments on NYUD v2 and TUM datasets show that SS-Net depth prediction has higher accuracy, and proves that the performance of SLAM system based on SS-Net is better than the original system.

Key words: depth estimation, simultaneous localization and mapping（SLAM）, deep learning, indoor positioning, supervised network

摘要： 现有的SLAM方案中，单目SLAM系统无法满足高精度定位。因此提出了一种基于深度估计网络的SLAM系统。此系统在ORB-SLAM的系统上，融合了Sobel边界引导和场景聚合网络（sobel-boundary-induced and scene-aggregated network，SS-Net）的系统，仅依靠单目实现精准定位。SS-Net考虑了不同区域的深度关系和边界在深度预测中的重要特征。基于边界引导和场景聚合网络（boundary-induced and scene-aggregated network，BS-Net），SS-Net提出了边界提取模块（edge detection，ED），改进了图像细化模块（stripe refinement，SR）。SS-Net网络能够考虑不同区域之间的深度相关性，提取重要的边缘，并融合不同层次下面的网络特征，可以处理单帧图像，从而获得整个序列的深度估计。在NYUD v2和TUM数据集上的大量实验表明，SS-Net深度预测有较高的准确性，并且证明了基于SS-Net的SLAM系统比原系统更优秀。

关键词: 深度估计, 同时定位与地图绘制（SLAM）, 深度学习, 室内定位, 有监督网络

WANG Heng, WU Bo, WANG Zhenming, YU Jianfeng. SLAM System Based on Depth Estimation Network SS-Net[J]. Computer Engineering and Applications, 2022, 58(24): 265-275.

王恒, 吴波, 王振明, 于剑峰. 基于深度估计网络SS-Net的SLAM系统[J]. 计算机工程与应用, 2022, 58(24): 265-275.

References

[1] SMITH R C.On the representation and estimation of spatial uncertainty[J].The International Journal of Robotics Research，1986，5（4）.
[2] DAVISON A J，REID I D，MOLTON N D，et al.MonoSLAM：real-time single camera SLAM[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2007，29（6）：1052-1067.
[3] KLEIN G，MURRAY D.Parallel tracking and mapping for small AR workspaces[C]//6th IEEE and ACM International Symposium on Mixed and Augmented Reality，2007：225-234.
[4] RAUL M A，MONTIEL J M M，TARDOS J D.ORB-SLAM：a versatile and accurate monocular SLAM system[J].IEEE Transactions on Robotics，2015，31（5）：1147-1163.
[5] EIGEN D，PUHRSCH C，FERGUS R.Depth map prediction from a single image using a multi-scale deep network[J].arXiv：1406.2283，2014.
[6] HU J，OZAY M，ZHANG Y，et al.Revisiting single image depth estimation：toward higher resolution maps with accurate object boundaries[C]//2019 IEEE Winter Conference on Applications of Computer Vision，2019.
[7] CHEN X T，CHEN X J，ZHA Z J.Structure-aware residual pyramid network for monocular depth estimation[C]//Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence，2019.
[8] LI J，YUCE C，KLEIN R，et al.A two-streamed network for estimating fine-scaled depth maps from single RGB images[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：3392-3400.
[9] LEE J H，KIM C S.Monocular depth estimation using relative depth maps[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019.
[10] XUE F，CAO J，ZHOU Y B，et al.Boundary-induced and scene-aggregated network for monocular depth prediction[J].Pattern Recognition，2021，115：107901.
[11] FISCHLER M A，BOLLES R C.Random sample consensus：a paradigm for model fitting with applications to image analysis and automated cartography[J].Communications of the ACM，1981，24（6）：381-395.
[12] LAINA I，RUPPRECHT C，BELAGIANNIS V，et al.Deeper depth prediction with fully convolutional residual networks[C]//Proceedings of 2016 Fourth International Conference on 3D Vision，2016：239-248.
[13] XU D，RICCI E，OIUAMG W，et al.Multiscale continuous crfs as sequential deep networks for monocular depth estimation[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition，2017：161-169.
[14] RAMAMONJISOA M，LEPETIT V.Sharpnet：fast and accurate recovery of occluding contours in monocular depth estimation[C]//2019 IEEE/CVF International Conference on Computer Vision Workshop，2019.
[15] YU F，KOLTUN V.Multi-scale context aggregation by dilated convolutions[J].arXiv：1511.07122，2015.
[16] WANG C，CHEN J.Multi-branch input structure for pyramid scene parsing network[C]//Lecture Notes in Electrical Engineering，2020：724-732.
[17] ZHOU Y，FU X.Research on the combination of improved sobel operator and ant colony algorithm for defect detection[C]//MATEC Web of Conferences，2021.
[18] LOWE D G.Distinctive image features from scale-invariant keypoints[J].International Journal of Computer Vision，2004，60（2）：91-110.
[19] RUBLEE E，RABAUD V，KONOLIGE K，et al.ORB：an efficient alternative to SIFT or SURF[C]//Proceedings of the IEEE International Conference on Computer Vision，2011：2564-2571.
[20] ROSTEN E，DRUMMOND T.Machine learning for high-speed corner detection[C]//European Conference on Computer Vision，2006.
[21] CALONDER M，LEPETIT V，STRECHA C，et al.BRIEF：binary robust independent elementary features[C]//European Conference on Computer Vision，2010.
[22] EIGEN D，FERGUS R.Predicting depth，surface normal and semantic labels with a common multi-scale convolutional architecture[C]//Proceedings of the IEEE International Conference on Computer Vision，2015：2650-2658.
[23] RUSSAKOVSKY O，DENG J，SU H，et al.Imagenet large scale visual recognition challenge[J].International Journal of Computer Vision，2015，115（3）：211-252.
[24] SAXENA A，MIN S，NG A Y.Make3d：learning 3d scene structure from a single still image[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2009，31（5）：824-840.
[25] KARSCH K，LIU C，KANG S B.Depth transfer：depth extraction from video using non-parametric sampling[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2014，36（11）：2144-2158.
[26] LIU M，SALZMANN M，HE X.Discrete-continuous depth estimation from a single image[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition，2014.
[27] DHARMASIRI T，SPEK A，DRUMMOND T.Joint prediction of depths，normals and surface curvature from RGB images using CNNs[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition，2017：1505-1512.
[28] LEE J H，HEO M，KIM K R，et al.Single-image depth estimation based on fourier domain analysis[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition，2018.
[29] XU D，OUYANG W，WANG X，et al.Pad-net：multi-tasks guided prediction-and-distillation network for simultaneous depth estimation and scene parsing[C]//31st IEEE/CVF Conference on Computer Vision and Pattern Recognition，2018：675-684.
[30] FU H，GONG M，WANG C，et al.Deep ordinal regression network for monocular depth estimation[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition，2018：2002-2011.
[31] YIN W，LIU Y，SHEN C，et al.Enforcing geometric constraints of virtual normal for depth prediction[C]//2019 IEEE/CVF International Conference on Computer Vision，2019.
[32] SWAMI K，BONDADA P V，BAJPAI K.ACED：accurate and edge-consistent monocular depth estimation[C]//2020 IEEE International Conference on Image Processing，2020.