Comprehensive Review of ROV Underwater Obstacle Detection and Avoidance Technology

doi:10.3778/j.issn.1002-8331.2312-0206

Abstract

Abstract: This paper provides a comprehensive review of the technological advancements in underwater obstacle detection and avoidance techniques for remotely operated vehicles (ROV). The research focuses on sonar systems, optical systems, and their integration with machine learning and artificial intelligence algorithms, analyzing how these technologies enhance the autonomy, efficiency, and safety of underwater operations. Despite significant achievements in environmental adaptability and obstacle detection accuracy achieved by sonar and optical systems, challenges remain in real-time identification of dynamic obstacles and adaptation to complex environments. Furthermore, the potential and challenges of machine learning and artificial intelligence technologies in enhancing ROV’s autonomous obstacle avoidance capability are discussed, highlighting the importance of these technologies in future ROV operations. This research provides new theoretical perspectives and practical applications for deep-sea exploration and marine science.

Key words: underwater obstacle detection, autonomous obstacle avoidance, sonar systems, optical systems, machine learning and artificial intelligence

摘要： 全面回顾了远程操作车（ROV）在水下障碍物检测和避障技术方面的技术进展。研究集中于声呐系统、光学系统及其与机器学习和人工智能算法的结合，分析了这些技术如何提高水下作业的自主性、效率和安全性。尽管声纳和光学系统在环境适应性和障碍物检测精度方面已取得显著成果，但动态障碍物实时识别和复杂环境适应性的挑战仍待克服。此外，探讨了机器学习和人工智能技术在增强ROV自主避障能力方面的潜力和挑战，指出了这些技术在未来ROV操作中的重要性。该研究为深海探索和海洋科学提供了新的理论视角和应用实践。

关键词: 水下障碍物检测, 自主避障, 声纳系统, 光学系统, 机器学习与人工智能

LI Minggui, ZHOU Huanyin, GONG Liwen. Comprehensive Review of ROV Underwater Obstacle Detection and Avoidance Technology[J]. Computer Engineering and Applications, 2024, 60(17): 34-47.

李明桂, 周焕银, 龚利文. ROV水下障碍物检测和避障技术的应用综述[J]. 计算机工程与应用, 2024, 60(17): 34-47.

References

[1] LI X, SHANG M, QIN H, et al. Fast accurate fish detection and recognition of underwater images with fast R-CNN[C]//Proceedings of the OCEANS 2015, 2015: 1-5.
[2] WU Y, TA X, XIAO R, et al. Survey of underwater robot positioning navigation[J]. Applied Ocean Research, 2019, 90: 101845.
[3] MA Q. Improving SAR target recognition performance using multiple preprocessing techniques[J]. Computational Intelligence and Neuroscience, 2021: 1-8.
[4] LIN C, WANG H, YUAN J, et al. An improved recurrent neural network for unmanned underwater vehicle online obstacle avoidance[J]. Ocean Engineering, 2019, 189: 106327.
[5] ZHANG L, LI C, SUN H. Object detection/tracking toward underwater photographs by remotely operated vehicles (ROVs)[J]. Future Generation Computer Systems, 2022, 126: 163-168.
[6] YUAN X, GUO L, LUO C, et al. A survey of target detection and recognition methods in underwater turbid areas[J]. Applied Sciences, 2022, 12(10): 4898.
[7] LUO X, CHEN L, ZHOU H, et al. A survey of underwater acoustic target recognition methods based on machine learning[J]. Journal of Marine Science and Engineering, 2023, 11(2): 384.
[8] DAKHIL R A, KHAYEAT A R H. Review on deep learning technique for underwater object detection[J]. arXiv:2209.
10151, 2022.
[9] ATANER E, ?ZDE? B, ?ZTüRK G, et al. Deep learning methods in unmanned underwater vehicles[J]. Avrupa Bilim ve Teknoloji Dergisi, 2020: 345-350.
[10] FU T, ZHENG W, LI Z W, et al. mproving the detection accuracy of underwater obstacles based on a novel combined method of support vector regression and gravity gradient[J]. Remote Sensing, 2023, 15: 2188.
[11] ANTONELLI G, CHIAVERINI S, FINOTELLO R, et al. Real-time path planning and obstacle avoidance for RAIS: an autonomous underwater vehicle[J]. IEEE Journal of Oceanic Engineering, 2001, 26(2): 216-227.
[12] HONG S. A robust underwater object detection method using forward-looking sonar images[J]. Remote Sensing Letters, 2023, 14(5): 442-449.
[13] ZHAO L, YUN Q, YUAN F, et al. YOLOv7-CHS: an emerging model for underwater object detection[J]. Journal of Marine Science and Engineering, 2023, 11(10): 1949.
[14] REHMAN F U, ANDERLINI E, THOMAS G. Assessment of collision avoidance strategies for an underwater transportation system[J]. Journal of Robotics and Control, 2021, 2(5): 385-394.
[15] LIU Z S, DING T F, WANG G. ROV based underwater blurred image restoration[J]. Journal of Ocean University of Qingdao, 2003, 2: 85-88.
[16] 陆地, 陈伟, 魏庆宇. 基于多深度对抗网络的 ROV 水下目标检测[J]. 计算机时代, 2023(4): 5-10.
LU D, CHEN W, WEI Q Y. Underwater image enhancement based on multi-scale adversarial network[J]. Computer Era, 2023(4): 5-10.
[17] WANG X, FAN X, SHI P, et al. An overview of key SLAM technologies for underwater scenes[J]. Remote Sensing, 2023, 15(10): 2496.
[18] VUK?I? M, JOSIPOVI? S, ?ORI? A, et al. Underwater ROV as inspection and development platform[J]. Transactions on Maritime Science, 2017, 6(1): 48-54.
[19] OKEREKE C E, MOHAMAD M M, WAHAB N H A, et al. An overview of machine learning techniques in local path planning for autonomous underwater vehicles[J]. IEEE Access, 2023, 11: 24894-24907.
[20] MANDAL R, CONNOLLY M R, THOMAS A, et al. Assessing fish abundance from underwater video using deep neural networks[C]//Proceedings of the 2018 International Joint Conference on Neural Networks, 2018: 1-6.
[21] SUNG M, YU S C, GIRDHAR Y. Vision based real-time fish detection using convolutional neural network[C]//Proceedings of the OCEANS 2017, 2017: 1-6.
[22] 黄珍伟, 陈伟, 王文杰, 等. 基于改进 RetinaNet 网络的水下机器人目标检测与实验[J]. 计算机工程与科学, 2024, 46(2): 264-271.
HUANG Z W, CHEN W, WANG W J, et al. Underwater vehicle target detection and experiment based on improved RetinaNet network[J]. Computer Engineering and Science, 2024, 46(2): 264-271.
[23] 韩冲, 苏涛, 谢基榕. 基于 OpenCV 的水下机器人单目定位技术研究与仿真[J]. 计算机测量与控制, 2017, 25(12): 219-223.
HAN C, SU T, XIE J R. Research and simulation of underwater vehicle monocular positioning technology based on OpenCV[J]. Computer Measurement and Control, 2017, 25(12): 219-223.
[24] ZHAO Z, LIU Y, SUN X, et al. Composited FishNet: fish detection and species recognition from low-quality underwater videos[J]. IEEE Transactions on Image Processing, 2021, 30: 4719-4734.
[25] NEUPANE D, SEOK J. A review on deep learning-based approaches for automatic sonar target recognition[J]. Electronics, 2020, 9(11): 1972.
[26] CAO X, REN L, SUN C. Research on obstacle detection and avoidance of autonomous underwater vehicle based on forward-looking sonar[J]. IEEE Transactions on Neural Networks and Learning Systems, 2023, 34(11): 9198-9208.
[27] 汤寓麟, 王黎明, 余德荧, 等. 基于 CSLS-CycleGAN 的侧扫声纳水下目标图像样本扩增法[J]. 系统工程与电子技术, 2024, 46(5): 1514-1524.
TANG Y L, WANG L M, YU D Y, et al. CSLS-CycleGAN based side-scan sonar sample augmentation method for
underwater target image[J]. Journal of systems Engineering & Electronics, 2024, 46(5): 1514-1524.
[28] WANG J, FENG C, WANG L, et al. Detection of weak and small targets in forward-looking sonar image using multi-branch shuttle neural network[J]. IEEE Sensors Journal, 2022, 22(7): 6772-6783.
[29] QIN R, ZHAO X, ZHU W, et al. Multiple receptive field network (MRF-Net) for autonomous underwater vehicle fishing net detection using forward-looking sonar images[J]. Sensors, 2021, 21(6): 1933.
[30] SZYMAK P, PISKUR P, NAUS K. The effectiveness of using a pretrained deep learning neural networks for object classification in underwater video[J]. Remote Sensing, 2020, 12(18): 3020.
[31] BABAEIZADEH M, FROSIO I, TYREE S, et al. GA3C: GPU-based A3C for deep reinforcement learning[J]. arXiv:1611.06256, 2016.
[32] FORTUNATO M, AZAR M G, PIOT B, et al. Noisy networks for exploration[J]. arXiv:1706.10295, 2017.
[33] PRACZYK T. Neural collision avoidance system for biomimetic autonomous underwater vehicle[J]. Soft Computing, 2020, 24(2): 1315-1333.
[34] ZHANG B, JI D, LIU S, et al. Autonomous underwater vehicle navigation: a review[J]. Ocean Engineering, 2023, 273: 113861.
[35] PETILLOT Y, RUIZ I T, LANE D M. Underwater vehicle obstacle avoidance and path planning using a multi-beam forward looking sonar[J]. IEEE Journal of Oceanic Engineering, 2001, 26(2): 240-251.
[36] SAVKIN A V, VERMA S C, ANSTEE S. Optimal navigation of an unmanned surface vehicle and an autonomous underwater vehicle collaborating for reliable acoustic communication with collision avoidance[J]. Drones, 2022, 6(1): 27.
[37] GE H L, DAI Y W, ZHU Z Y, et al. Improved YOLOv7 Acousto-optic fusion method for underwater target detection[J]. Ship Science and Technology, 2023, 45(12): 122-127.
[38] GANESAN V, CHITRE M, BREKKE E. Robust underwater obstacle detection and collision avoidance[J]. Autonomous Robots, 2016, 40: 1165-1185.
[39] ARAIN B, MCCOOL C, RIGBY P, et al. Improving underwater obstacle detection using semantic image segmentation[C]//Proceedings of the 2019 International Conference on Robotics and Automation , 2019: 9271-9277.
[40] JIA R, LV B, CHEN J, et al. Underwater object detection in marine ranching based on improved YOLOv8[J]. Journal of Marine Science and Engineering, 2023, 12(1): 55.
[41] ?MIERZCHALSKI R, KAPCZY?SKI M. Autonomous control of the underwater remotely operated vehicle in collision situation with stationary obstacle[J]. Polish Maritime Research, 2022, 29(4): 45-55.
[42] FAYAZ S, PARAH S A, QURESHI G J, et al. Intelligent underwater object detection and image restoration for autonomous underwater vehicles[J]. IEEE Transactions on Vehicular Technology, 2023, 73(2): 1726-1735.
[43] SUN M, ZHAO W, SONG G, et al. DDPG-based decision-making strategy of adaptive cruising for heavy vehicles considering stability[J]. IEEE Access, 2020, 8: 59225-59246.
[44] 王大方. 基于深度强化学习的机器人导航研究[D]. 北京: 中国矿业大学, 2019.
WANG D F. Research on robot navigation based on deep reinforcement learning[D]. Beijing: China University of Mining and Technology, 2019.
[45] MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518: 529-533.
[46] RAY J P. Development of underwater robots for under water inspection and cleaning applications[D]. Lappeenranta: Lappeenranta University of Technology, 2023.
[47] 董豪, 杨静, 李少波, 等. 基于深度强化学习的机器人运动控制研究进展[J]. 控制与决策, 2022, 37(2): 278-292.
DONG H, YANG J, LI S B, et al. Research progress of robot motion control based on deep reinforcement learning[J]. Control and Decision, 2022, 37(2): 278-292.
[48] GREFSTAD ?. Development of an obstacle detection and avoidance system for ROV[D]. South-Trondelag: Norges Teknisk-Naturvitenskapelige Universitet, 2018.
[49] 石立伟, 鲍鹏筱, 陈占, 等. 水下球形机器人视觉防碰撞编队策略[J]. 兵工自动化, 2022, 41(10): 79-83.
SHI L W, BAO P X, CHEN Z, et al. Visual anti-collision formation strategy for underwater spherical robots[J]. Ordnance Industry Automation, 2022, 41(10): 79-83.
[50] SIV?EV S, ROSSI M, COLEMAN J, et al. Collision detection for underwater ROV manipulator systems[J]. Sensors, 2018, 18(4): 1117.
[51] WEN C, HORIUCHI T K. The curved openspace algorithm and a spike-latency model for sonar-based obstacle avoidance[J]. Frontiers in Neurorobotics, 2022, 16: 850013.
[52] 张俞鹏. 基于多波束前视声呐的水下障碍物检测及避障算法研究[D]. 上海: 上海大学, 2021.
ZHANG Y P. Research on underwater obstacle detection and obstacle avoidance algorithm based on multi-beam forward looking sonar[D]. Shanghai: Shanghai University, 2021.
[53] RUIZ I T, PETILLOT Y, LANE D, et al. Tracking objects in underwater multibeam sonar images[J]. IEE Colloquium on Motion Analysis and Tracking, 1999, 103: 1-7.
[54] GREFSTAD ?, SCHJ?LBERG I. Navigation and collision avoidance of underwater vehicles using sonar data[C]//Proceedings of the 2018 IEEE/OES Autonomous Underwater Vehicle Workshop, 2018: 1-6.
[55] CYR R. Terrain obstacle avoidance sonar for autonomous vehicles[C]//Proceedings of the OCEANS’87, 1987: 134-138.
[56] FEDOROVICH A J, KLAMO T J, KRAGELUND S P, et al. Head-on object size determination using a forward looking sonar for obstacle avoidance[C]//Proceedings of the OCEANS 2022, 2022: 1-8.
[57] OCHOA E, GRACIAS N R, ISTENI? K, et al. Collision detection and avoidance for underwater vehicles using omnidirectional vision[J]. Sensors, 2022, 22(14): 5354.
[58] LEI Y, TIAN T, JIANG B, et al. Research and application of the obstacle avoidance system for high-speed railway tunnel lining inspection train based on integrated 3D LiDAR and 2D camera machine vision technology[J]. Applied Sciences, 2023, 13(13): 7689.
[59] ALHATTAB A Y, ABIDIN Z B Z, FAIZABADI A R, et al. Integration of stereo vision and MOOS-IvP for enhanced obstacle detection and navigation in unmanned surface vehicles[J]. IEEE Access, 2023, 11: 128932-128956.
[60] YESIN K, NELSON B J, PAPANIKOLOPOULOS N P, et al. Active video system for a miniature reconnaissance robot[C]//Proceedings of the IEEE International Conference on Robotics and Automation, 2000: 3919-3924.
[61] OCHOA E, GRACIAS N, ISTENI? K, et al. Allowing untrained scientists to safely pilot ROVs: early collision detection and avoidance using omnidirectional vision[C]//Proceedings of the Global Oceans 2020, 2020: 1-7.
[62] MA J. Dynamic obstacle avoidance and trajectory planning of five-axis redundant industrial manipulator[J]. Journal of Vibroengineering, 2023, 25(4): 778-791.
[63] PORTUGAL M, MARTA A C. Optimal multi-sensor obstacle detection system for small fixed-wing UAVs[J]. Modelling, 2023, 5(1): 16-36.
[64] PUTRA A A G J, TRILAKSONO B R, HIDAYAT E M I. Hybrid autonomous underwater glider (HAUG) obstacle detection and avoidance[C]//Proceedings of the 2021 IEEE 11th International Conference on System Engineering and Technology, 2021: 118-123.
[65] FANG M C, WANG S M, WANG M C, et al. Applying the self-tuning fuzzy control with the image detection technique on the obstacle-avoidance for autonomous underwater vehicles[J]. Ocean Engineering, 2015, 93: 11-24.
[66] CHU Z, WANG F, LEI T, et al. Path planning based on deep reinforcement learning for autonomous underwater vehicles under ocean current disturbance[J]. IEEE Transactions on Intelligent Vehicles, 2022, 8(1): 108-120.
[67] HADI B, KHOSRAVI A, SARHADI P. Deep reinforcement learning for adaptive path planning and control of an autonomous underwater vehicle[J]. Applied Ocean Research, 2022, 129: 103326.
[68] GAO W, HAN M, WANG Z, et al. Research on method of collision avoidance planning for uuv based on deep reinforcement learning[J]. Journal of Marine Science and Engineering, 2023, 11(12): 2245.
[69] CHENG C, SHA Q, HE B, et al. Path planning and obstacle avoidance for AUV: a review[J]. Ocean Engineering, 2021, 235: 109355.
[70] YUAN J, WANG H, ZHANG H, et al. AUV obstacle avoidance planning based on deep reinforcement learning[J]. Journal of Marine Science and Engineering, 2021, 9(11): 1166.
[71] JIANG D, YUAN M, XIONG J, et al. Obstacle avoidance USV in multi-static obstacle environments based on a deep reinforcement learning approach[J]. Measurement and Control, 2024, 57(4): 415-427.
[72] LI J, CHAVEZ-GALAVIZ J, AZIZZADENESHELI K, et al. Dynamic obstacle avoidance for usvs using cross-domain deep reinforcement learning and neural network model predictive controller[J]. Sensors, 2023, 23(7): 3572.
[73] ZHAO G, YING F, PANG Z, et al. Robotic motion planning with obstacle avoidance based on hierarchical deep reinforcement learning[C]//Proceedings of the 2023 3rd International Conference on Robotics and Control Engineering, 2023: 108-112.
[74] WANG P, LIU R, TIAN X, et al. Obstacle avoidance for environmentally-driven USVs based on deep reinforcement learning in large-scale uncertain environments[J]. Ocean Engineering, 2023, 270: 113670.
[75] CHEN P, PEI J, LU W, et al. A deep reinforcement learning based method for real-time path planning and dynamic obstacle avoidance[J]. Neurocomputing, 2022, 497: 64-75.
[76] KOT R. Review of collision avoidance and path planning algorithms used in autonomous underwater vehicles[J]. Electronics, 2022, 11(15): 2301.
[77] 倪生科, 刘正江, 蔡垚, 等. 基于混合遗传算法的船舶避碰路径规划[J]. 上海海事大学学报, 2019, 40(1): 21-26.
NI S K, LIU Z J, CAI Y, et al. Ship collision avoidance path planning based on hybrid genetic algorithm [J]. Journal of Shanghai Maritime University, 2019, 40(1): 21-26.
[78] QIN H, SHAO S, WANG T, et al. Review of autonomous path planning algorithms for mobile robots[J]. Drones, 2023, 7(3): 211.
[79] SGORBISSA A. Integrated robot planning, path following, and obstacle avoidance in two and three dimensions: wheeled robots, underwater vehicles, and multicopters[J]. The International Journal of Robotics Research, 2019, 38(7): 853-876.
[80] LIN Y, SARIPALLI S. Sampling-based path planning for UAV collision avoidance[J]. IEEE Transactions on Intelligent Transportation Systems, 2017, 18(11): 3179-3192.
[81] RAFAI A N A, ADZHAR N, JAINI N I. A review on path planning and obstacle avoidance algorithms for autonomous mobile robots[J]. Journal of Robotics, 2022, 2022: 1-14.
[82] SAHOO S P, DAS B, PATI B B, et al. Hybrid path planning using a bionic-inspired optimization algorithm for autonomous underwater vehicles[J]. Journal of Marine Science and Engineering, 2023, 11(4): 761.
[83] KOT R. Review of obstacle detection systems for collision avoidance of autonomous underwater vehicles tested in a real environment[J]. Electronics, 2022, 11(21): 3615.
[84] LV Q Y. Research and analysis of obstacle avoidance path planning for autonomous vehicles[C]//Proceedings of the 2021 International Conference on Signal Processing and Machine Learning, 2021: 225-228.
[85] SUN B, ZHANG W, LI S, et al. Energy optimised D* AUV path planning with obstacle avoidance and ocean current environment[J]. The Journal of Navigation, 2022, 75(3): 685-703.
[86] FU J, LV T, LI B. Underwater submarine path planning based on artificial potential field ant colony algorithm and velocity obstacle method[J]. Sensors, 2022, 22(10): 3652.
[87] HAN G, QI X, PENG Y, et al. Early warning obstacle avoidance-enabled path planning for multi-AUV-based maritime transportation systems[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 24(2): 2656-2667.
[88] JIN H, JI H, YAN F. An effective obstacle avoidance and motion planning design for underwater telescopic arm robots based on a tent chaotic dung beetle algorithm[J]. Electronics, 2023, 12(19): 4128.
[89] WANG C, YANG X, LI H. Improved Q-learning applied to dynamic obstacle avoidance and path planning[J]. IEEE Access, 2022, 10: 92879-92888.
[90] XU H L, LUAN K, JIA B Q, et al. Cooperative obstacle avoidance method of multiple unmanned underwater vehicles based on improved artificial potential field method[C]//Proceedings of the 2023 35th Chinese Control and Decision Conference, 2023: 2950-2955.
[91] ZHU L, XIE H, SONG Z. Robot obstacle avoidance path planning based on improved artificial potential field[C]//Proceedings of the 3rd International Conference on Control and Intelligent Robotics, 2023: 563-569.
[92] YU H, NING L. Coordinated obstacle avoidance of multi-AUV based on improved artificial potential field method and consistency protocol[J]. Journal of Marine Science and Engineering, 2023, 11(6): 1157.
[93] YAN Z, ZHAO L, WANG Y, et al. Path planning of AUV for obstacle avoidance with improved artificial potential field[C]//Proceedings of the Annual Conference of the IEEE Industrial Electronics Society, 2023: 1-5.
[94] HAN H, XI Z, CHENG J, et al. Obstacle avoidance based on deep reinforcement learning and artificial potential field[C]//Proceedings of the 2023 9th International Conference on Control, Automation and Robotics, 2023: 215-220.
[95] ZHANG W, XU G, SONG Y, et al. An obstacle avoidance strategy for complex obstacles based on artificial potential field method[J]. Journal of Field Robotics, 2023, 40(5): 1231-1244.
[96] WANG S M, FANG M C, WANG C N. Vertical obstacle avoidance and navigation of autonomous underwater vehicles with H∞ controller and the artificial potential field method[J]. Journal of Navigation, 2019, 72(1): 207-228.
[97] DEMIM F, ROUIGUEB A, BELAIDI H, et al. Smooth sliding mode control based technique of an autonomous underwater vehicle based localization using obstacle avoidance strategy[C]//Proceedings of the International Conference on Informatics in Control, Automation and Robotics, 2023: 529-537.
[98] CHEN S, LIN T, JHENG K, et al. Application of fuzzy theory and optimum computing to the obstacle avoidance control of unmanned underwater vehicles[J]. Applied Sciences, 2020, 10(17): 6105.
[99] MHDAWI A A K, WRIGHT N, BENSON S, et al. CART-Ⅱ: development of collision avoidance robotic tether with soft sensing capabilities for underwater nuclear inspection vehicles[C]//Proceedings of the 2023 9th International Conference on Automation, Robotics and Applications, 2023: 15-22.
[100] DREWS P, HERNáNDEZ E, ELFES A, et al. Real-time monocular obstacle avoidance using underwater dark channel prior[C]//Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2016: 4672-4677.
[101] TENG B, ZHAO H. Underwater target recognition methods based on the framework of deep learning: a survey[J]. International Journal of Advanced Robotic Systems, 2020, 17(6): 1729881420976307.