多死角场景中机器人深度强化学习导航

doi:10.3778/j.issn.1002-8331.2503-0329

计算机工程与应用 ›› 2026, Vol. 62 ›› Issue (8): 168-175.DOI: 10.3778/j.issn.1002-8331.2503-0329

多死角场景中机器人深度强化学习导航

曹青跃1，2，王雅栋1，2，王庆1，2，张羽佳1，2，阳媛1，2+

1.东南大学仪器科学与工程学院，南京 210096
2.微惯性仪表与先进导航技术教育部重点实验室，南京 210096
+ 通信作者 E-mail：yangyuancsi@163.com

收稿日期:2025-03-27 修回日期:2025-05-09 在线发布日期:2026-04-15 出版日期:2026-04-15
基金资助:
国家自然科学基金（42074039）；中交建筑集团科技项目（8522008786）。

Robot Deep Reinforcement Learning Navigation in Multi-Corner Scenarios

CAO Qingyue1，2, WANG Yadong1，2, WANG Qing1，2, ZHANG Yujia1，2, YANG Yuan1，2+

1.School of Instrument Science and Engineering, Southeast University, Nanjing 210096, China
2.Ministry of Education Key Laboratory of Micro-Inertial Instrument and Advanced Navigation Technology, Nanjing 210096, China
+ Corresponding author E-mail：yangyuancsi@163.com

Received:2025-03-27 Revised:2025-05-09 Online:2026-04-15 Published:2026-04-15

摘要/Abstract

摘要： 针对强化学习导航存在训练效率低、稳定性差及在多死角场景中目标遮挡时导航性能差的问题，对此提出了一种融合专家经验与混合奖励机制的深度强化学习导航方法。筛选高质量专家经验并基于此预训练了行为克隆模型初始化策略，用于提高训练效率；构建了包含死角避免约束的稠密奖励函数，实现目标牵引与死角避让之间的平衡；采用标准化折扣回报方式降低不同轨迹的回报方差以提高训练稳定性。仿真实验表明所提出的方法在随机起止点测试中取得了91.3%导航成功率，在固定起止点测试中取得了95%导航成功率且耗时最短。结果表明，该方法能够灵活调整目标牵引和死角避让策略，有效提高多死角场景中机器人自主导航水平。

关键词: 深度强化学习（DRL）, 自主导航, 目标遮挡, 死角避免

Abstract: Aiming at the problems of low training efficiency, poor stability, and poor navigation performance in multi-corner scenarios with target occlusion in reinforcement learning-based navigation methods, a deep reinforcement learning navigation method that integrates expert experience and a hybrid reward mechanism is proposed. A behavior cloning model is pretrained based on the filtered high-quality expert experience to initialize the policy and improve training efficiency. A dense reward function with corner avoidance constraints is constructed to balance target traction and corner avoidance. Normalized discounted returns are used to reduce return variance across different trajectories and enhance training stability. Simulation experiments show that the proposed method achieves a 91.3% navigation success rate in the random start stop test, and a 95% success rate in the fixed start stop test with the shortest time consumption. The results show that this method can flexibly adjust the target traction and corner avoidance strategies, and effectively improve the robot autonomous navigation level in multi-corner scenarios.

Key words: deep reinforcement learning (DRL), autonomous navigation, target occlusion, corner avoidance

曹青跃, 王雅栋, 王庆, 张羽佳, 阳媛. 多死角场景中机器人深度强化学习导航[J]. 计算机工程与应用, 2026, 62(8): 168-175.

CAO Qingyue, WANG Yadong, WANG Qing, ZHANG Yujia, YANG Yuan. Robot Deep Reinforcement Learning Navigation in Multi-Corner Scenarios[J]. Computer Engineering and Applications, 2026, 62(8): 168-175.

参考文献

[1] QIN H W，SHAO S L，WANG T，et al.Review of autonomous path planning algorithms for mobile robots[J].Drones，2023，7（3）：211.
[2] 尹洪玉，吴宇，梁天骄.固定翼无人机巡逻覆盖协同路径规划方法[J].航空学报，2024，45（6）：323-337.
YIN H Y，WU Y，LIANG T J.Cooperative path planning for patrol coverage of fixed wing UAV[J].Acta Aeronautica et Astronautica Sinica，2024，45（6）：323-337.
[3] LOQUERCIO A，MAQUEDA A I，DEL-BLANCO C R，et al.DroNet：learning to fly by driving[J].IEEE Robotics and Automation Letters，2018，3（2）：1088-1095.
[4] POKLE A，MARTíN-MARTíN R，GOEBEL P，et al.Deep local trajectory replanning and control for robot navigation[C]//Proceedings of the 2019 International Conference on Robotics and Automation.Piscataway：IEEE，2019：5815-5822.
[5] DONG L，HE Z C，SONG C W，et al.A review of mobile robot motion planning methods：from classical motion planning workflows to reinforcement learning-based architectures[J].Journal of Systems Engineering and Electronics，2023，34（2）：439-459.
[6] KALEEM Z，KHALID W，MUQAIBEL A，et al.Learning-aided UAV 3D placement and power allocation for sum-capacity enhancement under varying altitudes[J].IEEE Communications Letters，2022，26（7）：1633-1637.
[7] ZHANG C Y，LI Z K，HE C L，et al.Deep reinforcement learning based trajectory design and resource allocation for UAV-assisted communications[J].IEEE Communications Letters，2023，27（9）：2398-2402.
[8] ROGHAIR J，NIARAKI A，KO K，et al.A vision based deep reinforcement learning algorithm for UAV obstacle avoidance[C]//Proceedings of the 2021 Intelligent Systems and Applications Conference.Cham：Springer，2022：115-128.
[9] KALIDAS A P，JOSHUA C J，MD A Q，et al.Deep reinforcement learning for vision?based navigation of UAVs in avoiding stationary and mobile obstacles[J].Drones，2023，7（4）：245.
[10] ZIELI?SKI P，MARKOWSKA-KACZMAR U.3D robotic navigation using a vision-based deep reinforcement learning model[J].Applied Soft Computing，2021，110：107602.
[11] ZHU W，HAYASHIBE M.A hierarchical deep reinforcement learning framework with high efficiency and generalization for fast and safe navigation[J].IEEE Transactions on Industrial Electronics，2023，70（5）：4962-4971.
[12] DAWOOD M，DENGLER N，DE HEUVEL J，et al.Handling sparse rewards in reinforcement learning using model predictive control[C]//Proceedings of the IEEE International Conference on Robotics and Automation.Piscataway：IEEE，2023：879-885.
[13] PFEIFFER M，SHUKLA S，TURCHETTA M，et al.Reinforced imitation：sample efficient deep reinforcement learning for mapless navigation by leveraging prior demonstrations[J].IEEE Robotics and Automation Letters，2018，3（4）：4423-4430.
[14] M?BIUS M，FISCHER K，KALLFASS D，et al.Curriculum interleaved online behavior cloning for complex reinforcement learning applications[C]//Proceedings of the 2024 Winter Simulation Conference.Piscataway：IEEE，2025：1990-2001.
[15] WANG H C，HUANG S C，HUANG P J，et al.Curriculum reinforcement learning from avoiding collisions to navigating among movable obstacles in diverse environments[J].IEEE Robotics and Automation Letters，2023，8（5）：2740-2747.
[16] SONG Y L，LIN H，KAUFMANN E，et al.Autonomous overtaking in gran turismo sport using curriculum reinforcement learning[C]//Proceedings of the 2021 IEEE International Conference on Robotics and Automation.Piscataway：IEEE，2021：9403-9409.
[17] 顾金浩，况立群，韩慧妍，等.动态环境下共融机器人深度强化学习导航算法[J].计算机工程与应用，2025，61（4）：90-98.
GU J H，KUANG L Q，HAN H Y，et al.Deep reinforcement learning navigation algorithm for coexisting-cooperative-cognitive robots in dynamic environment[J].Computer Engineering and Applications，2025，61（4）：90-98.
[18] NAIK A，WAN Y，TOMAR M，et al.Reward centering[EB/OL].[2025-03-09].https：//arxiv.org/abs/2405.09999.
[19] CIMURS R，SUH I H，LEE J H.Goal-driven autonomous exploration through deep reinforcement learning[J].IEEE Robotics and Automation Letters，2022，7（2）：730-737.
[20] 王童，李骜，宋海荦，等.基于分层深度强化学习的移动机器人导航方法[J].控制与决策，2022，37（11）：2799-2807.
WANG T，LI A，SONG H L，et al.Navigation method for mobile robot based on hierarchical deep reinforcement learning[J].Control and Decision，2022，37（11）：2799-2807.
[21] 朱威，洪力栋，施海东，等.结合优势结构和最小目标Q值的深度强化学习导航算法[J].控制理论与应用，2024，41（4）：716-728.
ZHU W，HONG L D，SHI H D，et al.Deep reinforcement learning navigation algorithm combining advantage structure and minimum target Q-value[J].Control Theory & Applications，2024，41（4）：716-728.
[22] TAO W Z，HUANG H L.Fast and robust training and deployment of deep reinforcement learning based navigation policy[C]//Proceedings of the 2023 IEEE International Conference on Unmanned Systems.Piscataway：IEEE，2023：1581-1586.

多死角场景中机器人深度强化学习导航

Robot Deep Reinforcement Learning Navigation in Multi-Corner Scenarios

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 9

编辑推荐

Metrics

[1]	张玮奇, 赵一凡, 赵亮, 辛美婷. 无地图环境下的四足机器人鲁棒运动规划方法[J]. 计算机工程与应用, 2025, 61(24): 371-378.
[2]	李成健, 宋姝谊, 粟宇, 陈智斌. 深度强化学习求解多目标旅行商问题的研究综述[J]. 计算机工程与应用, 2025, 61(12): 28-44.
[3]	李兴洲, 李艳武, 谢辉. 基于CNN的深度强化学习算法求解柔性作业车间调度问题[J]. 计算机工程与应用, 2024, 60(17): 312-320.
[4]	刘子毅, 李铁军, 孙晨昭, 史家浩, 刘今越. 基于BIM的建筑机器人自主导航策略优化研究[J]. 计算机工程与应用, 2022, 58(15): 302-308.
[5]	邹承明，明成龙，李成龙. 融合相关粒子滤波目标跟踪算法[J]. 计算机工程与应用, 2020, 56(7): 184-192.
[6]	卫保国，赵思同，文绪亮. 基于局部直方图的多区域目标跟踪算法[J]. 计算机工程与应用, 2018, 54(14): 26-33.
[7]	杜志远，刘会杰. 导航星座中的载波相位星间精密测距技术[J]. 计算机工程与应用, 2011, 47(13): 139-142.
[8]	冯柯，陈临强. 户外环境下抗遮挡的运动目标跟踪方法[J]. 计算机工程与应用, 2011, 47(11): 187-189.
[9]	秦政,丁福光,边信黔. 强化学习在移动机器人自主导航中的应用[J]. 计算机工程与应用, 2007, 43(18): 215-217.