Abstract: An improved DDPG flight track planning algorithm is proposed, aiming at the problem of high processing complexity of intelligent algorithm due to unknown threats in UAV flight process which leads to the difficulty of real-time flight track planning, and long training time by adjusting the parameters of DDPG algorithm in deep reinforcement learning. The flight scene model is established under the background of UAV track planning. According to the flight dynamics theory, the action space is built. On the basis of the non-sparse idea, the reward function is designed. Combined with the artificial bee colony algorithm, the updating mechanism of the model parameters of DDPG algorithm is improved, and the network model is trained to achieve the flight track decision-making of UAV. Simulation results show that the overall training time of the proposed algorithm is only 1.98 times of the average training time of the prototype algorithm, the training efficiency is improved, and the cost of time is reduced. Besides, under the condition of satisfy real time flight, the proposed algorithm can meet the demand of UAV track quality, and provides a new idea for promoting the practical application of deep reinforcement learning in flight track planning.

Key words: deep deterministic policy gradient algorithm, unmanned aerial vehicle, track planning, deep reinforcement learning, artificial bee colony algorithm

摘要： 针对无人机飞行过程存在未知威胁使智能算法处理复杂度高，导致航迹实时规划困难，以及深度强化学习中调整DDPG算法参数，存在时间成本过高的问题，提出一种改进DDPG航迹规划算法。围绕无人机航迹规划问题，构建飞行场景模型，根据飞行动力学理论，搭建动作空间，依据非稀疏化思想，设计奖励函数，结合人工蜂群算法，改进DDPG算法模型参数的更新机制，训练网络模型，实现无人机航迹决策控制。仿真结果表明，所提算法整体训练时长仅为原型算法单次平均训练时长的1.98倍，大幅度提升网络训练效率，降低时间成本，且在满足飞行实时性情况下，符合无人机航迹质量需求，为推动深度强化学习在航迹规划的实际应用提供新思路。

关键词: 深度确定性策略梯度算法, 无人机, 航迹规划, 深度强化学习, 人工蜂群算法