关键词: 无人机, 航迹规划, 强化学习, 最小二乘法, Q学习, 连续状态空间

Abstract: Traditional reinforcement learning methods, in which the state space is discretized, can’t ensure the trajectory accuracy in complex flight applications. The route planning method with continuous state space based on Least-Squares Policy Iteration（LSPI） is presented in this paper. The approximate function is used to represent value function to ensure the trajectory accuracy without space discretization. By offline policy generation based on samples, the policy is evaluated and improved by LSPI directly. Compared with Q-learning, simulation results show that the trajectory planned by LSPI is smoother and more conducive to the actual flight of aircraft.

Key words: unmanned aerial vehicle, route planning, reinforcement learning, least squares method, Q-learning, continuous state spaces