关键词: 强化学习, 人工势场, 贪婪策略, 移动机器人, 路径规划

Abstract:

In order to solve the problem of slow convergence, large number of iterations and unstable convergence when the mobile robot plans a path in unknown environment by using traditional reinforcement learning algorithm, an improved Q-learning algorithm is proposed. The artificial potential field is used to initialize the state value to make the larger state value closer to the target position, so as to guide the agent to move towards the target position. In the early stage of the algorithm, a large number of invalid iterations due to the environment exploration are reduced. The improved [ε]-greedy strategy is employed as agent’s action selection, the greedy factor [ε] is adjusted dynamically according to the convergence degree of the algorithm so as to balance the relationship between exploration and exploitation better and accelerate the convergence rate of the algorithm and improve the stability of the convergence results. The proposed algorithm is simulated and verified in the grid map based on Python Tkinter standardized library. Simulation results show that, compared with the traditional Q-learning algorithm, the planning time of improved Q-learning algorithm is reduced by 85.1%, the number of iterations is reduced by 74.7% before convergence, and the stability of the convergence results is greatly improved.

Key words: reinforcement learning, artificial potential field, greedy strategy, mobile robots, path planning