关键词: 强化学习, 状态估计, 深度Q网络, 双深度Q网络

Abstract:

In the exploration problem of deep reinforcement learning, it has to make decisions based on the external reward given by the environment. However, in the sparse reward environment, no information can be acquired in the early stages, and it is difficult to dynamically adjust the exploration strategy with the acquired infomation in the latter stages. In order to alleviate this problem, prioritized state estimation method is proposed, which sets prioritized value to the state when it is accessed, and stores it in the experience buffer with external reward to guide the strategic direction of exploration. Combined with DDQN（Double Deep Q Network） and priortized experience replay, a comparative experiment is conducted in MountainCar classic control problem in OpenAI Gym and FreeWay game in Atari 2600. The results show that the method has better learning perfomance and achieves higher average score in sparse reward environment.

Key words: reinfocement learning, state estimation, Deep Q Network（DQN）, Double Deep Q Network（DDQN）