关键词: 强化学习, Q（λ）学习, 函数逼近, 最小二乘, 倒立摆

Abstract: The problem of slow convergence speed and low efficiency of experience exploitation in classical Q（λ） learning is analyzed.And then the Least-Squares approximation model of the state-action pair’s value function is constructed according to current and previous experience.A set of linear equations is derived，which is satisfied by the weight vector of function approximator on a set of bases.Thus the fast and practical Least-Squares Q（λ） algorithm and improved recursive algorithm are proposed.The experiment of inverted pendulum demonstrates that these algorithms can effectively improve convergence speed and the efficiency of experience exploitation.

Key words: reinforcement learning, Q（λ） learning, function approximation, Least-Squares, inverted pendulum