Abstract: Reinforcement learning is one of most important machine learning methods.In order to solve the problem of slow convergence speed and the error of value function in reinforcement learning systems，a multi-step Temporal Difference（TD（λ）） learning algorithm using Recursive Least-Squares（RSL） methods（RLS-TD（λ）） is proposed.The proposed algorithm is based on RLS-TD（0），its convergence is proved，and its formula of error estimation is obtained.The experiment on maze problem demons-
trates that the algorithm can speed up the convergence of the learning process compared with RLS-TD（0），and improve the learning precision compared with TD（λ）.