Abstract: Many multi-agent Q-learning problems can not be solved because of the number of joint actions is exponential in the number of agents.Based on the study of the cooperation in MAS in reinforcement learning and on the basis of the research in the system logic，this paper puts forward the hierarchical regional cooperation reinforcement learning based on learning process.By studying the knowledge of Agent reinforcement learning and improving the multi-Agent study efficiency，the performance of the regional cooperation reinforcement learning is further enhanced，combining with the mission action based on joint action and potential field model so as to solve the dimensional disaster in state space of reinforcement learning.This algorithm is used in a subtask of robot soccer and its effectiveness is validated by experiments.

Key words: Multi-Agent Systems（MAS）, regional cooperative, Q-learning, process stratification

摘要： 多智能体Q-学习问题往往因为联合动作的个数指数级增长而变得无法解决。从研究分层强化学习入手，通过对强化学习中合作MAS的研究，在基于系统工作逻辑的研究基础上，提出了基于学习过程分层的局部合作强化学习，通过对独立Agent强化学习的知识考察，改进多Agent系统学习的效率，进一步提高了局部合作强化学习的效能。从而解决强化学习中的状态空间的维数灾难，并通过仿真足球的2vs1防守证明了算法的有效性。

关键词: 多智能体系统, 局部合作, Q-学习, 过程分层