计算机工程与应用 ›› 2008, Vol. 44 ›› Issue (15): 4-7.

• 博士论坛 • 上一篇    下一篇

局部合作多智能体Q-学习研究

刘 亮,李龙澍   

  1. 安徽大学 计算智能与信号处理教育部重点实验室,合肥 230039
  • 收稿日期:2007-10-22 修回日期:2008-01-30 出版日期:2008-05-21 发布日期:2008-05-21
  • 通讯作者: 刘 亮

Research on regional cooperative multi-agent Q-learning

LIU Liang,LI Long-shu
  

  1. Key Lab of IC & SP at Anhui University,Ministry of Education,Hefei 230039,China
  • Received:2007-10-22 Revised:2008-01-30 Online:2008-05-21 Published:2008-05-21
  • Contact: LIU Liang

摘要: 强化学习在多Agent系统中面对的最大问题就是随着Agent数量的增加而导致的状态和动作空间的指数增长以及随之而来的缓慢的学习效率。采用了一种局部合作的Q-学习方法,只有在Agent之间有明确协作时才考察联合动作,否则,就只进行简单的个体Agent的Q-学习,从而使的学习时所要考察的状态动作对值大大减少。最后算法在捕食者-猎物的追逐问题和机器人足球仿真2D上的实验结果,与常用的多Agent强化学习技术相比有更好的效能。

关键词: 多Agent系统, 强化学习, Q-学习, 局部合作

Abstract: Reinforcement learning in Multi-Agent Systems suffers from the fact that both the state and the action space scale exponentially with the number of Agents,which also lead to low learning speed.In this paper,the authors investigate a regional cooperative of the Q-function by only considering the joint actions in those states in which coordination is actually requires.In all other states Single-Agent Q-learning is applies.This offers a compact state-action value representation,without compromising much in terms of solution quality.The authors have performed experiments in the predator-prey domain and robocup-simulation 2D which is the ideal testing platform of Multi-Agent Systems and compared this algorithm to other Multi-Agent reinforcement learning algorithms with promising results.

Key words: Multi-Agent Systems(MAS), reinforcement learning, Q-learning, regional cooperative