Computer Engineering and Applications ›› 2010, Vol. 46 ›› Issue (21): 43-46.DOI: 10.3778/j.issn.1002-8331.2010.21.012

• 研究、探讨 • Previous Articles     Next Articles

Multi-Agent Q-learning based on quantum theory and ant colony algorithm

MENG Xiang-ping1,WANG Sheng-bin2   

  1. 1.Department of Electrical Engineering,Changchun Institute of Technology,Changchun 130012,China
    2.Department of Computer Engineering,Northeast Dianli University,Jilin 132012,China
  • Received:2009-01-21 Revised:2009-04-24 Online:2010-07-21 Published:2010-07-21
  • Contact: MENG Xiang-ping

基于量子理论及蚁群算法的多Agent Q学习

孟祥萍1,王圣镔2   

  1. 1.长春工程学院 电气与信息学院,长春 130012
    2.东北电力大学 信息工程学院,吉林 132012
  • 通讯作者: 孟祥萍

Abstract: Due to the interactions among the Agents in the cooperative multi-Agent systems,multi-Agent learning problem complexity can rise rapidly with the number of Agents or their behavioral sophistication.In order to converge to desirable equilibrium,Agents generally require sufficient exploration of strategy space and coordinate their policies to achieve optimal equilibrium.A novel cooperative multi-Agent learning method is proposed based on quantum theory,ant algorithm and Q-learning.First,this method not only coordinates Agents’ behaviors using quantum entanglement and helps Agents make action selection under quantum superposition,but also adopts Grover’s searching algorithm which can probe the action,speed up learning.Second,according to ant algorithm,footmark thought is presented so that Agents can be indirectly enforced to communicate with others.At last,the theory analysis and result of experiment both demonstrate that the improved Q-learning is feasible and increases the learning efficiency.

Key words: multi-Agent system, cooperative, quantum computing, Q-learning, equilibrium, ant colony algorithm

摘要: 针对多Agent协作强化学习中存在的行为和状态维数灾问题,以及行为选择上存在多个均衡解,为了收敛到最佳均衡解需要搜索策略空间和协调策略选择问题,提出了一种新颖的基于量子理论和蚁群算法的多Agent协作学习算法。新算法首先借签了量子计算理论,将多Agent的行为和状态空间通过量子叠加态表示,利用量子纠缠态来协调策略选择,利用概率振幅进行动作探索,加快学习速度。其次,根据蚁群算法,提出“脚印”思想来间接增强Agent之间的交互。最后,对新算法的理论分析和实验结果都证明了改进的Q学习是可行的,并且可以有效地提高学习效率。

关键词: 多Agent系统, 协作, 量子计算, Q-学习, 均衡解, 蚁群算法

CLC Number: